Generative Models and Their Evaluation

Definition A generative model specifies a probability distribution over observations, a probability mass function or density $p_{\boldsymbol{\theta}}(\mathbf{x})$ with parameters $\boldsymbol{\theta}$, or $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{c})$ when it is conditioned on a prompt, a class or another input $\mathbf{c}$. It is fitted to training observations so that it approximates the unknown data distribution $p_{\text{data}}$. Some generative models can evaluate $p_{\boldsymbol{\theta}}(\mathbf{x})$ for a given $\mathbf{x}$, some can only draw samples, and some can do both. A discriminative classifier, by contrast, models only the label given the input, $P(Y = y \mid \mathbf{x})$, and says nothing about how the inputs themselves are distributed.

Definition Order the coordinates of an observation as $x_1, \dots, x_T$ and write $x_{<t} = (x_1, \dots, x_{t-1})$. Every outcome with positive probability satisfies

where the factor for $t = 1$ is $p(x_1)$. The derivation is the multiplication rule $P(A \cap B) = P(A)P(B \mid A)$ applied repeatedly. It gives $p(x_1, \dots, x_T) = p(x_1, \dots, x_{T-1})\, p(x_T \mid x_{<T})$, and induction on $T$ gives the product. The identity holds for every distribution and every order of the coordinates. An autoregressive model parameterizes each conditional $p_{\boldsymbol{\theta}}(x_t \mid x_{<t})$ directly, for example with a decoder-only transformer. Because each conditional adds to $1$ over $x_t$, the product is then automatically a valid joint distribution. For independent observations $\mathbf{x}^{(1)}, \dots, \mathbf{x}^{(n)}$ the likelihood is the product of their probabilities, so the log likelihood is the sum of $\ln p_{\boldsymbol{\theta}}(x^{(i)}_t \mid x^{(i)}_{<t})$ over all observations $i$ and positions $t$. Maximizing it is the same as minimizing the negative log likelihood. A reported average must say whether it is per observation, per token or per coordinate, because these denominators give different numbers. For continuous coordinates the factors are densities, and a density value is not the probability of one exact point.

Definition A latent-variable model introduces an unobserved variable $\mathbf{z}$ with a prior distribution $p(\mathbf{z})$ and a conditional distribution $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ of the observation. Its distribution of observations is the marginal

with an integral in place of the sum when $\mathbf{z}$ is continuous. Sampling draws $\mathbf{z}$ from the prior and then $\mathbf{x}$ from $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$. A Gaussian mixture is an example with a discrete $\mathbf{z}$. A variational autoencoder, or VAE, is a latent-variable model with a continuous latent vector, usually with the standard normal prior $N(\mathbf{0}, I)$, whose decoder is a neural network that outputs the parameters of $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$. With such a decoder the marginal and the posterior $p_{\boldsymbol{\theta}}(\mathbf{z} \mid \mathbf{x})$ generally have no closed form, so a second network, the encoder, outputs the parameters of an approximate posterior $q_{\boldsymbol{\phi}}(\mathbf{z} \mid \mathbf{x})$. Encoder and decoder are trained together by maximizing the evidence lower bound of the next card. Unlike an ordinary autoencoder, which maps each input to one code and back, a VAE defines a probability model, and it generates new observations by decoding draws from the prior.

Definition For discrete distributions $q$ and $r$ on the same set, the Kullback-Leibler divergence is

with the sum over the values $z$ where $q(z) > 0$. It is finite when $r(z) > 0$ wherever $q(z) > 0$, it is never negative, it is $0$ exactly when $q = r$, and it is not symmetric in $q$ and $r$. Now take a latent-variable model with prior $p(\mathbf{z})$ and likelihood $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$, an observation $\mathbf{x}$ with evidence $p_{\boldsymbol{\theta}}(\mathbf{x}) > 0$, and posterior $p_{\boldsymbol{\theta}}(\mathbf{z} \mid \mathbf{x}) = p(\mathbf{z})\,p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z}) / p_{\boldsymbol{\theta}}(\mathbf{x})$. For a distribution $q$ over the latent values that is positive only where the posterior is positive, the evidence lower bound is

and it satisfies the identity

with integrals and densities in place of sums when $\mathbf{z}$ is continuous. The last divergence is never negative, so $\operatorname{ELBO}(q) \le \ln p_{\boldsymbol{\theta}}(\mathbf{x})$, with equality exactly when $q$ is the posterior. A VAE uses $q = q_{\boldsymbol{\phi}}(\mathbf{z} \mid \mathbf{x})$ for each training observation and maximizes the sum of the ELBOs over $\boldsymbol{\theta}$ and $\boldsymbol{\phi}$. The first term rewards reconstructions that the stated observation model finds probable, and the second keeps the approximate posterior close to the prior.

Definition A generative adversarial network, or GAN, pairs a generator $G_{\boldsymbol{\theta}}$, which maps a noise vector $\mathbf{z}$ drawn from a fixed distribution $p(\mathbf{z})$ to a synthetic observation $G_{\boldsymbol{\theta}}(\mathbf{z})$, with a discriminator $D_{\boldsymbol{\phi}}$, which outputs a number in $(0, 1)$ read as the probability that its input came from the training data. The original objective is the game $\min_{\boldsymbol{\theta}} \max_{\boldsymbol{\phi}} V(\boldsymbol{\theta}, \boldsymbol{\phi})$ with the value function

where the first expectation is over training observations and the second over the noise. It is trained by alternating gradient steps, the discriminator ascending and the generator descending, and in practice the generator often maximizes $\ln D_{\boldsymbol{\phi}}(G_{\boldsymbol{\theta}}(\mathbf{z}))$ instead because that gives stronger gradients early in training. A standard GAN provides no normalized likelihood $p_{\boldsymbol{\theta}}(\mathbf{x})$, and the value of its loss depends on the current discriminator. In an idealized analysis with unlimited capacity and exact optimization, the game is solved when the generator’s distribution equals $p_{\text{data}}$, but finite networks trained by alternating steps need not reach that point. Mode collapse is the failure in which the generator produces only a limited part of the data distribution and misses other regions that carry substantial probability.

Definition A diffusion model defines a forward process that adds Gaussian noise to a training observation $\mathbf{x}_0$ over many steps and learns a reverse process that turns noise back into observations. In a common formulation the noisy version at step $t$ can be drawn directly as

where $\boldsymbol{\varepsilon}$ is drawn from $N(\mathbf{0}, I)$ independently of $\mathbf{x}_0$ and the schedule $\bar{\alpha}_t$ decreases from near $1$ at the first step to near $0$ at the last step $T$. A network $\boldsymbol{\varepsilon}_{\boldsymbol{\theta}}(\mathbf{x}_t, t)$ is trained to predict the added noise by minimizing the average of $\lVert \boldsymbol{\varepsilon} - \boldsymbol{\varepsilon}_{\boldsymbol{\theta}}(\mathbf{x}_t, t) \rVert_2^2$ over training observations, steps and noise draws. Sampling starts from pure noise $\mathbf{x}_T$ drawn from $N(\mathbf{0}, I)$ and applies learned reverse steps one after another. The objective, schedule and sampler differ between formulations. This noise-prediction loss is related to a variational bound, but with its usual weighting it is not the model’s log likelihood, and its value depends on the schedule and the weighting.