Generative Models: Quick recap

01 Latent variables and the ELBO

A generative model is a distribution $p_\theta(x)$ that we can sample; some families can also evaluate it exactly (autoregressive, flows), some only bound it (VAE, diffusion), some not at all (GAN).
Maximum likelihood minimizes the forward KL: $\mathbb E_{p_{\text{data}}}[-\log p_\theta] = H(p_{\text{data}}) + \mathrm{KL}(p_{\text{data}} \,\|\, p_\theta)$ . Forward KL is mode covering; reverse KL is mode seeking.
A latent variable model samples $z \sim p(z)$ , then $x \sim p_\theta(x \mid z)$ ; its evidence $p_\theta(x) = \int p(z)p_\theta(x \mid z)\,dz$ and its posterior are usually intractable.
For any $q$ : $\log p(x) = \mathrm{ELBO}(q) + \mathrm{KL}(q(z \mid x) \,\|\, p(z \mid x))$ , with $\mathrm{ELBO}(q) = \mathbb E_q[\log p_\theta(x \mid z)] - \mathrm{KL}(q(z \mid x) \,\|\, p(z))$ .
EM sets $q$ to the exact posterior (gap 0) and then maximizes over $\theta$ ; it never decreases $\log p(x)$ .
A VAE encoder has two gaps: the approximation gap (the family $\mathcal Q$ is too simple) and the amortization gap (the encoder misses the best member of $\mathcal Q$ ).
The IWAE bound $\mathcal L_K = \mathbb E[\log\frac1K\sum_k w_k]$ equals the ELBO at $K = 1$ , never decreases in $K$ , stays below $\log p(x)$ , and its gap shrinks like $\operatorname{Var}(w)/(2Kp(x)^2)$ .

02 Variational autoencoders

A VAE minimizes distortion $\mathbb E_q[-\log p_\theta(x \mid z)]$ plus rate $\mathrm{KL}(q_\phi(z \mid x) \,\|\, \mathcal N(0, I))$ , which is minus the ELBO, with a Gaussian encoder $q_\phi = \mathcal N(\mu_\phi(x), \operatorname{diag}\sigma_\phi^2(x))$ .
Reparameterization $z = \mu + \sigma \odot \varepsilon$ , $\varepsilon \sim \mathcal N(0, I)$ , gives the pathwise gradient; it is usually much less noisy than the score-function estimator, but needs a continuous $z$ and a differentiable $f$ .
$\mathrm{KL}(\mathcal N(\mu, \operatorname{diag}\sigma^2) \,\|\, \mathcal N(0, I)) = \tfrac12\sum_j(\mu_j^2 + \sigma_j^2 - 1 - \log\sigma_j^2)$ ; a dimension is free only at $\mu_j = 0$ , $\sigma_j = 1$ .
"Summed squared error $+\ \beta,$KL" is the ELBO of a Gaussian decoder with $\sigma^2 = \beta/2$ ; a mean-reduced MSE with a summed KL silently sets $\beta = n$ .
Posterior collapse ( $q(z \mid x) = p(z)$ ) can be the optimum of the ELBO; in a linear VAE, direction $i$ is active iff $\lambda_i > \beta\sigma^2$ .
The β-VAE minimizes $D + \beta R$ : larger β gives lower rate and higher distortion; it is a likelihood bound only for $\beta \ge 1$ .
VQ-VAE quantizes to the nearest code, passes gradients straight through, and has three loss terms: reconstruction, codebook and commitment; unused codes get no gradient and stay dead.

03 Autoregressive models

Chain rule: every joint distribution factorizes as $p(x) = \prod_{i=1}^{D} p(x_i \mid x_{<i})$ , for any order of the variables.
An autoregressive model is normalized when each factor is a normalized distribution and depends only on $x_{<i}$ ; a leak of $x_i$ into factor $i$ gives a near-zero training loss and an invalid model.
Training is maximum likelihood with teacher forcing: a sum of $D$ cross-entropy terms on true prefixes, computed in one masked pass. It minimizes $\mathrm{KL}(p_{\text{data}} \,\|\, q_\theta)$ .
Evaluating $\log p(x)$ costs one pass; sampling costs $D$ sequential passes (ancestral sampling).
Exposure bias is the mismatch between true prefixes in training and generated prefixes in sampling; the likelihood stays exact.
MADE masks: hidden units read units of degree $\le$ their own, outputs read only strictly lower degrees; the mask product is strictly lower triangular.
PixelCNN uses mask A (no center) in layer 1 and mask B (with center) after it; stacked masks leave a blind spot, $c' - c \le r - r'$ , removed by the two-stack design.
A transformer LM uses causal attention plus a one-token shift; past keys and values never change, so the KV cache is exact.

04 Normalizing flows

Change of variables: if $z = f(x)$ with $f$ a smooth bijection, then $p_X(x) = p_Z(f(x))\,|\det J_f(x)|$ ; where $f$ squeezes volume, the density of $x$ is high.
For a composition $f = f_K \circ \dots \circ f_1$ : $\log p_X(x) = \log p_Z(z_K) + \sum_k \log|\det J_{f_k}(z_{k-1})|$ ; in the sampling direction the log-determinants are subtracted.
An affine coupling layer $y_B = x_B \odot e^{s(x_A)} + t(x_A)$ has a triangular Jacobian, $\log|\det J| = \sum_j s_j(x_A)$ at cost $O(D)$ , and an exact inverse; $s$ and $t$ can be any networks.
Glow's $1 \times 1$ convolution adds $HW \log|\det W|$ , which costs $O(c)$ in the LU form.
MAF ( $u_i = (x_i - \mu_i(x_{<i}))e^{-\alpha_i(x_{<i})}$ ) evaluates densities in one pass and samples in $D$ passes; IAF is the reverse, and it gets $\log q$ of its own samples in one pass.
Continuous flows: $\frac{d}{dt}\log p_t(x(t)) = -\operatorname{tr}(\partial f/\partial x)$ ; Hutchinson's estimator $\mathbb E[\varepsilon^\top J \varepsilon] = \operatorname{tr} J$ needs one vector-Jacobian product per probe (FFJORD).
The probability-flow ODE of diffusion models and flow-matching models are continuous flows, so this formula gives their exact likelihood.
Flows keep the dimension, and their density is positive everywhere: separated modes need very large Jacobians.

05 GANs and Wasserstein GANs

The optimal discriminator is $D^* = p_{\text{data}}/(p_{\text{data}} + p_g)$ , so $D^*/(1 - D^*) = p_{\text{data}}/p_g$ : a perfect discriminator estimates the density ratio.
With $D^*$ , the GAN value is $C(G) = -\log 4 + 2\,\mathrm{JSD}(p_{\text{data}} \,\|\, p_g)$ , minimized only at $p_g = p_{\text{data}}$ .
The generator learns only through $\nabla_x D$ at its own samples; where $D$ is flat, it learns nothing.
The saturating generator loss has gradient $-\sigma(l)$ in the logit, which vanishes when $D$ rejects fakes; the non-saturating loss $-\log D$ has gradient $-\sigma(-l) \approx -1$ there.
For disjoint supports, JSD is $\log 2$ for every generator, so it gives no gradient; this is the normal case for thin, low-dimensional data.
Gradient play on a game can cycle: in the Dirac-GAN the Jacobian has eigenvalues $\pm i/2$ , simultaneous steps spiral out, alternating steps circle, and an R1 penalty converges.
The f-GAN bound $D_f(P \,\|\, Q) \ge \mathbb E_P[T] - \mathbb E_Q[f^*(T)]$ is tight at $T^* = f'(p/q)$ ; every such critic estimates a function of the density ratio.
$W_1(P, Q) = \sup_{\|f\|_L \le 1} \mathbb E_P[f] - \mathbb E_Q[f]$ ; it is continuous where JSD jumps, and the critic is kept Lipschitz by clipping, a gradient penalty, or spectral normalization.

06 EBMs, score matching, Langevin

An EBM is $p_\theta(x) = e^{-E_\theta(x)}/Z_\theta$ ; the partition function $Z_\theta$ is intractable, and only differences of energy matter.
$\nabla_\theta \log p_\theta(x) = -\nabla_\theta E_\theta(x) + \mathbb E_{p_\theta}[\nabla_\theta E_\theta]$ : the positive phase lowers the energy at the data, the negative phase raises it at model samples; for $E_\theta = \theta^\top\phi$ the optimum matches moments.
Langevin dynamics $x \leftarrow x - h\nabla E(x) + \sqrt{2h}\,\xi$ is noisy gradient descent; the continuous SDE has $p$ as stationary density (Fokker–Planck), and it needs only the score.
ULA with a finite step is biased: for $\mathcal N(\mu, \sigma^2)$ the stationary variance is $\sigma^2/(1 - h/(2\sigma^2))$ . MALA adds a Metropolis–Hastings step that removes the bias but needs the energy, not only the score.
Contrastive divergence starts short chains at the data (cheap, biased); persistent chains keep running between updates and stay closer to $p_\theta$ .
The score $\nabla_x\log p_\theta = -\nabla_x E_\theta$ does not contain $Z_\theta$ .
Denoising score matching regresses $s_\theta(x_0 + \sigma\varepsilon)$ on $-\varepsilon/\sigma$ ; it equals explicit score matching on $p_\sigma$ plus a constant, so its minimizer is $\nabla\log p_\sigma$ and its minimum is positive.
Tweedie: $\mathbb E[x_0 \mid x] = x + \sigma^2\nabla\log p_\sigma(x)$ . A single small noise level fails on separated modes (blind to weights, crossing time $\sim e^{\Delta E}$ ); annealing from large to small $\sigma$ fixes both.

07 Denoising diffusion (DDPM)

The forward process $x_t = \sqrt{\alpha_t}\,x_{t-1} + \sqrt{\beta_t}\,z_t$ has the closed form $x_t = \sqrt{\bar\alpha_t}\,x_0 + \sqrt{1 - \bar\alpha_t}\,\varepsilon$ with $\bar\alpha_t = \prod_{s \le t}(1 - \beta_s)$ , so training never runs the chain.
DDPM is a hierarchical latent variable model with a fixed encoder; it is trained with the ELBO of lesson 01.
The forward posterior $q(x_{t-1} \mid x_t, x_0)$ is Gaussian with variance $\tilde\beta_t = \frac{1 - \bar\alpha_{t-1}}{1 - \bar\alpha_t}\beta_t$ and mean $\tilde\mu_t = \frac{1}{\sqrt{\alpha_t}}\big(x_t - \frac{\beta_t}{\sqrt{1 - \bar\alpha_t}}\varepsilon\big)$ .
The negative ELBO is $L_T + \sum_{t \ge 2} L_{t-1} + L_0$ , with one Gaussian KL per step between the forward posterior and the learned reverse step.
With the ε-parameterization each $L_{t-1}$ is $w_t\,\|\varepsilon - \varepsilon_\theta(x_t, t)\|^2$ plus a constant; $L_{\text{simple}}$ sets all weights to 1, which gives better samples and a slightly worse likelihood.
Ancestral sampling: $x_{t-1} = \frac{1}{\sqrt{\alpha_t}}\big(x_t - \frac{\beta_t}{\sqrt{1 - \bar\alpha_t}}\varepsilon_\theta\big) + \sigma_t z$ , with $z = 0$ at the last step; $T$ network calls per sample.
$\sigma_t^2 = \tilde\beta_t$ is exact for a single data point and $\sigma_t^2 = \beta_t$ for unit-variance Gaussian data; with 1000 steps both give similar samples.
A Gaussian reverse step is accurate only for small forward steps; with few large steps the true reverse conditional can be multimodal.

08 Diffusion parameterizations

At one noise level $x_t = a_t x_0 + \sigma_t\varepsilon$ ; knowing $x_t$ and one of $\hat x_0$ , $\hat\varepsilon$ , $\hat v = a_t\hat\varepsilon - \sigma_t\hat x_0$ or the score gives all the others by linear maps (score $= -\hat\varepsilon/\sigma_t$ ).
In the VP case $(x_t, v)$ is $(x_0, \varepsilon)$ rotated by the angle $\phi$ with $a_t = \cos\phi$ , $\sigma_t = \sin\phi$ , so $x_0 = a_t x_t - \sigma_t v$ .
The squared-loss optimum is the conditional mean, and Tweedie's formula gives it: $\mathbb E[x_0 \mid x_t] = (x_t + \sigma_t^2\nabla\log p_t(x_t))/a_t$ . So all targets share one optimum.
Converting errors: $\varepsilon \to x_0$ multiplies by $\sigma_t/a_t$ (bad at high noise), $x_0 \to \varepsilon$ by $a_t/\sigma_t$ (bad at low noise), $v \to x_0$ or $\varepsilon$ by $\sigma_t$ or $a_t$ (never more than 1).
For one output, $\|x_0 - \hat x_0\|^2 = \|\varepsilon - \hat\varepsilon\|^2/\mathrm{SNR}$ and (VP) $\|v - \hat v\|^2 = (1 + 1/\mathrm{SNR})\|\varepsilon - \hat\varepsilon\|^2$ : the losses differ only by an SNR weight.
Every diffusion loss is $\tfrac12\int w(\lambda)\,\mathbb E\|\varepsilon - \hat\varepsilon\|^2 d\lambda$ over $\lambda = \log\mathrm{SNR}$ ; the ELBO has $w = 1$ , $L_{\text{simple}}$ has $w = p(\lambda)$ , and weights that decrease in $\lambda$ are ELBOs with noise augmentation.
The continuous-time ELBO depends on the schedule only through $\mathrm{SNR}_{\min}$ and $\mathrm{SNR}_{\max}$ ; the schedule still sets the training variance, the weight of $t$ -defined losses, and the sampler grid.
EDM: $c_{\text{skip}} = \sigma_{\text{data}}^2/(\sigma^2 + \sigma_{\text{data}}^2)$ , $c_{\text{out}} = \sigma\sigma_{\text{data}}/\sqrt{\sigma^2 + \sigma_{\text{data}}^2}$ , $c_{\text{in}} = 1/\sqrt{\sigma^2 + \sigma_{\text{data}}^2}$ give unit-variance inputs and targets; $\mathrm{SNR}(T) > 0$ leaks the image mean, fixed by zero terminal SNR with $v$ -prediction.

09 Reverse SDEs, PF-ODE and samplers

The forward process is an SDE $dx = f\,dt + g\,dw$ : VP $f = -\tfrac12\beta(t)x$ , $g = \sqrt{\beta(t)}$ (the limit of DDPM); VE $f = 0$ , $g = \sqrt{d\sigma^2/dt}$ ; EDM $\sigma(t) = t$ .
Reverse-time SDE (Anderson 1982): $dx = [f - g^2\nabla\log p_t]\,dt + g\,d\bar w$ , run from $t = T$ to $0$ , has the marginals $p_t$ ; ancestral DDPM sampling is its Euler–Maruyama discretization.
Probability-flow ODE (Song et al. 2021): $dx/dt = f - \tfrac12 g^2\nabla\log p_t$ has the same marginals (the Fokker–Planck equation is a continuity equation); in EDM coordinates $dx/d\sigma = (x - D(x;\sigma))/\sigma$ .
Reverse SDE and PF-ODE give the same distributions but different paths; the ODE is deterministic and invertible, the SDE's noise partly corrects earlier errors but needs more steps.
DDIM: $x_{t-1} = a_{t-1}\hat x_0 + \sqrt{\sigma_{t-1}^2 - \nu_t^2}\,\hat\varepsilon + \nu_t z$ ; every $\eta$ keeps the DDPM marginals, so one trained network serves all; $\eta = 1$ is ancestral DDPM with $\tilde\beta$ .
DDIM with $\eta = 0$ is Euler's method for the PF-ODE in $(\sigma_t/a_t,\ x_t/a_t)$ , so it is first order; Heun is second order and costs $2N - 1$ NFE for $N$ steps ending at $\sigma = 0$ .
Karras schedule: $\sigma_i = (\sigma_{\max}^{1/\rho} + \tfrac{i}{N-1}(\sigma_{\min}^{1/\rho} - \sigma_{\max}^{1/\rho}))^\rho$ ; $\rho = 7$ puts most levels at low noise, where the paths bend; compare samplers at equal NFE.

10 Flow matching and rectified flow

Flow matching learns the velocity $v_\theta(x, t)$ of an ODE from noise ( $t = 1$ ) to data ( $t = 0$ ) by regression on the conditional paths $x_t = (1 - t)x_0 + t\varepsilon$ with targets $\varepsilon - x_0$ ; no ODE is solved in training.
The marginal velocity $v(x, t) = \mathbb E[\varepsilon - x_0 \mid x_t = x]$ generates the marginal path (marginalization trick, proved with the continuity equation).
$L_{\text{CFM}} = L_{\text{FM}} + V$ with a constant $V \ge 0$ (the conditional variance of the target), so both losses have the same gradient; the CFM loss of a perfect model is $V$ , not 0.
For a Gaussian path $v = \dot a_t D + \dot\sigma_t\hat\varepsilon$ ; for the linear path $v = (x - D)/t$ , so velocity, denoiser and score are linear maps of each other, and diffusion's PF-ODE is the flow-matching ODE of its own path.
An Euler step of linear-path flow matching equals a DDIM step; one Euler step from $t = 1$ lands on the data mean, because the first velocity is an average direction.
Straight conditional paths give curved marginal paths that never cross; curvature is what costs sampling steps.
Rectified flow: the flow's own pairs $(Z_0, Z_1)$ have no higher convex transport cost; reflow retrains on them, the squared cost drops by $S + V$ , and paths become straighter (in 1D, straight after one round).
The time convention differs between papers: $t \mapsto 1 - t$ and $v \mapsto -v$ .

11 Guidance

Bayes' rule for scores: $\nabla \log p_t(x \mid y) = \nabla \log p_t(x) + \nabla \log p_t(y \mid x)$ at every noise level, with the noisy classifier $p_t(y \mid x_t) = \int p(y \mid x_0)\,p(x_0 \mid x_t)\,dx_0$ .
At high noise the noisy classifier is nearly flat; in the two-class toy its slope falls from $4.0$ (clean) to $0.10$ at $\bar\alpha = 0.01$ . A clean classifier used on $x_t$ is wrong.
Classifier guidance: $\tilde\varepsilon = \hat\varepsilon_\theta - s\,\sigma_t\,\nabla_{x_t}\log p_\phi(y \mid x_t)$ with a classifier trained on noisy inputs.
Classifier-free guidance: train one network with condition dropout (10–20%), sample with $\tilde\varepsilon = \hat\varepsilon_u + w(\hat\varepsilon_c - \hat\varepsilon_u)$ ; $w = 0$ unconditional, $w = 1$ conditional; Ho and Salimans' $w' = w - 1$ ; 2 NFE per guided step.
CFG equals the conditional model plus $(w - 1)$ times guidance with the implicit classifier $p_t(x \mid y)/p_t(x)$ .
Guided samples do not follow the tilted target $p(x)^{1-w}p(x \mid y)^w$ , because tilting and noising do not commute; ODE and SDE samplers give different results, and in a Gaussian model the ODE gives variance $v_c^w v_u^{1-w}$ .
Guidance raises precision and lowers recall; FID is best at a moderate $w$ . Limited-interval guidance (only middle noise levels) and autoguidance (a weaker copy of the same model) reduce the loss of diversity.

12 Few-step generation

Multimodal data bends PF-ODE paths: at high noise the denoiser predicts the mean of the modes, so one large step lands between them. One data point gives straight paths, and one Euler step is exact.
One Euler step from $\sigma$ to $0$ lands on the denoiser output $D(x; \sigma) = \mathbb E[x_0 \mid x]$ ; for data $\pm 1$ this is $\tanh(x/\sigma^2)$ .
Progressive distillation: the student target $\tilde x = \big(x_{t''} - (\sigma_{t''}/\sigma_t)x_t\big)/\big(a_{t''} - (\sigma_{t''}/\sigma_t)a_t\big)$ makes one student DDIM step equal two teacher steps; $\log_2(N/K)$ halving rounds reach $K$ steps.
Few-step students need $v$ - or $x_0$ -prediction: at $a_t = 0$ an $\varepsilon$ -output carries no information about $x_0$ , errors are multiplied by $\sigma_t/a_t$ , and the $\varepsilon$ -loss weight on $x_0$ is $\mathrm{SNR} = 0$ .
A consistency model learns $f(x_\sigma, \sigma) = x_{\sigma_{\min}}$ , the end of the PF-ODE path; $c_{\text{skip}}(\sigma_{\min}) = 1$ and $c_{\text{out}}(\sigma_{\min}) = 0$ enforce $f_\theta(x, \sigma_{\min}) = x$ and rule out constant solutions.
Zero consistency-distillation loss gives the teacher's consistency function up to $O(\Delta^p)$ (solver order $p$ ), so CD is capped by the teacher; consistency training uses the pair $(x_0 + \sigma_{n+1}z, x_0 + \sigma_n z)$ , an unbiased but noisy Euler step.
Regression-based few-step methods blur at 1–2 steps; adversarial and distribution-matching methods are sharp but mode-seeking and lose the seed-to-sample map. Always check diversity and calibration after distillation.

13 Latent diffusion and DiT

Latent diffusion trains an autoencoder once, runs diffusion on its latent $z = \mathcal E(x)$ of shape $(H/f) \times (W/f) \times c$ , and decodes once; the compression in values is $f^2 C/c$ (48 for $f = 8$ , $C = 3$ , $c = 4$ ).
Small $f$ keeps perceptual detail that the diffusion model must then learn; large $f$ loses information the decoder cannot restore; the decoder sets a ceiling on quality.
Latents with standard deviation $\sigma_z$ have effective SNR $\sigma_z^2\,\mathrm{SNR}(t)$ , a log-SNR shift of $2\ln\sigma_z$ ; Stable Diffusion's factor $0.18215 \approx 1/5.49$ makes them unit-variance.
A DiT with patch size $p$ has $T = (H/(fp))^2$ tokens; self-attention costs about $2T^2 d$ multiply-adds per block and the linear layers $12Td^2$ ; doubling $H$ or halving $p$ multiplies the attention term by 16.
Conditions enter by shape: maps by concatenation, global vectors by adaLN, token sequences by cross-attention (queries from the latent, keys and values from the text) or joint attention.
adaLN-Zero regresses shift, scale and gate from the conditioning vector, with the gate starting at 0, so every block is the identity at initialization and the gates learn first.
MM-DiT (Stable Diffusion 3) concatenates text and image tokens for one joint attention with separate weights per modality, so text and image read each other; the cost is $(N + M)^2 d$ .

14 Evaluating generative models

Three kinds of evaluation answer three questions: likelihood (density on test data), sample-based distances (does the set of samples look like the data), and forecast scores (was the predicted distribution good for the outcome that happened).
With uniform dequantization, $\mathbb E_u\log p(x + u) \le \log P(x)$ (Jensen), and $\mathrm{bpd} = (-\log p_y(y) + D\log 256)/(D\ln 2)$ for data rescaled to $[0, 1]$ ; the uniform model has 8 bpd.
FID is the Fréchet distance $\|\mu_r - \mu_g\|^2 + \mathrm{tr}(\Sigma_r + \Sigma_g - 2(\Sigma_r^{1/2}\Sigma_g\Sigma_r^{1/2})^{1/2})$ between Gaussians fitted to Inception features; compute it with the symmetric form and eigh.
The FID estimate is biased upward, about $2d/N + d(d+1)/(2(N-1))$ for white features, so FID values are comparable only at equal $N$ , features and preprocessing; KID (the unbiased MMD² with a cubic kernel) has mean 0 for equal distributions and can be negative.
k-NN precision measures fidelity (generated samples inside the real support) and recall measures diversity (real samples inside the generated support); mode collapse gives high precision and low recall.
A good likelihood does not imply good samples: mixing in 99% junk costs at most $\log 100$ nats per image, $0.0022$ bits/dim for $D = 3072$ .
The CRPS $\int(F - \mathbb 1\{x \ge y\})^2dx = \mathbb E|X - y| - \tfrac12\mathbb E|X - X'|$ is strictly proper; the standard ensemble estimator is biased up by $\mathbb E|X - X'|/(2M)$ , the fair estimator (with $1/(2M(M-1))$ ) is unbiased.
A calibrated ensemble has a flat rank histogram (U = too narrow, dome = too wide, slope = bias) and a corrected spread–skill ratio $\sqrt{(M+1)/M}\,\cdot$ spread/RMSE near 1.

15 Inverse problems and assimilation

The posterior score at every noise level splits: $\nabla\log p_t(x_t \mid y) = \nabla\log p_t(x_t) + \nabla\log p_t(y \mid x_t)$ , with the noisy likelihood $p_t(y \mid x_t) = \int p(y \mid x_0)\,p(x_0 \mid x_t)\,dx_0$ .
$p_t(y \mid x_t)$ is sharp at low noise and almost flat at high noise; in general it is intractable, because it needs the whole denoising posterior.
Linear-Gaussian case: $p_t(y \mid x_t) = \mathcal N(y;\ A\mu_{0|t},\ \sigma_y^2 I + A\Sigma_{0|t}A^\top)$ with $\Sigma_{0|t} = (\sigma_t^2/a_t)\,J$ (second-order Tweedie); adding the prior score gives the score of the noised exact posterior.
DPS uses $\mathcal N(y;\ \mathcal A(\hat x_0),\ \sigma_y^2 I)$ : it drops $\mathrm{Var}(x_0 \mid x_t)$ , so it is too strong at high noise by $1 + \Sigma_{0|t}/\sigma_y^2$ (scalar case), gives over-confident samples, and can point the wrong way for a multimodal $p(x_0 \mid x_t)$ .
ΠGDM adds $\sigma_t^2 I$ (exact for unit-variance Gaussian data); moment matching adds the second-order Tweedie covariance (exact for Gaussian priors).
Zero-shot guidance serves any operator but is approximate and slow; an amortized conditional model is fast but must be retrained for a new operator; SDEdit is not posterior sampling.
Score-based data assimilation uses a diffusion prior over trajectories plus observation guidance; for a linear-Gaussian model the exact version reproduces the Kalman smoother.
Test posterior samplers with known posteriors, SBC rank histograms and coverage (calibration), plus a proper score (sharpness): a sampler that ignores $y$ passes SBC.