ML for Science · 189

Glossary

The technical terms of this track, with short definitions (and the Persian term where available). Each term links to the lesson where it is introduced.

2/3 rule: Orszag's dealiasing rule: set the top third of the Fourier modes to zero, so the aliasing errors of a quadratic product fall only on modes that are discarded. Introduced in: Lesson 8
3D-Var: Find the state that minimizes $\tfrac12\lVert x - x^b\rVert_{B^{-1}}^2 + \tfrac12\lVert y - Hx\rVert_{R^{-1}}^2$ at one time. For linear $H$ its minimizer is the Kalman analysis mean. Introduced in: Lesson 6
4D-Var: Fit the initial state of a model trajectory to all observations in a time window plus a background term. The gradient comes from one adjoint run backward through the window. Introduced in: Lesson 6
A-stable: A method whose stability region contains the whole left half-plane $\operatorname{Re} z \le 0$ . Any stable linear decay is then stepped stably for every $h$ . Introduced in: Lesson 2
addition theorem: $\sum_{m=-l}^{l} (Y_l^m(\theta, \phi))^2 = (2l + 1)/(4\pi)$ at every point: each degree has the same total power everywhere on the sphere. Introduced in: Lesson 9
adjoint ODE: The linear backward equation $\lambda' = -J_u^\top \lambda$ with a final condition from the loss; the gradient is $\int_0^T J_p^\top \lambda\,dt$ . Introduced in: Lesson 3
adjoint state: The vector $\lambda(t)$ (the cotangent of the state) that is solved backward in time from the final time to give the gradient of the loss with respect to the parameters. Introduced in: Lesson 3
adjoint-state method: Reverse-mode differentiation applied to a simulator: one extra backward (adjoint) solve gives the gradient of a scalar misfit with respect to all parameters. Introduced in: Lesson 1
aliasing: On a grid of $n$ points, a wave with wavenumber $k$ looks exactly like one with $k \pm n$ . Products of fields create such high modes, which fold back onto low ones. Introduced in: Lesson 8
analysis: The updated state estimate $x^a$ after the observations are used, with error covariance $P^a$ . Introduced in: Lesson 6
anomaly correlation coefficient (ACC): The correlation between forecast anomalies $f' = f - c$ and observed anomalies $o' = o - c$ , where $c$ is the climatology. Introduced in: Lesson 16
anomaly matrix: The $n \times N$ matrix $A$ whose columns are the ensemble members minus the ensemble mean. The sample covariance is $P^f = AA^\top/(N - 1)$ . Introduced in: Lesson 6
associated Legendre function: The function $P_l^m(\cos\theta)$ that gives the latitude part of $Y_l^m$ . Stable code uses the normalised version $\bar P_l^m$ and a recurrence, not factorial formulas. Introduced in: Lesson 9
background: The prior estimate of the state before new observations are used, written $x^b$ with error covariance $B$ (in a filter, the forecast $x^f$ with $P^f$ ). Introduced in: Lesson 6
backsolve adjoint: An adjoint method that recreates $u(t)$ by solving the state ODE backward together with $\lambda(t)$ . It needs little memory but is unstable for dissipative or stiff problems. Introduced in: Lesson 3
backward Euler: The implicit method $u_{k+1} = u_k + h\,f(u_{k+1})$ . It has order 1, needs a solve at each step, and has $R(z) = 1/(1 - z)$ ; it is A-stable and L-stable. Introduced in: Lesson 2
Brownian motion: The process $W_t$ with $W_0 = 0$ , continuous paths and independent Gaussian increments $W_{t+h} - W_t \sim \mathcal N(0, h)$ . Introduced in: Lesson 12
calibration: A forecast is calibrated when its probabilities match observed frequencies: for example, 90% intervals contain the outcome 90% of the time. Introduced in: Lesson 7
causal weighting: Weight the residual at time $t$ by $\exp(-\varepsilon \cdot \text{loss at earlier times})$ , so the PINN fits early times before later ones; $\varepsilon$ is the causality parameter. Introduced in: Lesson 11
central difference: The finite-difference estimate $(f(x + \varepsilon v) - f(x - \varepsilon v))/(2\varepsilon)$ of a directional derivative. Its truncation error is $O(\varepsilon^2)$ ; it is the standard gradient check. Introduced in: Lesson 1
CFL condition: The numerical domain of dependence must contain the physical one; for the upwind scheme this means $\mathrm{Co} \le 1$ . It is necessary for convergence, but not sufficient for stability. Introduced in: Lesson 2
chaos: Bounded dynamics in which two nearby states move apart exponentially fast, so long-term prediction of a single trajectory is impossible. Introduced in: Lesson 5
cheap gradient principle: The gradient of a scalar function costs only a small constant times the cost of the function itself, independent of the number of inputs (at most about 5 times for arithmetic programs). Introduced in: Lesson 1
checkpointing: Store the forward state only at some steps and recompute the rest during the backward pass. It trades extra compute for less memory. Introduced in: Lesson 3
Clebsch–Gordan tensor product: The equivariant product of two features of degrees $l_1$ and $l_2$ , split into parts of degree $|l_1 - l_2| \le l \le l_1 + l_2$ . For two vectors it gives the dot product, the cross product and a symmetric traceless matrix. Introduced in: Lesson 10
climatology: The long-term average state for each place and time of year. It is the reference for anomalies and a baseline forecast. Introduced in: Lesson 16
colatitude: The angle $\theta \in [0, \pi]$ from the North Pole. Latitude is $\pi/2 - \theta$ . Introduced in: Lesson 9
collocation points: The points in space and time where the PDE residual is evaluated in the PINN loss. Introduced in: Lesson 11
condition number: The ratio $\sigma_{\max}/\sigma_{\min}$ of the largest and smallest singular values of a matrix. It bounds how much relative errors in the data are amplified. Introduced in: Lesson 4
conditional flow matching (CFM): Regress $v_\theta(x_t, t)$ on the conditional target $x_1 - x_0$ along $x_t = (1 - t)x_0 + t x_1$ . Its gradient equals that of the flow matching loss. Introduced in: Lesson 14
continuity equation: The PDE $\partial_t p + \nabla\cdot(p\,v) = 0$ : the density changes only because mass moves with the velocity field $v$ . Introduced in: Lesson 13
continuous adjoint: Derive the adjoint ODE first and then solve it numerically ("optimize-then-discretize"). Its gradient differs from the discrete one by a discretization error. Introduced in: Lesson 3
continuous normalizing flow: A generative model that transports a simple density with a neural ODE; the log-density changes by $-\int \operatorname{tr}(\partial f/\partial u)\,dt$ . Introduced in: Lesson 3
control variable: The quantity that a variational method optimizes, for example the initial state $x_0$ in strong-constraint 4D-Var. Introduced in: Lesson 6
cotangent: The gradient of the scalar loss with respect to an intermediate value $x$ , written $\bar x = \partial L/\partial x$ . Reverse mode carries cotangents backward. Introduced in: Lesson 1
coupling: A joint distribution of $(x_0, x_1)$ whose marginals are the source and the target. The independent coupling draws them independently. Introduced in: Lesson 14
Courant number: The dimensionless number $\mathrm{Co} = c\,\Delta t/\Delta x$ for advection with speed $c$ : how many grid cells the signal moves in one step. Introduced in: Lesson 2
CRPS: The continuous ranked probability score, $\operatorname{CRPS}(F, y) = \int (F(z) - \mathbf 1\{z \ge y\})^2\,dz = \mathbb E|X - y| - \tfrac12\mathbb E|X - X'|$ . It is proper and reduces to the absolute error for a point forecast. Introduced in: Lesson 7
cut locus: The set of points beyond which geodesics from $x$ stop being shortest. On the sphere it is the antipode $-x$ . Introduced in: Lesson 15
cycle skipping: A failure of FWI: when the simulated wave is off by more than half a period, the misfit fits the wrong cycle and the gradient points to a wrong local minimum. Introduced in: Lesson 4
data assimilation: The combination of a model forecast with new observations to estimate the current state and its uncertainty, repeated in cycles. Introduced in: Lesson 6
degree and order (spherical harmonics): The degree $l$ sets the total number of oscillations (the scale); the order $m$ sets how many of them are along longitude. Introduced in: Lesson 9
denoising score matching (DSM): Train $s_\theta(x)$ to match the score of the noising kernel, $-(x - x_0)/\sigma^2$ , on noisy samples. Its minimizer is the score of the noisy data distribution. Introduced in: Lesson 13
diffusion coefficient: The factor $b(X_t, t)$ (or $g(t)$ in diffusion models) that multiplies $dW_t$ and sets the size of the noise. Introduced in: Lesson 12
discrepancy principle: Choose the regularization weight $\alpha$ so that the residual $\lVert Gm_\alpha - d\rVert$ equals the expected size of the noise. Introduced in: Lesson 4
discrete adjoint: Reverse-mode differentiation of the actual solver steps ("discretize-then-optimize"). It gives the exact gradient of the computed loss. Introduced in: Lesson 3
discrete Fourier transform (DFT): The map from $n$ samples to $n$ Fourier coefficients $\hat u_k$ , computed in $O(n\log n)$ by the FFT. Introduced in: Lesson 8
double penalty: A sharp forecast of a feature in a slightly wrong place is penalized twice by RMSE (once where it is and once where it should be), so RMSE favours blurry forecasts. Introduced in: Lesson 16
drift: The deterministic part $a(X_t, t)$ of an SDE: the mean velocity of the process. Introduced in: Lesson 12
dynamic mode decomposition (DMD): Fit the best linear map $A$ with $x_{k+1} \approx A x_k$ to snapshots, usually in a rank- $r$ SVD basis. Each DMD mode has one growth rate and one frequency. Introduced in: Lesson 5
dynamical core: The part of a physics-based weather or climate model that solves the fluid equations. Hybrid models keep it and learn the rest. Introduced in: Lesson 16
emulator: A fast learned model that replaces an expensive simulator, often by stepping the state forward autoregressively. Introduced in: Lesson 2
encoder–processor–decoder: The common layout of ML weather models: map the grid to a latent mesh or token space, apply many learned steps there, and map back to the grid. Introduced in: Lesson 16
ensemble Kalman filter (EnKF): A Kalman filter that replaces the covariance by the sample covariance of an ensemble of $N$ model runs. It works with nonlinear models and very large states. Introduced in: Lesson 6
entropic OT: Optimal transport with an added entropy term of weight $\varepsilon$ . It is solved by Sinkhorn iterations and gives a smooth coupling. Introduced in: Lesson 14
equivariant: A function $f$ is equivariant if $f(\rho_{\text{in}}(g)x) = \rho_{\text{out}}(g)f(x)$ : transforming the input transforms the output in a known way, for example forces under rotations. Introduced in: Lesson 10
Euler–Maruyama: The scheme $X_{k+1} = X_k + a\,h + b\,\Delta W_k$ with $\Delta W_k \sim \mathcal N(0, h)$ . It has strong order 1/2 and weak order 1 (strong order 1 for additive noise). Introduced in: Lesson 12
explicit method: A time-stepping method that computes the new state only from known states. It is cheap per step but has a bounded stability region. Introduced in: Lesson 2
exponential integrator: A method for semilinear problems $u' = Lu + N(u)$ that solves the linear part exactly with $e^{hL}$ , for example exponential Euler or ETDRK4. Introduced in: Lesson 2
exponential map: $\exp_x(v)$ : follow the geodesic from $x$ with initial velocity $v$ for unit time. On the sphere $\exp_x(v) = \cos\lVert v\rVert\,x + \sin\lVert v\rVert\,v/\lVert v\rVert$ . Introduced in: Lesson 15
fair CRPS: An estimator of the CRPS of an ensemble that uses $\tfrac{1}{2M(M - 1)}$ for the spread term, so a small ensemble is not punished for its size. Introduced in: Lesson 16
filter divergence: The filter becomes overconfident: its covariance gets too small, it ignores the observations, and the estimate drifts away from the truth. Introduced in: Lesson 6
fixed point: A state $u^*$ with $f(u^*) = 0$ , so the dynamics stay there. Its stability follows from the eigenvalues of the Jacobian at $u^*$ when it is hyperbolic. Introduced in: Lesson 5
flow matching: Train a velocity field $v_\theta(x, t)$ so that its ODE moves noise at $t = 0$ to data at $t = 1$ , by regression on the velocity of a chosen probability path. Introduced in: Lesson 14
Fokker–Planck equation: The PDE for the density of an SDE: $\partial_t p = -\partial_x(a\,p) + \tfrac12\partial_{xx}(b^2p)$ . Introduced in: Lesson 12
forward Euler: The explicit method $u_{k+1} = u_k + h\,f(u_k)$ . It has order 1 and the stability function $R(z) = 1 + z$ . Introduced in: Lesson 2
forward map: The map from a model to the data it predicts: $G$ when it is linear, $F$ when it is nonlinear. Introduced in: Lesson 4
forward mode: Automatic differentiation that pushes a tangent $\dot x$ forward together with the values. One pass gives one JVP $J\dot x$ ; it is efficient when there are few inputs. Introduced in: Lesson 1
forward sensitivity equation: The ODE $S' = J_u S + J_p$ for the sensitivities $S = \partial u/\partial p$ . Its cost grows with the number of parameters. Introduced in: Lesson 3
Fourier neural operator (FNO): A neural operator built from Fourier layers: FFT, multiply the lowest modes by learned weights, inverse FFT, add a pointwise linear term, apply a nonlinearity. Introduced in: Lesson 8
full-waveform inversion (FWI): Seismic imaging that fits whole recorded waveforms with a wave-equation simulation. The gradient comes from the adjoint-state method: one forward and one adjoint simulation per source. Introduced in: Lesson 4
gate nonlinearity: An equivariant nonlinearity that multiplies each vector or higher-degree feature by a nonlinear function of a scalar feature. Introduced in: Lesson 10
Gaussian grid: A latitude–longitude grid whose latitudes are the Gauss–Legendre nodes in $\cos\theta$ and whose longitudes are equally spaced. The SHT on it is exact for band-limited fields. Introduced in: Lesson 9
Gaussian process (GP): A distribution over functions such that the values at any finite set of points are jointly Gaussian, with mean $m(x)$ and covariance given by a kernel $k(x, x')$ . Introduced in: Lesson 7
Gauss–Newton method: An iterative method for nonlinear least squares that linearizes the forward map at each step and solves the resulting linear least-squares problem. Introduced in: Lesson 4
geodesic: A locally shortest curve on a manifold. On the sphere the geodesics are great circles. Introduced in: Lesson 15
geometric Brownian motion: The SDE $dX = \mu X\,dt + \sigma X\,dW$ with solution $X_t = X_0\exp((\mu - \sigma^2/2)t + \sigma W_t)$ . Introduced in: Lesson 12
Gram matrix: The matrix $K$ with entries $K_{ij} = k(x_i, x_j)$ for a set of inputs. Introduced in: Lesson 7
group: A set with an associative product, an identity and inverses, such as the rotations $SO(3)$ or the rotations and reflections $O(3)$ . Introduced in: Lesson 10
ill-posed: A problem that is not well-posed. In practice small noise in the data causes large errors in the recovered model. Introduced in: Lesson 4
IMEX method: An implicit–explicit method: it treats the stiff linear part of $u' = Lu + N(u)$ implicitly and the nonlinear part explicitly. Introduced in: Lesson 2
implicit method: A time-stepping method in which the new state appears on both sides of the update, so each step solves an equation (often a linear system). Introduced in: Lesson 2
inflation: Multiply the ensemble anomalies by a factor slightly above 1 to counter the under-dispersion that sampling error and model error cause. Introduced in: Lesson 6
innovation: The difference $y - Hx^f$ between the observation and the forecast mapped to observation space. Its covariance is $S = HP^fH^\top + R$ . Introduced in: Lesson 6
invariant: A function $f$ is invariant under a group if $f(g \cdot x) = f(x)$ for every group element $g$ , for example an energy under rotations. Introduced in: Lesson 10
inverse problem: Recover a model $m$ (for example the wave speed inside the Earth) from noisy data $d = G(m) + \text{noise}$ , where the forward map $G$ predicts data from a model. Introduced in: Lesson 4
irreducible representation (irrep): A representation with no smaller invariant subspace. The irreps of $SO(3)$ are the Wigner-D matrices $D^l$ of size $2l + 1$ . Introduced in: Lesson 10
Itô integral: The stochastic integral $\int b\,dW$ defined with the integrand at the left end of each interval. It has mean zero and satisfies the Itô isometry. Introduced in: Lesson 12
Itô SDE: An equation $dX_t = a(X_t, t)\,dt + b(X_t, t)\,dW_t$ with drift $a$ and diffusion coefficient $b$ , where the noise integral is the Itô integral. Introduced in: Lesson 12
Itô's lemma: The chain rule for SDEs: $df(X) = (f'a + \tfrac12 b^2 f'')\,dt + f'b\,dW$ . The extra term $\tfrac12 b^2 f''$ is the Itô correction. Introduced in: Lesson 12
Jacobian: For a map $y = F(x)$ with $x \in \R^n$ and $y \in \R^m$ , the $m \times n$ matrix $J$ of first partial derivatives. The derivative is the linear map $dy = J\,dx$ . Introduced in: Lesson 1
Jacobian-vector product (JVP): The product $J\,\dot x$ : how the output changes when the input moves in the direction $\dot x$ . Forward mode computes one JVP per pass. Introduced in: Lesson 1
Kalman filter: The exact Bayesian filter for linear models with Gaussian errors. It alternates a forecast step (propagate mean and covariance) and an analysis step (Gaussian update with the Kalman gain). Introduced in: Lesson 6
Kalman gain: The matrix $K = P^fH^\top(HP^fH^\top + R)^{-1}$ that weights the innovation: $x^a = x^f + K(y - Hx^f)$ . Introduced in: Lesson 6
kernel: A symmetric function $k(x, x')$ that gives the covariance of the function values at $x$ and $x'$ . It is valid when every Gram matrix it makes is positive semidefinite. Introduced in: Lesson 7
Koopman operator: The linear operator that advances functions of the state (observables): $(\mathcal K g)(u) = g(\Phi(u))$ , where $\Phi$ is the flow map. It is linear even when the dynamics are not. Introduced in: Lesson 5
kriging: The name of GP regression in geostatistics: the best linear unbiased prediction of a spatial field from scattered measurements. Introduced in: Lesson 7
L-curve: The curve of the solution norm against the residual norm for many values of $\alpha$ , on log axes. Its corner is a common choice of $\alpha$ . Introduced in: Lesson 4
L-stable: A method that is A-stable and also has $R(z) \to 0$ as $z \to -\infty$ , so very fast modes are damped in one step. Introduced in: Lesson 2
latitude weights: Weights proportional to the cosine of latitude in scores and losses, so that each grid cell counts by its area. Introduced in: Lesson 16
Lax equivalence theorem: For a consistent linear scheme for a well-posed linear problem, stability is equivalent to convergence ("consistency + stability = convergence"). Introduced in: Lesson 2
lead time: The time between the initialisation of a forecast and the time it is valid for. Introduced in: Lesson 16
length scale: The kernel hyperparameter $\ell$ : the input distance over which function values stay strongly correlated. Introduced in: Lesson 7
LETKF: The local ensemble transform Kalman filter: a deterministic square-root ensemble filter that computes the analysis in the space of the ensemble, separately for each local region. Introduced in: Lesson 6
local truncation error: The error that one step makes when it starts from the exact solution. A method is consistent if this error divided by $h$ goes to 0. Introduced in: Lesson 2
localisation: Remove spurious long-distance correlations of a small ensemble, either by tapering the covariance (covariance localisation) or by local analyses (domain localisation). Introduced in: Lesson 6
log score: The negative log predictive density $-\log p(y)$ at the observed outcome. It is strictly proper but punishes overconfident forecasts very strongly. Introduced in: Lesson 7
logarithm map: $\log_x(y)$ : the tangent vector at $x$ that the exponential map sends to $y$ . It is not defined at the cut locus. Introduced in: Lesson 15
Lorenz-63: The three-variable ODE of Lorenz (1963), $x' = \sigma(y - x)$ , $y' = x(\rho - z) - y$ , $z' = xy - \beta z$ . With $\sigma = 10$ , $\rho = 28$ , $\beta = 8/3$ it is chaotic. Introduced in: Lesson 5
loss imbalance: The terms of a PINN loss have gradients of very different sizes, so training reduces one term and ignores the others. Introduced in: Lesson 11
Lyapunov exponent: The average exponential rate $\lambda_1$ at which small perturbations grow: $\lVert\delta(t)\rVert \approx e^{\lambda_1 t}\lVert\delta_0\rVert$ . A positive value means chaos. Introduced in: Lesson 5
Lyapunov time: The time scale $1/\lambda_1$ over which small errors grow by a factor $e$ . Making the initial error 10 times smaller adds only $\ln 10/\lambda_1$ of forecast time. Introduced in: Lesson 5
marginal likelihood: The probability of the observed data under the GP with the function integrated out, $p(y \mid \text{hyperparameters})$ ; also called the evidence. Maximizing it fits the hyperparameters. Introduced in: Lesson 7
marginal velocity: The velocity $u_t(x) = \mathbb E[x_1 - x_0 \mid x_t = x]$ that the CFM model learns: the average of the conditional targets through the point $x$ . Introduced in: Lesson 14
marginalization trick: The fact that the conditional loss and the marginal loss differ by a constant that does not depend on $\theta$ , so training on single pairs learns the marginal velocity. Introduced in: Lesson 14
Matérn kernel: A family of kernels with a smoothness parameter $\nu$ . Its sample functions are rougher than RBF samples, which often fits physical data better. Introduced in: Lesson 7
method of lines: Discretize a PDE in space only. The result is a large system of ODEs in time, which a time-stepping method then solves. Introduced in: Lesson 2
Milstein scheme: Euler–Maruyama plus the term $\tfrac12 b\,b'(\Delta W^2 - h)$ . It has strong order 1 in one dimension; it changes nothing when the noise is additive. Introduced in: Lesson 12
minibatch OT: Pair noise and data inside each minibatch with an optimal transport plan instead of at random. It gives straighter paths at a small cost. Introduced in: Lesson 14
neural ODE: A model $u' = f_\theta(u, t)$ whose right-hand side is a neural network; the output is the state at a later time, computed with an ODE solver. Introduced in: Lesson 3
neural operator: A network that learns a map between functions (for example from an initial condition to a later solution) and can be evaluated on different grids. Introduced in: Lesson 8
neural tangent kernel (NTK): The kernel $\Theta(x, x') = \nabla_\theta u(x)^\top\nabla_\theta u(x')$ . For wide networks it describes training by gradient descent and explains spectral bias and loss imbalance. Introduced in: Lesson 11
Nyquist mode: The highest wavenumber $k = n/2$ on an even grid of $n$ points. It has no sign, so it must be set to zero in odd derivatives. Introduced in: Lesson 8
observable: A scalar function $g(u)$ of the state, such as a measurement. The Koopman operator acts on observables. Introduced in: Lesson 5
observation operator: The map $H$ from the model state to the observed quantities, as in $y = Hx + v$ with $v \sim \mathcal N(0, R)$ . Introduced in: Lesson 6
optimal transport (OT): The problem of moving one distribution onto another at the smallest total cost $\mathbb E\,c(x_0, x_1)$ . Monge seeks a map; Kantorovich seeks a coupling. Introduced in: Lesson 14
order (of a method): A method has order $p$ if its global error after a fixed time behaves like $C h^p$ as the step $h \to 0$ . Its local truncation error is then $O(h^{p+1})$ . Introduced in: Lesson 2
Ornstein–Uhlenbeck process: The SDE $dX = -\theta X\,dt + \sigma\,dW$ . It is Gaussian, mean-reverting at rate $\theta$ , and its stationary distribution is $\mathcal N(0, \sigma^2/(2\theta))$ . Introduced in: Lesson 12
overdamped Langevin dynamics: The SDE $dX = -\nabla U(X)\,dt + \sqrt 2\,dW$ with potential $U$ ; its stationary density is $p \propto e^{-U}$ . With $U = -\log\pi$ it samples from $\pi$ . Its Euler–Maruyama version is the unadjusted Langevin algorithm. Introduced in: Lesson 12
parity: The sign that a feature takes under inversion $x \mapsto -x$ : even (e) or odd (o). A vector is 1o; a pseudovector such as a cross product is 1e. Introduced in: Lesson 10
perturbed observations: In the stochastic EnKF, each member is updated with its own noisy copy of the observation. Without them the analysis spread is too small. Introduced in: Lesson 6
physics-informed neural network (PINN): A network $u_\theta(x, t)$ trained to make the PDE residual small at collocation points, together with losses for the initial and boundary conditions and any data. Introduced in: Lesson 11
Picard plot: A log-scale plot of the singular values $\sigma_i$ , the data coefficients $|u_i^\top d|$ and their ratio. It shows where noise starts to dominate. Introduced in: Lesson 4
power spectrum (on the sphere): The energy per degree, $\sum_m (a_l^m)^2$ . It does not change under rotations and shows whether a field (for example an ML forecast) is too smooth. Introduced in: Lesson 9
prior mismatch: The error from starting the reverse process at the simple prior instead of the true $p_T$ . Introduced in: Lesson 13
probability integral transform (PIT): The value $u = F(y)$ of the forecast CDF at the outcome. For a calibrated forecast $u$ is uniform on $(0, 1)$ , so the PIT histogram is flat. Introduced in: Lesson 7
probability-flow ODE: The deterministic ODE $dx/dt = f - \tfrac12 g^2\nabla\log p_t(x)$ , which has the same marginals $p_t$ as the forward SDE. Introduced in: Lesson 13
pseudo-spectral method: Compute derivatives in Fourier space and nonlinear products on the grid, switching with FFTs. Introduced in: Lesson 8
quadratic grid: A Gaussian grid with $N_{\text{lat}} \ge (3N + 1)/2$ and $N_{\text{lon}} \ge 3N + 1$ , so quadratic products are computed without aliasing; the spherical form of the 2/3 rule. Introduced in: Lesson 9
quadratic variation: The limit of the sum of squared increments. For Brownian motion it equals $t$ , which gives the rule $(dW)^2 = dt$ . Introduced in: Lesson 12
rank histogram: A histogram of the rank of the observation among the ensemble members. It is flat for a reliable ensemble, U-shaped for an under-dispersive one. Introduced in: Lesson 16
RBF kernel: The squared-exponential kernel $k(x, x') = \sigma_f^2\exp(-\lVert x - x'\rVert^2/(2\ell^2))$ , with signal variance $\sigma_f^2$ and length scale $\ell$ . Its sample functions are infinitely smooth. Introduced in: Lesson 7
reanalysis: A consistent historical record of the atmosphere made by running one data assimilation system over past decades, such as ERA5. It is the usual training data of ML weather models. Introduced in: Lesson 16
rectified flow: The ODE flow defined by the marginal velocity of straight-line interpolation between coupled pairs. Introduced in: Lesson 14
reflow: Retrain flow matching on the pairs (start, end) produced by a trained flow. Paths become straighter and convex transport costs do not increase. Introduced in: Lesson 14
representation: A map $\rho$ from a group to invertible matrices with $\rho(g_1g_2) = \rho(g_1)\rho(g_2)$ ; it says how a feature transforms. Introduced in: Lesson 10
residual (PDE): The amount by which a candidate solution fails the PDE, for example $u_t + \beta u_x$ for advection; it is zero for an exact solution. Introduced in: Lesson 11
retraction: A cheap map from the tangent plane back to the manifold, for example $x \mapsto (x + hv)/\lVert x + hv\rVert$ on the sphere (projected Euler). Introduced in: Lesson 15
reverse-mode automatic differentiation: A method that computes the gradient of a scalar output with respect to all inputs in one backward sweep over the tape. Cotangents of a value that is used several times add up. Introduced in: Lesson 1
reverse-time SDE: For a forward SDE $dX = f\,dt + g\,dW$ , the SDE $dX = [f - g^2\nabla\log p_t(X)]\,dt + g\,d\bar W$ run from $t = T$ back to 0 has the same marginals (Anderson 1982). Introduced in: Lesson 13
Riemannian conditional flow matching (RCFM): Flow matching on a manifold: regress a tangent velocity field on the velocity of the geodesic conditional path; the marginalization trick still holds. Introduced in: Lesson 15
Riemannian manifold: A smooth curved space with an inner product on each tangent plane, which defines lengths, angles and geodesics. The sphere $S^2$ is the example of the lesson. Introduced in: Lesson 15
RK4: The classical four-stage explicit Runge–Kutta method. It has order 4. Introduced in: Lesson 2
rollout: Apply a one-step model repeatedly to its own outputs to forecast many steps ahead. Fine-tuning on rollouts reduces error growth. Introduced in: Lesson 16
Schur's lemma: An equivariant linear map between two irreps is zero if they differ and a multiple of the identity if they are the same. So equivariant linear layers only mix channels of the same degree and parity. Introduced in: Lesson 10
score: The gradient of the log-density, $\nabla_x\log p(x)$ . It does not depend on the normalising constant. Introduced in: Lesson 13
scoring rule: A function $S(F, y)$ that rates a probabilistic forecast $F$ after the outcome $y$ is known. It is proper if the expected score is best when $F$ is the true distribution. Introduced in: Lesson 7
selection rule: The rule for which output degrees and parities a tensor product can produce: $|l_1 - l_2| \le l \le l_1 + l_2$ and $p = p_1p_2$ . Introduced in: Lesson 10
sharpness: How concentrated the predictive distributions are. The goal is maximal sharpness subject to calibration. Introduced in: Lesson 7
SINDy: Sparse identification of nonlinear dynamics: fit $u' \approx \Theta(u)\,\Xi$ from a library $\Theta$ of candidate terms with a sparse coefficient matrix $\Xi$ (for example by thresholded least squares). Introduced in: Lesson 5
Sinkhorn algorithm: Alternating scaling of the rows and columns of the Gibbs kernel $e^{-C/\varepsilon}$ until the coupling has the right marginals. Small $\varepsilon$ needs the log domain. Introduced in: Lesson 14
slerp: Spherical linear interpolation along the great circle: $x_t = \exp_{x_0}(t\log_{x_0}x_1)$ . It is the geodesic conditional path on the sphere. Introduced in: Lesson 15
spectral accuracy: For smooth periodic functions, the error of spectral methods falls faster than any power of $1/n$ (exponentially for analytic functions). Introduced in: Lesson 8
spectral bias: Gradient-trained networks fit low frequencies first and high frequencies much more slowly. Introduced in: Lesson 11
spectral derivative: Differentiate a periodic function by multiplying each Fourier coefficient $\hat u_k$ by $ik$ and transforming back. Introduced in: Lesson 8
spherical Fourier neural operator (SFNO): An FNO on the sphere: SHT, multiply the coefficients by learned weights that depend on the degree $l$ only, inverse SHT. This makes the layer equivariant to rotations. Introduced in: Lesson 9
spherical harmonic: A function $Y_l^m(\theta, \phi)$ on the sphere with degree $l \ge 0$ and order $-l \le m \le l$ . It is an eigenfunction of the Laplacian on the sphere with eigenvalue $-l(l+1)$ . Introduced in: Lesson 9
spherical harmonic transform (SHT): The map between grid values on the sphere and spherical harmonic coefficients: an FFT along each circle of latitude, then Gauss–Legendre quadrature in latitude. Introduced in: Lesson 9
spread–skill ratio: The ensemble spread divided by the RMSE of the ensemble mean. For a perfect ensemble of size $M$ the expected skill$^2$ is $\tfrac{M + 1}{M}$ spread$^2$. Introduced in: Lesson 16
stability function: The factor $R(z)$ with $y_{k+1} = R(z)\,y_k$ when a method is applied to the test equation $y' = \lambda y$ , where $z = h\lambda$ . Introduced in: Lesson 2
stability region: The set of complex numbers $z = h\lambda$ with $|R(z)| \le 1$ . A linear system is stepped stably only if every $h\lambda_i$ lies in it. Introduced in: Lesson 2
stationary density: A density that does not change in time under the Fokker–Planck equation. For overdamped Langevin dynamics with potential $U$ , the zero-flux condition gives $p \propto e^{-U}$ . Introduced in: Lesson 12
stiff: An ODE is stiff when it has very fast decaying modes next to the slow dynamics of interest, so explicit methods need tiny steps for stability, not for accuracy. The stiffness ratio $\max\lvert\operatorname{Re}\lambda\rvert / \min\lvert\operatorname{Re}\lambda\rvert$ measures the gap. Introduced in: Lesson 2
straightness: $S = \int_0^1 \mathbb E\lVert (Z_1 - Z_0) - u_t(Z_t)\rVert^2\,dt$ . $S = 0$ means straight paths at constant speed, so one Euler step is exact. Introduced in: Lesson 14
Stratonovich integral: A stochastic integral with the integrand at the midpoint. It obeys the ordinary chain rule; it differs from the Itô integral when the noise depends on the state. Introduced in: Lesson 12
tangent plane: The plane of velocities at a point $x$ of the sphere: all $v$ with $x^\top v = 0$ . Introduced in: Lesson 15
tangent-linear model: The linearized dynamics $\delta' = J_u(u(t))\,\delta$ of a small perturbation along a trajectory. Introduced in: Lesson 5
tape: The record of the operations and intermediate values of a forward pass. The backward pass walks it in reverse order and applies one VJP per step. Introduced in: Lesson 1
Tikhonov regularization: Minimize $\lVert Gm - d\rVert^2 + \alpha^2\lVert m\rVert^2$ . In the SVD it multiplies each component by the filter factor $\sigma_i^2/(\sigma_i^2 + \alpha^2)$ . Introduced in: Lesson 4
time-delay embedding: Stack past values of a measured signal into a vector (a Hankel matrix). This adds rows when there are few observables. Introduced in: Lesson 5
triangular truncation: Keep all spherical harmonic coefficients with $0 \le l \le N$ and $-l \le m \le l$ , written $T_N$ ; $N$ is the band limit. The resolution is then the same everywhere on the globe. Introduced in: Lesson 9
truncated SVD (TSVD): A regularized solution that keeps only the components of the largest singular values and drops the rest. Introduced in: Lesson 4
truncation error: The error of a numerical formula that comes from cutting off a Taylor series, for example $O(\varepsilon^2)$ for a central difference. It falls as the step gets smaller, while rounding error grows. Introduced in: Lesson 1
Tweedie's formula: For $x = x_0 + \sigma\varepsilon$ with Gaussian noise, $\mathbb E[x_0 \mid x] = x + \sigma^2\nabla\log p_\sigma(x)$ : the score gives the best denoiser. Introduced in: Lesson 13
vector-Jacobian product (VJP): The product $\bar x = J^\top \bar y$ : it turns the sensitivity of the loss to the output into the sensitivity to the input. Reverse mode computes one VJP per pass and never forms $J$ . Introduced in: Lesson 1
viscous Burgers equation: The PDE $u_t + u u_x = \nu u_{xx}$ : nonlinear advection plus diffusion with viscosity $\nu$ . The Cole–Hopf transform gives exact solutions. Introduced in: Lesson 8
Wasserstein-2 distance: $W_2(\mu, \nu) = \big(\min_\pi \mathbb E_\pi\lVert x_0 - x_1\rVert^2\big)^{1/2}$ over couplings $\pi$ . For Gaussians it has a closed form with the Bures term. Introduced in: Lesson 14
well-posed: Hadamard's conditions: a solution exists, it is unique, and it depends continuously on the data. Introduced in: Lesson 4
Wigner-D matrix: The matrix $D^l(R)$ of size $(2l+1)\times(2l+1)$ that tells how the spherical harmonics of degree $l$ mix under a rotation $R$ . Introduced in: Lesson 9