Interactive figures
Every figure in the track. Each one is a small experiment: drag, slide, and watch the numbers. The link under each figure takes you to the lesson that explains it.
Lesson 1 · Reverse-mode autodiff
Step through the tape. Press Next step. The forward pass fills the values (blue), one node at a time. Then the backward pass fills the cotangents (violet), in the reverse order. Watch node : it is used twice by , so it receives two contributions (highlighted arrows), and they add up to . At the end, a central difference checks one entry of .
Choose a shape. Pick a preset or move the sliders for the number of inputs and outputs . The bars (log scale) count the passes needed for the full Jacobian: forward mode needs , reverse mode needs . Notice the memory line: only reverse mode stores a tape.
Test a gradient code. The function is for a matrix. The curves show the relative error of the central (solid) and forward (dashed) differences against the step . Switch to the buggy gradient ( instead of ): the curves stop at a floor that no choice of removes. Then move the asymmetry slider all the way to the left (). The bug becomes invisible, because for a symmetric matrix.
Lesson 2 · Time stepping and stiffness
Read the slopes. The figure solves , up to (exact solution ) with forward Euler, the midpoint method (RK2) and RK4. Move the slider: halving divides the error by about 2, 4 and 16, so the observed orders are 1, 2 and 4. Go to the smallest steps: RK4 hits the rounding level near , and the slope there means nothing.
Drag (yellow) and switch the method. The shaded set is the stability region of the method; the chart below compares the numerical solution with the exact . Forward Euler is the disk of radius 1 around . Use the stiff preset (): forward Euler explodes, backward Euler damps the mode to almost 0, and the trapezoid rule keeps it near , flipping its sign each step.
Set and press Play. Top: forward Euler against the exact solution of the semi-discrete system, starting from a hat function. Bottom: the 31 numbers and the forward-Euler interval (aqua). At every point is inside. Move just above and press Play again: the leftmost points (magenta) leave the interval, and the sawtooth mode grows until it covers the solution.
Move the Courant number. Left: the space-time grid. The value at the yellow point after 6 steps depends on the orange cone of grid values; the exact solution depends on the foot of the characteristic line. For the foot leaves the cone (magenta), and the upwind solution on the right explodes. For it is stable but smeared (numerical diffusion); at it is exact on this grid.
Change and . The equation is , . After a transient of length about , the exact solution follows . With and , : forward Euler oscillates and explodes, while backward Euler stays smooth with the same step. Lower below and forward Euler recovers, at the price of many more steps.
Lesson 3 · Adjoints and neural ODEs
Drag the time slider from down to 0, or press Play. The blue curve is the forward solution , computed first and stored. The violet curve is the adjoint ; it starts at and is filled in backward. The dashed violet curve is the running integral ; at it is , and the readout compares it with finite differences. For the logistic model, notice that is smallest in the middle, where the growth is fastest: there a small push only moves the S-curve a little earlier in time, which changes very little.
Move the sliders for the number of parameters and the number of time steps . The upper chart is the work in units of one forward solve; the lower chart is the memory in units of one state vector (the numbers of the gradient itself are not counted). Both axes are logarithmic. Blue bars give the exact gradient of the computed loss, magenta bars an approximation. Notice that only finite differences and forward sensitivities grow with , and only the stored adjoint grows with . Checkpointing keeps about states for one extra forward pass.
Switch between Euler and RK4, and move the step size . The problem is , , , , , with exact gradient . The orange line is the error of the discrete adjoint, the violet line the error of the continuous adjoint (computed by backsolve with the same method), and the magenta line their difference. All three have slope 1 for Euler and slope 4 for RK4: both gradients converge, but the gap between them never reaches zero.
Move the decay rate of the fast mode. The system is with having eigenvalues and (in rotated axes). RK4 with solves it forward to , then backward from the computed . The magenta curve is the error of the re-created state ; the dashed grey line is . The violet curve is the size of the adjoint , solved backward on the stored states: it stays bounded. Notice that for of about 35 and more the re-created is useless, while the forward solution was accurate.
Choose the number of steps and the number of checkpoints , then press Backward step. Blue boxes are checkpoints stored in the forward pass; orange boxes are states recomputed for the current segment; the violet outline is the step whose adjoint is being computed. The chart shows the peak memory and the number of recomputed steps for every . Notice the minimum of the memory curve near .
Lesson 4 · Inverse problems and seismic imaging
Move the regularization weight and the noise level, and switch the method. Upper chart: the true model (blue), the blurred noisy data (aqua dots) and the reconstruction (orange). Lower chart, on a log scale (the Picard plot): the singular values (dark line), the data coefficients (aqua), the naive coefficients (magenta) and the filter factors (orange). Notice that stops falling at the noise level, and from there the naive coefficients grow. With "naive", the reconstruction leaves the chart.
Move along the L-curve, and change the noise level. Upper chart: the L-curve on log–log axes, with the current as a dot. Lower chart: the true error (magenta) against . The vertical marks show the corner of the L (largest curvature, dark), the discrepancy-principle choice (aqua) and the smallest error (blue). Notice that both rules land near the flat valley of the error curve without knowing ; the discrepancy choice is usually closer to the best , and the corner can miss it by a factor of several.
Drag the stations (triangles) and the starting guess (orange), then press Gauss–Newton step. The shading is the misfit (stronger colour is larger), minimized over the origin time; the blue star is the true epicentre. The dashed orange ellipse is the linearized 95% uncertainty region at the current estimate. Try "one side" and "in a line": the misfit valley becomes long, the ellipse stretches, and with stations in a line there are two mirror minima.
Press Play (or drag the time) to run the adjoint backward from to 0. The source and the receiver are at the surface ; the starting model has , and the true model has a layer (shaded). Upper chart: the forward wavefield of the starting model (blue) and the adjoint field (violet) at the current time. Middle chart: the recorded trace (aqua), the predicted trace (orange, dashed) and the residual (magenta). Lower chart: the gradient accumulated so far (magenta). Notice that it grows only where and meet, at the edges of the layer, and that the image of the top of the layer changes sign when you switch to a slower layer.
Drag the starting velocity and change the high frequency. The curves are the misfit against the model velocity for a low-frequency (blue) and a high-frequency (orange) wavelet; the true velocity is 2 km/s. The dots show where gradient descent from the starting velocity ends. Notice that the high-frequency curve has local minima, and that descent finds the true velocity only when the time error is below about half a period.
Lesson 5 · Chaos, DMD and SINDy
Drag the orange point in the trace–determinant plane (left), or use the presets. The right side shows the phase portrait of a matrix with this trace and determinant: blue trajectories with arrows, and the eigenvector lines in yellow when the eigenvalues are real. Below the dashed parabola the eigenvalues are real; above it they are a complex pair. Notice that the sign of alone decides stability when .
Press Play, or move the time slider. Two Lorenz-63 trajectories start apart (blue and orange, shown in the plane). For a long time they look like one curve. The magenta curve below is their distance on a log axis. It wiggles, but on average it follows the dashed line of slope , until it saturates at the size of the attractor. Change by a factor 1000: the time to reach distance 1 moves by only about .
Drag the yellow start point, then switch the observables. The system is , . The blue curve is the true trajectory; it falls fast onto the grey parabola (the slow manifold) and then slides slowly along it. DMD uses the aqua snapshots () and its linear model is the orange dashed curve. With the state the model misses the bend and one rate is wrong. With the error drops to rounding level and the rates are exactly , and . The chart shows why: along the trajectory, the logarithms of the three Koopman eigenfunctions are straight lines.
Move the rank , then add noise. The data (left map) are two travelling waves, one neutral and one decaying, plus sensor noise; violet is negative, yellow positive. The right map is the rank- DMD model, rolled out from the first snapshot. In the complex plane, the orange dots are the DMD eigenvalues and the blue rings the true ones. With they match: two on the unit circle, two just inside. With a wave is missing. With noise and , extra eigenvalues appear deep inside the circle: they fit noise. The singular values (top right) show the gap after 4.
Move the threshold, then add noise. The bars are the 30 coefficients of the first, dense least-squares fit, on a log axis: blue for the 7 true terms, magenta for the 23 terms that should be 0. The yellow line is the threshold . Below it, the equations that STLSQ returns; wrong extra terms appear in magenta. STLSQ succeeds when the line separates the blue bars from the magenta bars. Notice that the smallest true coefficient is 1, so no threshold above 1 can work.
Lesson 6 · Data assimilation
Move the three sliders. In this lesson blue is the forecast (prior), aqua is the observation and orange is the analysis (posterior). With equal variances the analysis sits halfway, at . Make small and the orange curve jumps to the observation; make it large and the orange curve stays with the forecast. Notice that the orange curve is always the narrowest, and its width does not change when you move .
Drag the aqua handle to change the observed value of , and move the correlation slider. The blue ellipses are the forecast (1 and 2 standard deviations), the aqua band is the observation , and the orange ellipses are the analysis. The orange arrow is the analysis increment. Notice: the arrow has a vertical part (a change of the unobserved ) only when , and the ellipse shrinks most along the observed axis.
Press the button several times. It alternates the two steps of the Kalman filter on an oscillator whose state is (position, velocity). Only the position is observed. After a forecast step the ellipse has moved and grown. After an analysis step it has shrunk, mostly along the position axis, where the observation line is. The grey dashed curve is the truth. Watch the time chart: after a few cycles the orange band of standard deviations has a steady width, and the truth stays inside it. The velocity is never observed, but its uncertainty also falls, because the model mixes position and velocity.
A twin experiment on Lorenz-63. One model run is the truth (grey); all three variables are observed every 0.15 time units with error variance (aqua). Step through a cycle: the forecast members spread out along the attractor, and the analysis members are pulled toward the observation. The lower chart shows the error of the ensemble mean, the free run (the model without data, which loses the truth after a few cycles), and the ensemble spread (dashed). Then try: ; perturbed observations off with ; and off with . Watch for filter divergence: the spread falls far below the error.
Press "Gradient step" a few times. The square plane is the space of the control variable = (initial position, initial velocity). Grey curves are contours of ; the blue point is the background , the grey point the truth, and the orange point the current (drag it anywhere). The violet arrow is the next step along , and comes from one backward adjoint sweep; the readout compares it with finite differences. The time chart shows the trajectories over the window: the orange one bends toward the observations. Then set : with one observation time the contours become a long valley, and the velocity is fixed only by the background.
Lesson 7 · Gaussian processes and UQ
Click "Clear points" first to see the prior: three random functions (thin orange) and the 95 % band . Move and watch how fast the samples wiggle. Then click to add data points (aqua). Notice three things: the posterior mean (thick orange) passes near the points, the band is narrow near the data and opens up to the prior band in gaps, and the band does not depend on the values, only on where the points are (drag a point up and down). Switch the band to "new y" to add the noise variance .
Move the three sliders. Top: the forecast CDF (orange), the step of the observation (aqua) and the squared gap (magenta), whose area is the CRPS. Make very small: the CDF becomes a step at and the CRPS becomes . Bottom: suppose the observations come from the truth . The orange curve is the average CRPS of your forecast over many observations, as a function of . Set : the curve is lowest at , the true spread, and it reaches the blue line , the score of the honest forecast. No other forecast does better on average.
Move the length scale (the yellow line) along the evidence curve. The orange curve is the log marginal likelihood, the sum of the data fit (aqua), the complexity penalty (magenta) and a constant. The data fit rises as gets shorter; the penalty falls. For these 10 points and there are two local maxima: (a wiggly fit that explains almost every bump) and (a smooth fit that calls the bumps noise). Look at the fit below for each. Now raise to 0.5: one maximum remains.
Choose a forecaster. The truth is a predictable part plus noise of variance 1. The forecaster issues ; the correct choice is , . Left: the PIT histogram (orange) against the flat ideal. Right: the observed coverage of the central intervals against the nominal level; the magenta segments show the gap from the diagonal. Try "too narrow" (U shape, coverage below the diagonal) and "too wide" (a hump). Then try "climatology": its histogram is flat too, but it is less sharp, and its mean CRPS is about instead of .
Lesson 8 · Spectral methods and FNO
Move the slider for . Top: the exact derivative, the spectral derivative and the second-order finite difference at the grid points. Bottom: the largest error against on log–log axes. For the analytic function the spectral error curve bends down (faster than any power of ) until round-off near ; the finite-difference error is a straight line of slope . Switch to and : with less smoothness the advantage shrinks or disappears.
Move above . The true wave is sampled at points (aqua dots). When , the same samples belong to a lower mode (dashed). The bottom panel is the folding map from to the mode the grid sees: a zigzag, like a strip of paper folded at . Try , : the grid sees mode .
Choose and press Play. The initial wave steepens into a front at near . Top: the exact Cole–Hopf solution and the pseudo-spectral solution on 64 points. Bottom: the energy , exact (blue), with the 2/3 rule (orange) and without it (magenta). With the rule, the energy never increases, as Theorem 3 says. Without it, the energy rises after the front forms and the run blows up near . For the two runs agree: aliasing matters only when the grid is too coarse for the solution.
Switch the activation to "none", then back to GELU. A small FNO block (Fourier layer, pointwise weight, activation, second Fourier layer, 8 modes each) runs on the chosen grid. Top: its output and the reference on 1024 points. Bottom: the largest output error against . Without the activation the error is round-off on every grid (Theorem 4). With GELU it is at and falls fast. Now add to the input with no activation: grids with fold mode 20 onto a kept mode, so the error is large before any activation.
Lesson 9 · Spherical harmonics and SFNO
Move the latitude slider and compare the two bars: the share of the grid points (orange) with |latitude| ≥ and the share of the area (blue) there, which is . Notice that at the polar caps (yellow) hold 34 % of the points of a 1° grid but only 13 % of the area. Drag the sphere to look at a pole: the rows become tiny circles.
Move the degree and the order . Orange is positive, blue is negative, and the dark lines are the nodal lines. Count them: great circles through the poles and circles of latitude. Try "zonal" (: bands of latitude), "sectoral" (: only meridians) and "tesseral" (a checkerboard). Switch to the map view to see the same harmonic on a longitude–latitude rectangle.
Move the band limit and watch the map and the power spectrum (one dot per degree). The orange dots are kept, the grey ones are dropped, and the dashed line is . Notice that the small storm (the narrow bump) needs about to appear with its full height. Switch to "sharp front": its spectrum falls slowly, and the truncated field shows ripples along the front (the Gibbs effect of lesson 08). Use "error" to see where the truncated field differs from the original.
Choose the number of nodes and the degree of the test integrand (blue curve). The orange dots and bars are the nodes and the weights. The lower chart shows the relative error for every degree: it stays at rounding level inside the aqua band () and jumps up right after it. The yellow dot is your .
Compare the two lower maps ("A: rotate, then filter" and "B: filter, then rotate") while you move the tilt . With the factor the maps are identical and the difference is about . Switch to the factor (it smooths only in the east–west direction): now the maps differ. Then set and move only the spin : the difference drops to rounding level again, because a rotation about the polar axis does not mix orders.
Lesson 10 · Equivariant networks
Move the rotation angle and the shift. The model computes an energy and a force on each atom. The orange arrows are the model run on the moved molecule, . The dashed blue arrows are the old forces turned by . For the pair potential () they coincide and the energy does not change. Switch to the "dense layer on raw coordinates": it reads absolute positions, so the arrows split and the energy changes.
Pick an irrep and switch the inversion on and off. The aqua arrows are
the inputs ; the orange object is built from the moved inputs. Under
the inversion, the vector (1o) flips, but the cross product (1e) does not; the
dashed blue arrow shows where an object of the opposite parity would point.
Move the rotation angle: all objects turn with the inputs. For 0e and 0o read the value in the
panel: the triple product changes sign under the inversion.
Compare the four nonlinearities while you move the angle. The orange arrows are (rotate, then apply), the dashed blue arrows are (apply, then rotate). The gate and the norm nonlinearity give the same arrows (difference about ). Componentwise tanh and ReLU do not. Try with tanh: the difference is zero there, because a rotation only swaps components and changes signs, and tanh is odd. Equivariance must hold for every angle, and breaks it.
Choose the degree , the rotation axis and the angle. The heat map shows (orange positive, blue negative). Check the readouts: to rounding level, and the trace equals the character , which depends only on the angle (change the axis: the matrix changes, the trace does not). For the matrix is with the rows and columns in the order . For the -axis, notice the rotation blocks on the pairs .
Top: choose , and the parities. The allowed output degrees light up with their parity, and the dimension count checks the rule. Bottom: drag the tips of (blue) and (aqua). The panel splits into the three pieces. Notice that the squared norms add up, because the pieces are orthogonal, and that the piece vanishes when and are parallel.
Lesson 11 · PINNs and failure modes
Move from 1 to 30. Four candidate fields for advection with : the exact solution, the frozen initial condition , the zero field and a slower wave . The table gives the residual loss , the weighted initial-condition loss , the total and the relative error; the bars show on a log scale. The frozen field costs , while the zero field always costs . The magenta marker shows the wrong field with the lowest loss: the one an optimiser is pulled towards. Then raise and watch which wrong field wins.
Choose , then , and move the step slider. A random one-hidden-layer tanh network is trained, in its linearised (NTK) form, to fit the target ; the fit after gradient steps is shown on top. The bottom panel gives the relative error against for five values of (same network, same learning rate). is fitted in about ten steps; still has a large error after a million steps. The low frequencies of the target are fitted first.
Press Play, or step through training. A small network (one hidden layer of 64 tanh units) learns advection at in the one-mode form ; the run was computed in NumPy with Adam. Top: the learned and the exact coefficient . Middle: the residual at each of 32 times (log scale). Bottom: the causal weights with . The weights switch on from left to right, and the learned curve follows the exact one from forward. After 20,000 steps the causal run has an RMS error of 0.045 against 0.068 for the plain run. Switch to "plain": all weights are 1.
Compare , and . Space-time images ( across, up) of the exact solution and of two PINN fits. The PINN here is a random-feature network (200 fixed tanh features of ) whose output weights minimise the PINN loss exactly by linear least squares, so no optimiser can be blamed. One window over works at and fails at (error 98%): these features hold mostly slow variations in . Ten windows of length 0.1 (time marching) give 0.16%. The Fourier solver of lesson 08 gives round-off error.
Lesson 12 · SDEs and Fokker–Planck
Move the slider to refine the grid. All grids use the same Brownian path (the coarse increments are sums of fine ones). Top: the path on the current grid. Bottom, on log–log axes: the total variation grows like , while the quadratic variation settles at ; grey dashed lines are the expected values, the yellow dots mark the current . Press New path a few times: the quadratic variation is close to 1 on every path when is large.
Increase the volatility and watch the two lines separate. Grey: 40 of 1000 exact GBM paths (log scale). Orange, dashed: the mean , which is also what the ordinary chain rule predicts for the typical path. Blue: the true median . Dots are the sample mean and median of the 1000 paths. Bottom: the histogram of is Gaussian around . Try : the mean grows, but half of the paths end below 1.
Press Play, or move the time . Top: 25 of 2000 OU paths; the dashed blue line is the mean . Bottom: the histogram of all 2000 paths at time , the Fokker–Planck solution and the grey dashed stationary density. Each path is random; the histogram follows one deterministic curve. Increase : the process forgets faster, and the stationary density gets narrower.
Press Play and watch the fluxes. The density solves the Fokker–Planck equation of with reflecting walls at . Top: the density and the grey dashed Gibbs density . Bottom: the drift flux , the diffusion flux and their sum, the total flux (positive means "to the right"). At stationarity the two fluxes are mirror images and . In the double well, notice the long final phase: a small flux over the barrier at slowly fills the left well.
Read the slopes. For GBM, 2000 Brownian paths are shared by all step sizes and by the exact solution. The EM strong error falls with slope about 1/2, the Milstein strong error with slope about 1, and the weak error (the same for both schemes) with slope 1. Grey dashed lines show slopes 1/2 and 1. Choose small noise: the EM error is then smaller, but its slope is still 1/2 once is small.
Lesson 13 · Score matching and diffusion
Move the noise level and drag the yellow point. Shading: the density of a noised 2D mixture with three components. Grey arrows: the direction of the score (long arrows are capped). The orange arrow starts at the point and has length ; it ends at the denoised point (Tweedie's formula, section 3). Notice two things: at small the field is strong and points to the nearest component; at large it is smooth and weak and points to the overall centre.
Drag the noisy point and change . The data are (blue). Grey dashed: the noised density . Orange: the posterior of the clean value (scaled) and its mean, which is exactly . Bottom: the denoiser for every . For small it is close to the identity; for large it snaps every toward one of the two modes, with a jump between them. The preset worked example is section 4: , gives .
Press Run, then compare the presets. 600 Langevin chains with the exact score of all start at , in the smaller mode. Top: three chains over the iterations. Bottom: the histogram of all chains against the target (blue). Sharp target: the chains stay in the left mode; the share with stays far below 0.7. Smoothed target: they mix quickly. Step too large: the histogram is too wide, because ULA has a bias of order (a weak error, lesson 12).
Press Play. First half: 1500 particles from are noised forward with (aqua paths); the histogram spreads into one wide bump. Second half: 1500 new particles drawn from run the reverse-time SDE with the exact score (orange paths, Euler–Maruyama, 200 steps). At every time the histogram follows the exact (blue), and at both modes come back with the weights 0.3 and 0.7.
Change the number of steps and press New noise. The same 1500 start points feed the reverse SDE (jagged paths, Euler–Maruyama) and the probability-flow ODE (smooth paths, Heun). The thick paths start at the same point and end at different places. Bottom: both end histograms match (blue). New noise changes only the SDE: the ODE result is a fixed map from noise to data. With 5–10 steps Heun on the ODE is more accurate. Switch the start to (the practical choice, slightly different from ): the SDE partly corrects the wrong start, the ODE keeps the error.
Lesson 14 · Flow matching and OT
Switch the coupling between "independent" and "minibatch OT", and move the time slider. Grey: one straight line per pair, from a noise point to a data point (aqua). Orange: with the independent coupling, the paths of the exact marginal velocity , started at the same noise points. They first move toward the centre and then turn, because early on the model cannot know which data point a path will reach. With minibatch OT the pairs are re-matched to minimise : the transport cost drops, and no pair can be made cheaper by swapping partners. For an exact OT coupling the straight lines are the paths of the flow (Theorem 6).
Move the slider for the number of Euler steps ; then switch the coupling. The background is the velocity over time (horizontal) and position (vertical): violet where , yellow where . Dark lines: exact paths from 13 noise values. Orange: Euler with steps from the same values. With the independent coupling, one step sends every sample to , the mean of the data; the mean distance to the nearest data point is 2.0 for , about 0.5 for and about 0.01 for . With the monotone (OT) coupling, negative noise goes to and positive noise to : the paths are straight and one step is exact.
Drag the points. Blue: source points; aqua: target points; each has weight . Orange: the sorted matching. Magenta dashed: the pairing in index order . The readout checks all pairings by brute force: the best one always has the sorted cost, and the orange lines never cross.
Drag the centres and the small axis handles, and move . Blue: the source ; aqua: the target , as 1-standard-deviation ellipses. Grey segments: the straight paths of the OT map for 12 points of the source ellipse. Yellow: the distribution at time (the law of ), which is again Gaussian. Try "shift only": the Bures term is . Try "rotation": the means agree, but the shape term is not zero.
Move ; then press Reset and Step a few times. Rows are five source points with weights , columns five target points with weights , cost . Darker orange cells carry more mass; blue outlines mark the exact OT plan. After a row step the row sums (right) equal ; after a column step the column sums (top) equal . Small : a sharp plan close to the exact one, but thousands of iterations (about 11 500 for ). Large : a blurred plan close to the independent plan .
Lesson 15 · Flow matching on the sphere
Move the direction and the length of ; then press "|v| > π". The point (dark), its tangent plane (blue patch) and a tangent vector (orange, drawn at half length). walks the distance along the great circle (yellow) in the direction of . Violet: , the way back. For it equals (the arrows overlap). For the walk passes the opposite point, and the log gives the shorter way round: a different vector. Drag the sphere to turn it.
Turn β, then make δ small. is the north pole. Two points and sit at distance from the south pole, on opposite sides. They are only apart, but their log vectors at (arrows, half length) point in opposite directions, each of length : the shortest great circle goes down one side of the sphere or the other. As β turns, turns all the way round. A training target built from is discontinuous at the antipode.
Move , then try the presets. Blue: the geodesic path from to . Yellow: ; orange: its velocity at half length. The speed readout is the same for every : it is the distance . Magenta dashed: the straight chord of flow matching in . It runs inside the ball: from the North Pole to the equator, its midpoint is 1866 km below the surface.
Move the number of steps and change the target. All three samplers follow the conditional velocity from the North Pole to . Magenta: plain Euler, whose points float above the surface (norm 1.86 after one step to the equator). Orange: projected Euler; it stays on the sphere but lags behind, and the lag shrinks quickly with . Blue: exponential-map Euler, exact for every , because each step covers the fraction of the remaining distance along the same great circle.
Lesson 16 · ML weather and climate models
Choose a family and press "Next stage" (or click a box). The blue boxes are the state on the grid, the orange boxes are learned networks, the violet box is physics (the dynamical core of the hybrid model), and the yellow box is the current stage. Notice that every family has the same rollout loop (the dashed arrow), and that the hybrid and the SFNO use the spherical-harmonic tools of lesson 09.
A toy atmosphere: Lorenz-96 with 40 variables (one step = 6 hours). Learned surrogates (orange) were trained on noisy data with a one-step loss (dashed) or a -step rollout loss (solid). Compare with the true equations started from a small initial error: one run (blue) and the mean of 20 runs (violet). Change : the rollout loss lowers the error over the first days. Move : a better start helps the physics model for a while, then chaos wins. Notice where the curves level off: one run near , the ensemble mean near .
Move the shift of the true feature (blue). Forecast A (orange, dashed) puts a sharp feature at the expected place. Forecast B (violet) is the conditional mean: the feature averaged over the possible positions, with uncertainty . Notice three things. For a clear miss, A has a larger RMSE than even "no feature". Averaged over the possible shifts, B always wins. In the spectra (second chart), A has exactly the spectrum of the truth, while B has too little power at high wavenumbers.
Change the ensemble size and the spread factor . Each of 3000 cases has a predictable part and unit noise; the members are drawn with spread ( is the perfect ensemble). Left: the rank histogram. Right: the mean CRPS against , with the standard and the fair estimator (lines: expected values; dots: this sample). Try "perfect, M = 2": the fair CRPS stays at , the standard one is much higher, and spread/skill is about . Then try "too narrow".