learn.mohebi.ai

    Type at least 2 letters. Enter opens the first result, Esc closes.
    16 lessons · about 27 hours · 73 figures

    ML for Science

    Scientific models are differential equations, data are noisy and sparse, and answers need uncertainty. Learn the mathematics that joins them: differentiable simulation and adjoints, chaos and data assimilation, Gaussian processes, spectral methods and neural operators, physics-informed and equivariant networks, and diffusion and flow models, all the way to machine-learning weather and climate models.

    For students, engineers and researchers in machine learning or in a physical science (weather, climate, seismology, fluids, materials) who want to understand the methods of scientific machine learning from first principles. You need linear algebra (eigenvalues, SVD, Cholesky), multivariable calculus, basic probability (the multivariate Gaussian) and some Python with NumPy. Each lesson links to the exact lessons it builds on.

    0 / 16
    lessons marked as done
    Start the first lesson

    Quick recap

    The key takeaways of all lessons on one page

    All interactive figures

    Every figure of the track on one page

    Videos and books

    The best free courses

    Glossary

    Every technical term of the track with a short definition

    Concept map

    How the lessons of this track connect to each other and to other tracks.
    Part 1

    Differentiable simulation

    Gradients of programs and of ODE solvers, stable time stepping, and inverse problems such as seismic imaging.

    1. 01 Matrix calculus and reverse-mode automatic differentiationA derivative is a linear map, and the gradient of a scalar loss is computed by pulling cotangents back through it (vector-Jacobian products). Learn the VJPs of matmul, softmax, linear solve and log-determinant, why reverse mode is one backward sweep over a tape, what it costs, and how to check a gradient. 75 min
      3 figures
      Intermediate
    2. 02 Time stepping: accuracy, stability and stiffnessEvery simulator and every neural ODE rolls out a time stepper. Two numbers decide whether it works, the order (how fast the error falls with the step) and the stability limit (the largest step that does not blow up). Learn stability functions and regions, the explicit heat-equation limit, the CFL condition, stiffness, and how to choose between explicit, implicit, IMEX and exponential methods. 90 min
      5 figures
      Intermediate
    3. 03 Adjoint methods for ODEs and neural ODEsHow to get the gradient of a loss through an ODE solver for any number of parameters with one backward solve. The continuous and the discrete adjoint, why they differ, why re-integrating the state backward can fail, checkpointing, and neural ODEs as models. 100 min
      5 figures
      Advanced
    4. 04 Inverse problems, regularization and seismic imagingRecover a model from noisy, indirect data. Why small singular values make inverse problems ill-posed, how Tikhonov regularization and its Bayesian reading fix them, Gauss–Newton for earthquake location, and the adjoint-state method that gives the full-waveform-inversion gradient with two wave simulations. 100 min
      5 figures
      Advanced
    Part 2

    Chaos, data and uncertainty

    Why weather is hard to predict, how forecasts are joined with observations, and how to measure uncertainty honestly.

    1. 05 Chaos, predictability, Koopman, DMD and SINDyWhy small errors in a chaotic model grow exponentially and limit every forecast (Lyapunov exponents, Lorenz-63), and three ways to learn dynamics from data. The Koopman operator makes nonlinear dynamics linear on functions of the state, dynamic mode decomposition (DMD) fits the best linear model to snapshots, and SINDy finds sparse equations. 100 min
      5 figures
      Intermediate
    2. 06 Data assimilation: Kalman filter, ensemble Kalman filter and 4D-VarHow a forecast model and noisy, sparse observations are combined by Bayes' rule. The Kalman update from Gaussian conditioning, the Kalman filter, the ensemble Kalman filter with localisation and inflation, and 3D-Var and 4D-Var as MAP estimation with the gradient from one adjoint run. 120 min
      5 figures
      Advanced
    3. 07 Gaussian processes, scoring rules and calibrationA Gaussian process fits a smooth function to few data points and gives an error bar everywhere. Derive its posterior and its marginal likelihood, compute both with one Cholesky factor, and check the error bars with proper scoring rules (the CRPS), PIT histograms and coverage. 100 min
      4 figures
      Intermediate
    Part 3

    Neural operators, symmetry and physics

    Fourier and spherical spectral methods, neural operators, equivariant networks and physics-informed training.

    1. 08 Fourier spectral methods and Fourier neural operatorsOn a periodic grid, the FFT turns derivatives into multiplications and gives errors that fall faster than any power of the grid size. Learn spectral derivatives, aliasing and Orszag's 2/3 rule, a pseudo-spectral Burgers solver checked against the exact Cole–Hopf solution, and the Fourier neural operator (FNO) layer, with an exact statement of when it is resolution-invariant. 100 min
      4 figures
      Intermediate
    2. 09 Spherical harmonics and spherical Fourier neural operatorsSpherical harmonics are the Fourier basis of the sphere. Learn why a latitude–longitude grid is a bad periodic box, prove that the harmonics are eigenfunctions of the Laplacian, build the exact spherical harmonic transform from an FFT and Gauss–Legendre quadrature, and see why the spherical Fourier neural operator (SFNO) uses weights that depend only on the degree. 100 min
      5 figures
      Advanced
    3. 10 Symmetry, group representations and E(3)-equivariant networksMolecules, materials and vector fields have no preferred orientation. Learn how rotations and reflections act on scalars, vectors and tensors (group representations and irreps), why Schur's lemma fixes every equivariant linear layer, how Clebsch–Gordan tensor products combine features, and how to build and test an E(3)-equivariant message-passing network. 110 min
      5 figures
      Advanced
    4. 11 Physics-informed neural networks and their failure modesA physics-informed neural network (PINN) is trained so that it satisfies a PDE, with the PDE residual in the loss. Learn the PINN loss and its residual by nested automatic differentiation, why plain PINNs fail on fast advection (loss imbalance, spectral bias, no causality) although a small residual does guarantee a small error, the fixes (curriculum, time marching, causal weights), and an honest comparison with a classical solver. 90 min
      4 figures
      Intermediate
    Part 4

    Stochastic and generative models

    Stochastic differential equations, score-based diffusion, flow matching with optimal transport, and generative models on the sphere.

    1. 12 Stochastic differential equations, Itô calculus and the Fokker–Planck equationA stochastic differential equation adds random kicks to an ODE. Learn Brownian motion and why (dW)2=dt(dW)^2 = dt, the Euler–Maruyama method, Itô's lemma with its extra second-order term, the Ornstein–Uhlenbeck process, the Fokker–Planck equation for the density of all paths, and the strong and weak orders of SDE solvers. These are the tools behind diffusion models and stochastic climate models. 110 min
      5 figures
      Advanced
    2. 13 Score matching, the reverse-time SDE and the probability-flow ODEA diffusion model learns the score ∇xlog⁡p\nabla_x\log p of noised data by denoising, and then runs a stochastic or deterministic differential equation backward from noise to data. Learn why denoising score matching gives the true score, Tweedie's formula, the reverse-time SDE and the probability-flow ODE, both derived from the Fokker–Planck equation, and how to choose between them. These ideas power ensemble weather models such as GenCast. 110 min
      5 figures
      Advanced
    3. 14 Flow matching, rectified flow and optimal transportFlow matching trains the velocity field of an ODE from noise to data by regression on straight lines between paired samples. The pairing (the coupling) decides how curved the learned paths are and how many solver steps sampling needs. Learn the marginalization trick, conditional flow matching, rectified flow and reflow, and the optimal transport behind straight paths - Monge and Kantorovich, 1D OT by sorting, the Gaussian (Bures) formula for W2, entropic OT with Sinkhorn, and minibatch OT couplings. 120 min
      5 figures
      Advanced
    4. 15 Flow matching on the sphere (Riemannian flow matching)Earthquake epicentres, volcanoes and fires are points on a sphere, and straight lines in R3 cut through the Earth. Riemannian flow matching replaces straight lines by great circles. Learn the tangent plane, the exponential and logarithm maps of the sphere and where the logarithm fails, the geodesic conditional path and its velocity, the Riemannian CFM loss, and samplers that keep points on the sphere. 80 min
      4 figures
      Advanced
    Part 5

    ML weather and climate models

    How the pieces come together in modern weather and climate models, and how to evaluate them fairly.

    1. 16 ML weather and climate models: architectures, training and evaluationHow modern ML weather models are built (graph networks on a mesh, transformers, spherical neural operators, hybrid models, diffusion ensembles), how they are trained with rollouts, and how to evaluate them fairly with latitude weights, RMSE, ACC, spectra, the fair CRPS and the spread–skill ratio. 90 min
      4 figures
      Intermediate