MLOps
A model is useful only when it runs reliably for other people. Learn how to make training runs reproducible, test models and data, package and serve them, release new versions safely, and detect when the world drifts away from the training data. Every rule comes with the small piece of math that tells you how big to make a test, a queue or a canary.
For data scientists, ML engineers and researchers who can train a model in Python and want to run it in production. You need basic Python and NumPy, and the basics of probability (distributions, the normal distribution, p-values); each lesson links the exact prerequisites.
Quick recap
The key takeaways of all lessons on one page
All interactive figures
Every figure of the track on one page
Videos and books
The best free courses
Glossary
Every technical term of the track with a short definition
Concept map
How the lessons of this track connect to each other and to other tracks.
Keeps every lesson page and figure of this track in this browser, so you can read it without internet.
Reproducible runs and tested models
Make every run a recorded function of known inputs, check the data, and test what a model has learned.
-
01
Reproducible runs, versioning and data contractsWhat MLOps is, and the two silent failures to fix first. Make every training run a recorded function of known inputs, split data with hashes so that splits never move, and check every batch against a data contract.
70 min
4 figures
Beginner -
02
Testing ML systems: unit, data, behavioural, slice and regression testsCode is tested with exact asserts; learned behaviour is tested with properties and rates. The test types of an ML system, behavioural tests (MFT, INV, DIR), the rule of three, slice regression tests with McNemar's test, and the ML Test Score.
60 min
4 figures
Intermediate
Build and run
Package the model in an image, test and build it in CI, and run jobs and services on Kubernetes and in the cloud.
-
03
Containers, GPU images and CI pipelinesFreeze the software environment of a model in a container image, order its layers so that rebuilds are fast, make it run on GPUs, and build and test it on every change with a CI pipeline.
55 min
3 figures
Beginner -
04
Pipelines, Kubernetes and the cloudRun ML jobs and services on shared machines. Make scheduled tasks safe to retry, keep a model server alive with Kubernetes probes and autoscaling, store data and checkpoints in object storage, and choose a checkpoint interval for cheap spot GPUs.
70 min
4 figures
Intermediate
Serve and release
Serve a model under a latency target, and replace it with a new version without hurting users.
-
05
Serving APIs, dynamic batching and queueing latencyPut a model behind an API that rejects bad input and tells Kubernetes when it is ready. Then use queueing theory to size it - why latency explodes near full load, how batching raises throughput, and how many replicas a p99 target needs.
75 min
5 figures
Intermediate -
06
Model registry, CD and safe deployment: shadow, canary, A/BA registry stores fixed model versions and a movable pointer; a gate decides when the pointer moves. Shadow, canary and A/B releases limit the harm of a bad version, hash routing keeps users on one version, and a power calculation says how long a canary must run.
80 min
5 figures
Advanced
Monitor
Detect distribution shift, design alerts people trust, and keep training and serving features the same.
-
07
Distribution shift and drift statistics: PSI, KS and chi-squareThe kinds of distribution shift and what each does to a model, and the three standard drift statistics (PSI, the two-sample KS test, the chi-square test) with their proofs, their noise and their limits.
60 min
4 figures
Intermediate -
08
Monitoring in production: alerts, delayed labels, feature stores and skewHow to turn drift statistics into alerts that people trust (windows, calibrated thresholds, hysteresis), what delayed labels do to performance monitoring, how feature stores and point-in-time joins keep training and serving features the same, and how to find training-serving skew.
60 min
4 figures
Intermediate
LLM applications
Version prompts and models together, gate changes with evals, calibrate LLM judges and price guardrails.