ML Interview Prep
A complete guide to machine learning interviews: coding patterns, ML coding from scratch, breadth questions on classic ML, deep learning and LLMs, training and serving at scale, system design for ML and generative AI, and behavioral rounds and research talks. Every lesson has worked problems, quizzes and recall cards, so that you remember what you learn.
For students, engineers and researchers who prepare for ML engineer, research engineer or research scientist interviews. You need Python and a first course in machine learning; the math and ML tracks of this site fill the gaps that a lesson names.
Quick recap
The key takeaways of all lessons on one page
All interactive figures
Every figure of the track on one page
Videos and books
The best free courses
Glossary
Every technical term of the track with a short definition
Concept map
How the lessons of this track connect to each other and to other tracks.
Keeps every lesson page and figure of this track in this browser, so you can read it without internet.
Coding rounds
The patterns behind most coding problems, with worked problems, and the practical multi-part round.
-
01
Arrays and strings: hashing, two pointers, sliding windowThe interview routine for coding problems, the cost of Python operations, and four patterns that turn an search over pairs into one pass: hash maps, prefix sums, two pointers (with Floyd's cycle detection) and the sliding window.
80 min
4 figures
Beginner -
02
Stacks, binary search, heaps and greedy algorithmsPatterns that use order to avoid work: the stack and the monotonic stack ( by push once, pop once), binary search that is correct by construction, the binary heap and top-, interval sweeps, and greedy algorithms proved with an exchange argument.
90 min
5 figures
Intermediate -
03
Trees and graphs: traversals, BFS and DFS, topological sort, union-find, DijkstraTree traversals and binary search trees, then one search loop for graphs whose bag decides the algorithm (DFS, BFS, Dijkstra), plus topological sort and union-find, each with its proof and its cost.
95 min
5 figures
Intermediate -
04
Backtracking and dynamic programmingBacktracking walks a tree of partial solutions to list them all; dynamic programming merges the equal nodes of that tree to count or optimize. The DP script, the common state shapes, edit distance with its proof, the longest increasing subsequence, and when greedy fails.
80 min
4 figures
Intermediate -
05
Practical coding rounds: key-value store, LRU cache, rate limiter, crawlerMany interviews now test practical coding instead of short puzzles. Build a key-value store with nested transactions, an LRU cache, a token-bucket rate limiter and a concurrent crawler, each with tests and a one-line invariant.
70 min
3 figures
Intermediate
ML coding from scratch
Classic algorithms in NumPy, a training loop with AdamW and mixed precision, and a transformer decoder with a KV cache and sampling.
-
06
ML coding in NumPy: softmax, logistic regression, k-means, AUC, MLP backpropThe classic "implement it from scratch" round. Stable softmax and cross-entropy, logistic regression by gradient descent, k-means with k-means++, ROC AUC with ties, and a 2-layer MLP with its backward pass and a gradient check, each with the result that makes it correct.
75 min
4 figures
Intermediate -
07
A training loop from scratch: AdamW, accumulation, mixed precision, BPEThe training step that LLM labs ask you to write in an interview: AdamW with bias correction and decoupled weight decay, gradient accumulation with correct token weights, bf16 and fp16 mixed precision with loss scaling, and a byte-level BPE tokenizer.
80 min
5 figures
Intermediate -
08
A transformer decoder from scratch: attention, RoPE, KV cache and decodingThe whole inference path of a decoder-only transformer, as interviewers ask for it: causal multi-head attention with the right shapes, RMSNorm, RoPE with its relative-position proof, a KV cache that is provably exact, and the decoding rules (temperature, top-k, top-p, beam search).
105 min
6 figures
Advanced
ML breadth
Short, precise answers to the common questions on classic ML and A/B tests, deep learning, and large language models.
-
09
Breadth: classic ML, evaluation and A/B testingShort, precise answers to the classic breadth questions: bias and variance, L1 vs L2, trees, bagging and boosting, metrics under class imbalance, leakage, probability questions with base rates, and the design and pitfalls of an A/B test.
85 min
5 figures
Intermediate -
10
Breadth: deep learning, from initialization to diffusion and flow matchingShort, precise answers to the common deep-learning questions: optimizers and schedules, initialization from variance propagation, normalization and residual connections, regularization, the inductive biases of architectures, generative models from autoregressive to flow matching, and how to debug a training run.
95 min
5 figures
Intermediate -
11
Breadth: LLM pretraining, scaling laws, fine-tuning with LoRA, RLHF, DPO and evalsHow a chat model is trained, stage by stage, with one equation and one trade-off per stage: the pretraining loss, , the Chinchilla compute-optimal split, SFT, LoRA, the reward model and RLHF, DPO, GRPO with verifiable rewards, and how to evaluate a model without fooling yourself.
90 min
5 figures
Advanced
Training and serving at scale
Memory, parallelism and rooflines for training, and the KV cache, batching and quantization for serving.
-
12
Training at scale: memory, parallelism, rooflines and infrastructure designThe arithmetic behind "will it fit?" and "how would you train this on N GPUs?": 16 bytes per parameter, activations, the ring all-reduce, ZeRO and FSDP, tensor and pipeline parallelism, rooflines for compute, network and storage, failures and Young's checkpoint interval, and the 45-minute training-infrastructure design round.
105 min
5 figures
Advanced -
13
LLM serving: KV cache, batching, paged attention and quantizationHow an LLM inference engine works and how to size one: prefill and decode, the KV-cache size per token, why decode is memory-bound, continuous batching and paged attention, chunked prefill, which quantization helps which phase, speculative decoding, and Little's law and queueing for capacity planning.
85 min
5 figures
Advanced
System design
A 45-minute framework for ML system design, applied to recommendation and search, and to RAG, agents and text-to-image.
-
14
ML system design: the framework, recommendation, search and adsA 45-minute framework for the ML system design round, applied to recommendations, search and ads: the retrieval–ranking funnel, metrics at four levels, back-of-envelope numbers, the two-tower model with the log-Q correction, labels and calibration, A/B tests, position bias, and deployment and drift monitoring with PSI.
110 min
5 figures
Intermediate -
15
GenAI system design: RAG, assistants, agents and text-to-imageThe design round for generative AI products: when to prompt, retrieve or fine-tune; a RAG assistant end to end (chunking, BM25 and dense retrieval, reciprocal rank fusion, reranking, prompt assembly, latency and GPU budget); chat state and prefix caching; agents, compounding errors and prompt injection; text-to-image with latent diffusion and classifier-free guidance; and how to evaluate all of them.
100 min
5 figures
Advanced
Behavioral rounds and research talks
STAR stories, the research talk and the paper deep dive, and how to defend your decisions.