Videos and books

Every resource on this page is free to watch or read, from the author's or the publisher's own page. Where a classic book is not free, we say so and point to its official page. Each lesson of the track also ends with a Watch and read section that names the exact lecture or section.

How to use it. Watch for the picture, read for the proofs, and simulate to check both. A good default: one short video before a lesson (3Blue1Brown or StatQuest), then the lesson and its notebook, then one lecture of Statistics 110 or MIT 18.650 and one book section from the table at the end of this page.

Video courses #

For the picture #

Bayes theorem, the geometry of changing beliefs3Blue1Brown · 15 min

• 3Blue1Brown, probability videos (YouTube channel). Animated videos on Bayes' rule, the medical test paradox, the central limit theorem, the sum of two Gaussians and the beta distribution ("probabilities of probabilities"). No proofs, but a clear picture of each idea. Pairs with lessons 01, 03, 05, 06 and 10.
• StatQuest with Josh Starmer, Statistics Fundamentals (YouTube playlist). Short videos with simple drawings: standard error, confidence intervals, p-values, power, false discovery rates, the bootstrap, maximum likelihood, linear regression. Good before lessons 06, 08 and 11–13.
• Mutual Information (YouTube channel). Careful, visual videos at research level, for example on the Fisher information and on importance sampling. Pairs with lessons 09 and 14.

University courses #

Lecture 6: Monty Hall, Simpson's Paradox | Statistics 110Harvard University · Joe Blitzstein, Statistics 110 · 49 min

• Harvard Statistics 110: Probability (Joe Blitzstein) (YouTube playlist, book and course material). The best first course in probability: lectures on counting, conditioning, random variables, the common distributions, joint distributions, conditional expectation, limit theorems and Markov chains. It follows the Blitzstein–Hwang book below. Pairs with lessons 01–07 and 15.
• MIT 6.041, Probabilistic Systems Analysis and Applied Probability (John Tsitsiklis, 2010) (course). A more engineering-style first course, with recitations and solved problems: conditioning, random variables, iterated expectations, Poisson processes, Markov chains, limit theorems. The shorter video version with many solved problems is RES-6-012, Introduction to Probability (2018). Pairs with lessons 01–06 and 15.

5. Maximum Likelihood Estimation (cont.)MIT OpenCourseWare · Philippe Rigollet, 18.650 Statistics for Applications · 77 min

• MIT 18.650, Statistics for Applications (Philippe Rigollet, 2016) (course). The mathematical statistics course that matches the second half of this track: estimation, maximum likelihood and the Fisher information, the method of moments, hypothesis tests, regression, Bayesian statistics, GLMs. Pairs with lessons 08–11 and 13.
• Richard McElreath, Statistical Rethinking (2023 lectures) (YouTube playlist, course page with slides and code). A full course in applied Bayesian modelling: priors, posteriors, regression, causal graphs, multilevel models and MCMC (HMC with Stan). The lectures are free; the book is not. Pairs with lessons 10, 13 and 15.

Monte Carlo and MCMC #

The intuition behind the Hamiltonian Monte Carlo algorithmBen Lambert · 32 min

• Nando de Freitas, Machine learning (UBC 2013), lectures on importance sampling and MCMC (the lecture Importance sampling and MCMC I is linked in lesson 14). A classic lecture that goes from Monte Carlo integration to importance sampling and Metropolis–Hastings.
• Ben Lambert and ritvikmath (videos linked in lessons 14 and 15). Short explanations of Monte Carlo, Metropolis–Hastings, Gibbs sampling and HMC with simple pictures.
• Chi Feng, MCMC interactive gallery (online). Run random-walk Metropolis, Gibbs, HMC and NUTS on 2-D targets in the browser and watch them move. Use it with lesson 15.

Free books #

Probability #

• Joseph Blitzstein and Jessica Hwang, Introduction to Probability, 2nd edition (free PDF and course material). The book of Statistics 110: clear stories, many examples, and strategies for solving problems. The main reading for lessons 01–07 and 15.
• Charles Grinstead and Laurie Snell, Introduction to Probability (free PDF, GNU license). An older, gentle textbook with many computer simulations: discrete and continuous probability, the law of large numbers, the central limit theorem, Markov chains. A second view on lessons 01–06.
• Roman Vershynin, High-Dimensional Probability, 2nd edition (free PDF on the author's page). Concentration inequalities, sub-Gaussian and sub-exponential variables, random vectors and matrices. Advanced; read chapter 2 for lesson 07.

Statistics and machine learning #

• Kevin Murphy, Probabilistic Machine Learning: An Introduction (free PDF) and Advanced Topics (free PDF). The reference that connects every lesson of this track to machine learning: probability (ch. 2–3), statistics (ch. 4), decision theory and tests (ch. 5), linear regression (ch. 11); the second book has the exponential family, the Fisher information and Monte Carlo methods.
• Christopher Bishop, Pattern Recognition and Machine Learning (free PDF from Microsoft Research). Chapter 2 is the clearest treatment of the multivariate Gaussian (conditionals, marginals, linear Gaussian models); chapter 3 is Bayesian linear regression. Lessons 05 and 13.
• Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani, An Introduction to Statistical Learning, 2nd edition (free PDF). Chapter 3 is linear regression with standard errors, tests and diagnostics; chapter 5 is the bootstrap. Lessons 12 and 13.
• Bradley Efron and Trevor Hastie, Computer Age Statistical Inference (free PDF). Frequentist and Bayesian inference, the bootstrap, empirical Bayes and large-scale testing (false discovery rates), written by the inventor of the bootstrap. Lessons 10–12.
• Andrew Gelman et al., Bayesian Data Analysis, 3rd edition (free PDF from the authors). The standard reference for applied Bayesian statistics: conjugate models, hierarchical models, MCMC and its diagnostics. Lessons 10 and 15.
• Allen Downey, Think Bayes 2 (free online, with code). Bayesian statistics with Python: Bayes tables, grid posteriors, conjugate priors, A/B tests. Lessons 01 and 10.
• Art Owen, Monte Carlo theory, methods and examples (free chapters). Simple Monte Carlo, sampling from distributions, variance reduction, importance sampling, Markov chain Monte Carlo, with careful error analysis. The main reading for lessons 14 and 15.
• Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar, Foundations of Machine Learning, 2nd edition (free PDF from the authors). Chapter 2 (PAC learning) and appendix D (concentration inequalities). Lesson 07.
• Cosma Shalizi, The Truth About Linear Regression (free lecture notes). What each assumption of the linear model gives you and what to do when it fails. Lesson 13.
• Larry Wasserman, All of Statistics (official page). Not free, but a compact tour of the whole of mathematical statistics at the level of this track: convergence, the delta method, MLE, the bootstrap, tests, Bayesian inference. Find it in a library for lessons 06–12.

Short reads and tools #

• Seeing Theory (Brown University; online). Interactive pictures of basic probability, distributions, the central limit theorem, confidence intervals, Bayesian inference and regression. A good warm-up for lessons 01–06.
• Greenland et al., "Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations" (2016; free on PMC). A numbered list of common misreadings of p-values, intervals and power, each with a correction. Lessons 06 and 11.
• Tim Hesterberg, "What Teachers Should Know about the Bootstrap" (arXiv). How the bootstrap and permutation tests work, how many resamples you need, and when bootstrap intervals are inaccurate. Lesson 12.
• Evan Miller, "How not to run an A/B test" (online). Why peeking at the p-value breaks the significance level. Lesson 12.
• Deng, Xu, Kohavi and Walker, "Improving the sensitivity of online controlled experiments by utilizing pre-experiment data" (CUPED, 2013; PDF). Variance reduction with a pre-experiment covariate. Lesson 12. For the whole practice of online experiments, see the book Trustworthy Online Controlled Experiments (Kohavi, Tang and Xu; not free).
• Michael Betancourt, "A Conceptual Introduction to Hamiltonian Monte Carlo" (arXiv). Why HMC works in high dimension, explained with pictures of typical sets and phase space. Lesson 15.
• Radford Neal, "MCMC using Hamiltonian dynamics" (arXiv). The classic tutorial on HMC, with the leapfrog proofs and tuning advice. Lesson 15.
• Hoffman and Gelman, "The No-U-Turn Sampler" (arXiv). How NUTS chooses the trajectory length automatically. Lesson 15.
• Vehtari et al., "Rank-normalization, folding, and localization: an improved $\hat R$" (arXiv). The modern split-$\hat R$ and ESS diagnostics and the recommendations used in lesson 15.

Which lesson pairs with what #

The first choice for each lesson. The Watch and read section at the end of each lesson gives more, with exact section numbers. "Blitzstein–Hwang" is Introduction to Probability, "Murphy 1" and "Murphy 2" are the two Probabilistic Machine Learning books.

Lesson	Watch	Read
01 Probability and Bayes' rule	3Blue1Brown, Bayes' theorem and the medical test paradox; Stat 110 lecture 6	Blitzstein–Hwang ch. 1–2; Think Bayes 2 ch. 1–2
02 Random variables and expectation	Stat 110 lecture 9; 6.041 lecture 8	Blitzstein–Hwang ch. 3–5; Owen ch. 4 (inversion)
03 Common distributions	Stat 110 lectures 11 and 16; 6.041 lecture 15	Blitzstein–Hwang §4.7–4.8, §6.4–6.6, §8.3–8.5, ch. 13; Murphy 1 §2.4–2.7
04 Joint distributions and covariance	Stat 110 lectures 19 and 21; 6.041 lecture 12	Blitzstein–Hwang §7.1, §7.3, ch. 9; Murphy 1 §3.1
05 The multivariate Gaussian	3Blue1Brown, Gaussian + Gaussian; Stat 110 lectures 22 and 30	Bishop §2.3; Murphy 1 §3.2–3.3
06 LLN, CLT and confidence intervals	3Blue1Brown, central limit theorem; Stat 110 lecture 29; StatQuest, confidence intervals	Blitzstein–Hwang ch. 10; Owen ch. 2
07 Concentration inequalities	Stat 110 lecture 28; MIT "Hoeffding's inequality"	Vershynin §2.1–2.3, §2.6; Mohri et al. ch. 2
08 Maximum likelihood	StatQuest, maximum likelihood; 18.650 lectures 4–6	Murphy 1 §4.2; Murphy 2 §2.4 (exponential family)
09 Fisher information	Mutual Information, the Fisher information; 18.650 lecture 6	Ly et al., "A Tutorial on Fisher Information"; Murphy 2 §3.3
10 Bayesian inference	3Blue1Brown, probabilities of probabilities; 18.650 lecture 17; Statistical Rethinking	Murphy 1 §4.5–4.6; Think Bayes 2; Gelman et al. ch. 2
11 Hypothesis testing	StatQuest, p-values, power and FDR; 18.650 lecture 7	Greenland et al.; Efron–Hastie ch. 15 (large-scale testing)
12 Bootstrap, permutation tests and A/B tests	StatQuest, bootstrapping and power analysis; Stanford Statistical Learning 5.4	Hesterberg; Deng et al. (CUPED); Evan Miller
13 Linear regression	18.650 lectures 13–14; StatQuest, linear regression	ISL ch. 3; Bishop §3.1, §3.3; Shalizi
14 Monte Carlo and importance sampling	Mutual Information, importance sampling; Nando de Freitas	Owen ch. 2, 4, 8, 9; Murphy 2 ch. 11
15 Markov chains and MCMC	Stat 110 lectures 31–32; Ben Lambert, HMC; MCMC interactive gallery	Blitzstein–Hwang ch. 11–12; Owen ch. 11–12; Betancourt; Vehtari et al.