Probability and Statistics · 71 figures

Interactive figures

Every figure in the track. Each one is a small experiment: drag, slide, and watch the numbers. The link under each figure takes you to the lesson that explains it.

Lesson 1 · Probability and Bayes' rule

Press "Condition on B". Everything outside B (blue) fades, and $B$ is stretched to fill the whole square: it is the new sample space. The share of the new square covered by $A \cap B$ (aqua) is $P(A \mid B)$ . Compare it with the share of the old square covered by A (violet), which is $P(A)$ . Then drag the corner dots, and try the presets: in "independent", $A$ covers the same share of $B$ as of $\Omega$ . In "disjoint", $P(A \mid B) = 0$ .

In context: Lesson 1

First set your guess, then press "Test everyone". Each small square is one of 10,000 people. Aqua squares are sick people who test positive; faint aqua squares are sick people the test misses. Magenta squares are healthy people with a positive test (false alarms). $P(\text{sick} \mid +)$ is the aqua count divided by aqua plus magenta. Raise the prevalence to 20% and watch the answer change, with the same test. Then press "Second test": only the positives are tested again.

In context: Lesson 1

Add test results with + and −. The axis is the log-odds, with probabilities written on top. The violet dot is the prior. Each aqua arrow is one update: every independent positive is a step of the same length, $\log_{10} \text{LR}_+$ , to the right. Now raise the dependence $\rho$ : with probability $\rho$ the second test only copies the first (same blood sample, same cable). Repeated results then carry less evidence. The magenta dashed track is the naive update that still multiplies the full likelihood ratios, so it counts the same evidence twice.

In context: Lesson 1

Lesson 2 · Random variables and expectation

Move $a$ and $b$ , then switch the distribution. The top panel shows the distribution in orange: bars for the sum of two dice, a curve for the exponential and the normal. The probability of the interval $(a, b]$ is yellow: the sum of the shaded bars, or the shaded area. In the CDF below, the same number is the rise of the curve from $F(a)$ to $F(b)$ (the thick yellow bar). For the exponential with rate 2, notice that the curve starts at height 2: a density value is not a probability.

In context: Lesson 2

Predict where the beam balances, then drag the dark triangle (the fulcrum) there. The orange discs are the probabilities $p_i$ , placed at the values $x_i$ . The readout shows the torque $\sum_i p_i (x_i - f)$ ; the beam is level when it is 0. Press "Show the mean" to see $\mathbb{E}[X]$ (yellow) and the band $\mathbb{E}[X] \pm \sigma$ (violet). Then try "one far mass": a mass of 0.1 at $x = 10$ moves the balance point from 1 to 1.7 and makes the variance much larger.

In context: Lesson 2

Move the level $u$ , then press "Draw 500". The yellow path goes from $u$ on the vertical axis across to the orange CDF and down to $x = F^{-1}(u)$ . Each draw uses a new uniform $u$ . The blue histogram of the samples matches the orange density below. Notice that for $\text{Exp}(1)$ the CDF is steepest at $x = 0$ , and most samples land there. Try "discrete": the CDF is a staircase, and each step catches the levels $u$ that belong to one value.

In context: Lesson 2

Move the interval, then switch the map $g$ . $X$ is uniform on $(0, 1)$ . The yellow interval $I$ on the $x$ -axis holds probability equal to its width. The map $g$ (orange curve) sends it to the yellow interval $g(I)$ on the $y$ -axis, which holds the same probability. Where $g(I)$ is wide, the density of $Y$ is low; where it is narrow, the density is high. The blue histogram of simulated values of $g(X)$ matches the orange formula for $p_Y$ . For $g(x) = x^2$ , the values pile up near 0.

In context: Lesson 2

Lesson 3 · Common distributions

Choose a family and move its parameters, then press "Sample 1000". The orange bars or curve are the formula (PMF or PDF). The dark vertical line marks the mean, and the bar at the top spans mean ± one standard deviation. The blue histogram shows 1000 samples; compare the sample mean and variance with the formulas in the side panel. Try the gamma with $k = 1$ (it is the exponential), and the beta with $a = b = 1$ (it is the uniform distribution).

In context: Lesson 3

Increase the number of trials $n$ , with the mean $\lambda = np$ fixed. The blue bars are $\text{Bin}(n, \lambda/n)$ ; the orange bars are $\text{Poisson}(\lambda)$ . With $n = 10$ the binomial is visibly narrower. As $n$ grows, the bars agree. The side panel shows the total variation distance (the largest possible difference between the probabilities of any event) and Le Cam's bound $np^2 = \lambda^2/n$ : both shrink like $1/n$ .

In context: Lesson 3

Press "New run", then make the run longer. Blue ticks on the timeline are events at rate $\lambda$ . The left histogram collects the gaps between events, the right one the counts in windows of length 1. The orange curves are the formulas $\text{Exp}(\lambda)$ and $\text{Poisson}(\lambda)$ . Then lower the keep probability $q$ : each event is kept (aqua) or removed (magenta) by an independent coin. The kept events again form a Poisson process, with rate $q\lambda$ (thinning).

In context: Lesson 3

Drag "already waited $s$ ", then change the lifetime family. The orange curve is the survival curve $P(X > t)$ of a new item, with mean lifetime 10 years. The yellow part is the curve beyond $s$ . Rescaled by $1/P(X > s)$ and moved back to time 0, it becomes the aqua curve: the survival curve of an item that has already lasted $s$ . For the exponential, aqua and orange are the same curve. For a gamma with $k = 3$ (wear-out), an item that lasted 10 years survives 10 more with probability 0.146 instead of 0.423 for a new one. Press "Simulate" to count 10,000 simulated lifetimes (blue).

In context: Lesson 3

Move the three $\alpha$ sliders, or try the presets. Each point is one probability vector $(x_1, x_2, x_3)$ , drawn by normalising three independent gammas; each corner is one class with probability 1. Blue points are samples, and the orange dot is the mean $\alpha/\alpha_0$ . With all $\alpha = 1$ the points fill the triangle evenly. With all $\alpha = 0.2$ they crowd into the corners and edges (sparse vectors: most mass on one or two classes). With all $\alpha = 5$ they gather near the centre. The side panel compares the variance and covariance with the formulas.

In context: Lesson 3

Lesson 4 · Joint distributions and covariance

Click a column to see its conditional distribution. The cells are the joint PMF (blue, darker = more mass); the bars on the right and at the bottom are the marginals. The side panel compares the conditional $p(x \mid y)$ with the marginal $p_X(x)$ . Click a cell and move the slider to change its mass (the other cells rescale so the total stays 1). Press Make independent: the table becomes $p_X\,p_Y^\top$ , every column gets the same shape and the rank drops to 1.

In context: Lesson 4

Move the sliders and watch the bar. Three classes with their own scores (blue dots) and their own means (orange marks). The bar splits $\operatorname{Var}(X)$ into the within-class part $\mathbb{E}[\operatorname{Var}(X \mid Y)]$ and the between-class part $\operatorname{Var}(\mathbb{E}[X \mid Y])$ . Set the gap between the means to 0: the between part disappears. New sample draws new students; the sample variance (black tick) stays close to the exact total.

In context: Lesson 4

Drag the yellow handle to turn the direction $u$ , and watch the variance of the projections $u^\top x$ . The samples are blue; the orange ellipses are 1 and 2 standard deviations of $\Sigma$ , and the orange arrows are the eigenvectors scaled by $\sqrt{\lambda_k}$ . The projected variance $u^\top \Sigma u$ is largest ( $= \lambda_1$ ) when $u$ points along the long axis, and smallest ( $= \lambda_2$ ) along the short axis. Set $\rho = 0.8$ and $\sigma_1 = \sigma_2$ to get the two-thermometer matrix up to scale.

In context: Lesson 4

Lesson 5 · The multivariate Gaussian

In context: Lesson 5

Press Play: z → x to watch the round cloud of $z$ become the ellipse of $x = \mu + Lz$ . Then change $\sigma_1, \sigma_2, \rho$ . The blue points are samples; the orange ellipses are Mahalanobis distance 1 and 2; the violet arrows are the columns of $L$ . Drag the yellow point: its Euclidean distance and its Mahalanobis distance can be very different. In 2-D, about 39% of the samples fall inside distance 1 and about 86% inside distance 2.

In context: Lesson 5

Press Draw 1000 more a few times and compare the two cases. In " $Y = WX$ ", both histograms of $X$ and $Y$ (blue) match the orange $\mathcal N(0, 1)$ curve, and the correlation is about 0. But the scatter is an X shape, and the histogram of $X + Y$ has a spike at 0 that holds half of the samples, and the correlation of $|X|$ and $|Y|$ is exactly 1. In "independent $Y$ ", $X + Y$ follows the $\mathcal N(0, 2)$ curve. In both cases $\operatorname{Var}(X + Y) \approx 2$ : the variance cannot see the difference.

In context: Lesson 5

Drag the yellow handle at the left edge up and down to set the observation $y$ of $x_2$ , and move the noise slider $r$ . The sensor reads $y = x_2 + e$ with noise $e \sim \mathcal N(0, r^2)$ . The violet ellipses are the prior of $(x_1, x_2)$ ; the blue band is the observation $y \pm r$ ; the aqua ellipses are the posterior (the conditional given the observation). The curves at the bottom are the distribution of $x_1$ before (violet) and after (aqua). $x_1$ is never observed, but its uncertainty shrinks because it is correlated with $x_2$ . Drag the handle: only the centre moves, the size stays. Set the noise to 0: the posterior collapses onto the line $x_2 = y$ , and the readouts become the 2-D formulas of section 2.

In context: Lesson 5

Lesson 6 · LLN, CLT and confidence intervals

Press "Draw 1,000" several times, then change $n$ and draw again. The top chart shows the population density (orange) and the latest sample (blue dots). The bottom chart is the histogram of all sample means (blue) with the normal curve $\mathcal N(\mu, \sigma^2/n)$ (orange). Notice: the observed SD of the means matches $\sigma/\sqrt n$ , and for $n \ge 20$ the histogram is close to a bell even for the skewed population.

In context: Lesson 6

Choose a base distribution and increase $n$ . The blue histogram shows 20,000 simulated values of $Z_n$ ; the orange curve is $\mathcal N(0, 1)$ . The KS distance (the largest gap between the two CDFs) shrinks as $n$ grows. Notice: the uniform and the die look Gaussian already at $n = 5$ , the exponential needs about $n = 30$ , and Bernoulli(0.1) is still lumpy at $n = 30$ . Then choose Cauchy: the mean of $n$ Cauchy values never gets narrower.

In context: Lesson 6

Lesson 7 · Concentration inequalities

Move $n$ and $\varepsilon$ and compare the curves on the log scale. Blue is the exact binomial tail, dashed orange the CLT approximation. The bounds: Markov (grey), Chebyshev (violet), Hoeffding (aqua) and Chernoff (magenta). Notice: at $n = 500$ Chebyshev still gives 0.05 while the truth is about $4 \times 10^{-6}$ ; the exponential bounds stay parallel to the truth. Then set $p = 0.1$ : Chernoff becomes much better than Hoeffding, because it uses the small variance of a rare event.

In context: Lesson 7

Move $\lambda$ to the lowest point of the orange curve. The curve is the bound $e^{-\lambda t}\,\mathbb{E}[e^{\lambda S}]$ for each $\lambda$ ; every point on it is a valid bound. The yellow dot is your $\lambda$ , the aqua dot the best one, $\lambda^*$ , and the blue line the exact tail. For the Gaussian, $\lambda^* = t$ and the bound is $e^{-t^2/2}$ . Try "Exp(1)" with $a = 2$ : the best Chernoff bound (0.74) is worse than Markov (0.5). Chernoff is strong for large deviations and for sums, not everywhere.

In context: Lesson 7

Choose $\varepsilon$ and $\delta$ , then press Simulate. The widget runs 1,000 experiments of size $n$ and plots their errors $\bar X - \mu$ (blue); errors beyond the orange lines $\pm\varepsilon$ are magenta. With the Hoeffding $n$ , the failure rate is always below $\delta$ , usually far below (it is a worst case over all distributions on $[0, 1]$ ). Switch "Use n from" to CLT: for the fair coin the failure rate moves up to about $\delta$ .

In context: Lesson 7

Press "Run 50 experiments" and change $n$ . $k$ classifiers guess at random, so each has true accuracy 0.5. The blue curve is the best test accuracy among the first $k$ ; it grows with $k$ . The dashed orange curve is the bound $0.5 + \sqrt{\ln(2k/\delta)/(2n)}$ with $\delta = 0.05$ ; the blue curves almost never cross it. The aqua dot is the selected classifier on a fresh test set: back near 0.5. The high test score was selection, not skill.

In context: Lesson 7

Lesson 8 · Maximum likelihood

Flip the coin and watch the curve. Press Flip 10 a few times. The blue flips are the data. The top panel is the orange likelihood $L(p)$ , divided by its maximum so that the peak is at 1; the bottom panel is the log-likelihood $\ell(p) - \ell(\hat p)$ . The yellow line marks the MLE $\hat p = k/n$ , the dashed grey line the true $p$ of the coin. Notice that the peak gets narrower as $n$ grows: more data pin $p$ down. The readout gives the curvature $-\ell''(\hat p)$ ; lesson 09 turns it into an error bar.

In context: Lesson 8

Move $\mu$ and $\sigma$ to make the total cost small. The blue points are the data; the orange curve is the Gaussian model, and each stem shows the density $p(x_i)$ of a point. The magenta bars below are the costs $-\log p(x_i)$ ; the readout adds them up to the negative log-likelihood. Try a tiny $\sigma$ : the points near $\mu$ become cheap, the far ones very expensive. Press Jump to MLE and compare with your best guess.

In context: Lesson 8

Throw darts: draw samples and compare three estimators. Each press of Draw 500 simulates 500 samples of size $n$ from $\mathcal N(0, 1)$ , so the true variance is $\sigma^2 = 1$ (dashed orange line). Each blue dot is one estimate; the black tick is the average of the dots in its row. The table compares the simulated bias, variance and MSE with the exact formulas. Notice that the unbiased row is centred on 1 but has the widest spread. Then move $n$ : all three rows shrink toward 1 (consistency), and their differences vanish.

In context: Lesson 8

Fit a gamma by matching two averages. The blue histogram is a data set of 200 positive numbers; the orange curve is the gamma density with your shape $k$ and rate $\beta$ . The two readout rows compare the model's $\mathbb{E}[\log X]$ and $\mathbb{E}[X]$ with the data's averages of $\log x$ and $x$ . Move the sliders until both rows match, then press Jump to MLE: the MLE is the one point where both match, and there the average log-likelihood is largest.

In context: Lesson 8

Lesson 9 · Fisher information

Change $n$ and watch the peaks. Each orange curve is the log-likelihood $\ell(\theta) - \ell(\hat\theta)$ of one simulated data set (8 data sets from the same true $\theta$ , dashed grey). Each blue dot is that data set's MLE. As $n$ grows the curves get sharper, and the dots crowd closer to the truth. The readout compares the spread of the MLEs with $1/\sqrt{nI(\theta)}$ . Switch to the coin: there the curvature also depends on $p$ .

In context: Lesson 9

Evaluate the score at a guess $\theta_0$ . The blue histogram shows the score of the whole sample, $s_n(\theta_0) = \sum_i s(\theta_0; X_i)$ , over 2000 simulated samples. The orange curve is $\mathcal N(0, nI(\theta))$ , the theory at the truth. With $\theta_0$ equal to the true value, the histogram is centred at 0 and its variance matches $nI$ . Move $\theta_0$ away from the truth: the histogram shifts. The sign of the score says in which direction the truth lies.

In context: Lesson 9

Move $p$ and compare the two curves. The orange curve (top) is the information $I(p) = \frac{1}{p(1-p)}$ of one flip. The bottom panel shows $\operatorname{Var}(\hat p) = \frac{p(1-p)}{n}$ (orange) and the variance of $\hat p$ in 2000 simulated experiments (blue dot, press Simulate). Where one curve is low, the other is high: they are reciprocals, up to the factor $n$ . The readout also shows the relative error $\operatorname{SE}/p$ , which grows near $p = 0$ .

In context: Lesson 9

Simulate many data sets and look at the error of the MLE. The blue histogram shows $\sqrt n(\hat\theta - \theta)$ over 4000 simulated data sets; the orange curve is the limit $\mathcal N(0, 1/I(\theta))$ . For the exponential rate at small $n$ the histogram is skewed (the MLE $1/\bar x$ is biased); increase $n$ and it becomes Gaussian. For Uniform( $0, \theta$ ) the regularity condition (R1) fails: the histogram sits on one side of 0 and collapses as $n$ grows, because the error shrinks like $1/n$ , not $1/\sqrt n$ .

In context: Lesson 9

Check the promise "95%". The orange curve is the exact coverage of the Wald interval $\hat p \pm 1.96\sqrt{\hat p(1 - \hat p)/n}$ for every true $p$ (computed by summing the binomial probabilities); the aqua curve is the Wilson interval. The bottom panel draws 30 simulated intervals at your $p$ : magenta ones miss the truth. Move $p$ close to 0: the Wald coverage collapses, because with $\hat p = 0$ the interval is $[0, 0]$ . Increase $n$ and the problem moves closer to the edge.

In context: Lesson 9

Lesson 10 · Bayesian inference

Flip the coin (or click Heads and Tails) and watch the posterior. The violet curve is the prior Beta( $a, b$ ), the orange curve the likelihood (rescaled to area 1), and the aqua curve the posterior, with its 95% credible interval shaded. The markers show the MLE, the MAP and the posterior mean. The posterior always lies between the prior and the likelihood. Make the prior strong ( $a + b$ large) and see how many flips it takes to move it. Set $a = b = 1$ (uniform prior): then the MAP equals the MLE.

In context: Lesson 10

Move the sliders and compare the three curves. Violet: the prior $\mathcal N(\mu_0, \tau_0^2)$ . Orange: the likelihood of $\mu$ , which has the shape $\mathcal N(\bar x, \sigma^2/n)$ . Aqua: the posterior. The bar below shows the precisions: prior plus data equals posterior. Increase $n$ : the data precision $n/\sigma^2$ grows and the posterior moves to $\bar x$ . Make the prior wide: it stops mattering.

In context: Lesson 10

Compare three estimators of a coin's $p$ at every true $p$ . The curves are exact MSEs (sums over all outcomes $k$ ): the orange MLE $k/n$ , the aqua MAP $\frac{k + a - 1}{n + a + b - 2}$ and the dashed aqua posterior mean $\frac{k + a}{n + a + b}$ . The violet line marks the prior mean. Where the shading is aqua, the MAP beats the MLE. Press Simulate to check the three MSEs at the yellow $p$ with 4000 experiments (blue dots). A strong prior wins big near its mean and loses far from it; a larger $n$ makes all curves agree.

In context: Lesson 10

Run a Bayesian A/B test. Set the visitors and conversions of each variant. The top panel shows the posteriors of $p_A$ (blue) and $p_B$ (aqua) with a uniform prior; the bottom panel the histogram of $p_B - p_A$ from 4000 posterior draws, aqua where B wins and magenta where A wins. The readout gives $P(p_B > p_A \mid \mathcal D)$ by sampling (with its Monte Carlo error) and by numerical integration, and the expected loss of each choice. Multiply both sample sizes by 10 with the same rates and see the decision become clear.

In context: Lesson 10

Lesson 11 · Hypothesis testing

Press "Repeat 1,000 times" to flip a fair coin 100 times, again and again. This is the world where $H_0$ is true. The violet bars show how often each number of heads appears; the orange marks are the exact binomial probabilities. The magenta bars are the runs at least as far from 50 as the observed count (yellow line). Their share is the p-value: it approaches the exact 0.0352. Move the observed count to 55 or 70 and predict the p-value first. The second view, Many p-values, runs many tests and shows the histogram of their p-values.

In context: Lesson 11

Move the sliders for the effect $d = \delta/\sigma$ and the sample size $n$ . The violet curve is the null distribution of $z$ , the orange curve the distribution when the effect is real. The magenta area is $\alpha$ , the aqua area is the power, and the light orange area is $\beta_{\text{II}}$ . The lower chart shows the power as a function of $n$ . The start values are the coin: $d = 0.1/0.5 = 0.2$ , $n = 100$ . Notice that only $\lambda = \sqrt n\, d$ moves the orange curve: to double $\lambda$ you need four times as much data.

In context: Lesson 11

Slide the degrees of freedom $\nu$ and compare the t density (orange) with the normal density (violet, dashed). The solid orange lines are $\pm t_{\nu, 1-\alpha/2}$ , the dashed violet lines $\pm z_{1-\alpha/2}$ . Press Simulate to draw 2,000 samples of size $n = \nu + 1$ from a normal distribution and compute $T = \bar x/(s/\sqrt n)$ for each: the blue histogram follows $t_\nu$ , not $\mathcal N(0, 1)$ . With $\nu = 9$ , about 8.2% of the $T$ values fall beyond $\pm 1.96$ (magenta tails), not 5%. This is why small samples need $t$ quantiles.

In context: Lesson 11

Switch between the three rules and watch which p-values are rejected. Each dot is one test, sorted by its p-value (log scale). Aqua dots are real effects, violet dots are true nulls, and magenta dots are false discoveries. The orange line is the threshold of the rule: $\alpha$ , $\alpha/m$ , or the BH line $iq/m$ ; everything left of the grey line is rejected. Press Run 500 studies: without a correction the FWER is close to 1; Bonferroni keeps it below 0.05 but finds few real effects; BH keeps the average FDP near $q\,m_0/m$ and finds many more.

In context: Lesson 11

Lesson 12 · Bootstrap and A/B tests

Press "Shuffle labels once" a few times: the blue dots move to a random new split, and the orange marks show the two group means. Each shuffle adds one value to the violet histogram of the shuffled statistic $T^*$ . Then press All splits: the histogram shows all 20 splits, and the magenta bars beyond the observed $T$ (yellow) give the exact p-value. Switch to 20 users (time to finish an old and a new sign-up form): there are 184,756 splits, so the widget samples 1,000 of them and uses $p = (1 + C)/(1 + B)$ .

In context: Lesson 12

Press "One resample" and count: the blue dots are the 20 data points; the small dots above each point show how often it was drawn (aqua = once, yellow = twice or more), and faded points were not drawn. Then press 1,000 resamples: the aqua histogram is the bootstrap distribution, and the orange lines are the 95% percentile interval. The dashed line is the true $\theta$ , which we know here because we generated the data. Check coverage repeats everything on 200 new data sets. Now switch to the maximum: the interval turns magenta, because it can never contain $\theta$ .

In context: Lesson 12

Move the MDE slider and watch the users per arm. The orange curve is the power as a function of $n$ (log scale); the yellow lines mark the planned $n$ and the target power. Halve the MDE: $n$ grows about 4 times, because $n \propto 1/\delta^2$ . Raise the baseline rate toward 50%: the variance $p(1-p)$ grows, so $n$ grows too. Simulate runs 1,000 whole A/B tests at the planned $n$ ; the magenta dot is the share that reject, close to the target power.

In context: Lesson 12

Press "Run 500 experiments" and compare the two histograms. The scatter shows one experiment: control users (blue) and treatment users (aqua), with the common slope $\theta$ . The violet histogram is the plain difference in means, the aqua histogram the CUPED estimate: same centre $\tau = 0.1$ , but narrower by about $\sqrt{1 - \rho^2}$ . Move $\rho$ to 0 and to 0.9. Then switch the covariate to in-experiment Z, a value that the treatment changes: the magenta histogram is centred at a negative effect.

In context: Lesson 12

Lesson 13 · Regression: the statistics view

Press "New data set" a few times, then "Add 100 data sets". The orange line is the true line. Each data set gives a fitted line: the newest is yellow, earlier ones are faint blue. The histograms collect $\hat\beta_0$ and $\hat\beta_1$ over all data sets; the orange curves are the theory $\mathcal N(\beta, \sigma^2(X^\top X)^{-1})$ from section 3. Make $n$ larger or $\sigma$ smaller and watch the fan and the histograms get narrower.

In context: Lesson 13

Move the correlation slider from 0 to 0.99. Each blue dot is $(\hat\beta_1, \hat\beta_2)$ from one data set with the same features and new noise; the orange ellipse is the 95% region of $\mathcal N(\beta, \sigma^2(X^\top X)^{-1})$ around the true $\beta = (1, 1)$ . With correlated features the ellipse grows along the direction $\beta_1 - \beta_2$ and tilts: the estimates are negatively correlated. The sum $\beta_1 + \beta_2$ (the short axis) stays precise.

In context: Lesson 13

Lesson 14 · Monte Carlo and importance sampling

In context: Lesson 14

Try the four presets, and press "New seed" several times for each. The target $p$ is orange and the proposal $q$ is violet. The lower plot shows the normalised weight of each of the 200 draws as a blue stick; the largest is yellow, and the dashed line is the equal weight $1/n$ . The effective sample size (ESS) says how many equally weighted draws the weighted sample is worth. With the "too narrow" proposal, most runs look calm, and then one draw takes most of the weight.

In context: Lesson 14

Press "+500", then move $M$ below 1.65. The target $\tilde p(x) = e^{-x^2/2}$ ( $x \ge 0$ ) is orange, the envelope $M e^{-x}$ is violet. Proposals under $\tilde p$ are accepted (aqua), the others rejected (magenta); "Step" shows one proposal in yellow. The histogram compares the accepted samples with the target density. Below $M = \sqrt e \approx 1.649$ a magenta region appears where the envelope is too low, and the histogram misses mass near $x = 1$ .

In context: Lesson 14

Press "Run again" several times, then move the threshold $a$ . The chart shows running estimates divided by the true value, so the target is the orange line at 1. Plain Monte Carlo (blue) is 0 until its first hit, then jumps. Importance sampling with $q = \mathcal N(a, 1)$ (aqua) settles near 1 after a few hundred draws. Dashed lines are $1 \pm 2$ relative standard errors from the theory. At $a = 4$ the variance ratio is about 7,000.

In context: Lesson 14

Move the slider $\beta$ to the bottom of the parabola, then switch "Antithetic pairs" on. The problem is $\ln 2 = \mathbb{E}[1/(1 + U)]$ with the control $g(U) = U$ . Top: draws $(u, h(u))$ in blue, the line with slope $\beta$ in yellow, the least-squares slope $\beta^*$ dashed in orange. Bottom: the variance factor of the control-variate estimator as a function of $\beta$ ; its minimum $1 - \rho^2$ is at $\beta^*$ . The aqua line is the antithetic factor $1 + \rho$ . Then try $h(u) = (u - \frac12)^2$ : the control is useless and the antithetic pairs double the variance. "Check by simulation" measures both factors.

In context: Lesson 14

Lesson 15 · Markov chains and MCMC

Press Step and watch the blue bars $\mu_t$ move toward the orange marks $\pi$ . The arrows are the transition probabilities, and the size of each circle is $\mu_t$ . The lower chart shows the distance to $\pi$ on a log scale; it falls like $|\lambda_2|^t$ (dashed orange line). Edit the matrix or try the presets. Flip never converges (period 2), and Cycle has a stationary distribution but fails the detailed-balance check: its mass circulates in one direction.

In context: Lesson 15

Press Step to draw a momentum and follow the violet leapfrog path. The end point is aqua if it is accepted and magenta if it is rejected. The readout shows the energy error $\Delta H$ . Increase the step size $\varepsilon$ past about 0.45 on the correlated Gaussian: the path explodes (a divergence) and every proposal is rejected. Run 100 also runs random-walk Metropolis with the same number of gradient-or-density evaluations; compare the two effective sample sizes.

In context: Lesson 15

Choose a preset. Four random-walk Metropolis chains (four colours) start far apart. The first half of each chain is warm-up (shaded) and is dropped. Well mixed: the traces overlap like noise, $\hat R \approx 1$ , and the autocorrelations die fast. Too small step: the traces still remember their starts, and the autocorrelation decays slowly. Stuck in one mode: each chain looks fine alone, but the chains disagree, so $\hat R$ is large. All in the same mode: the diagnostics pass, but half of the target is missing. Then make the chains longer: which problems go away?

In context: Lesson 15