Probability and Statistics: Quick recap

01 Probability and Bayes' rule

With equally likely outcomes, $P(A) = |A|/|\Omega|$ ; count with the multiplication rule, $n!/(n-k)!$ ordered lists and $\binom{n}{k}$ subsets. Often it is easier to compute the complement.
The axioms ( $P \ge 0$ , $P(\Omega) = 1$ , countable additivity) imply the complement rule, inclusion–exclusion $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ and the union bound $P(\bigcup A_i) \le \sum P(A_i)$ .
Conditioning on $B$ keeps only the outcomes in $B$ and rescales: $P(A \mid B) = P(A \cap B)/P(B)$ . Law of total probability: $P(A) = \sum_i P(A \mid B_i)P(B_i)$ for a partition $B_1, \dots, B_n$ .
Bayes' rule: posterior = likelihood × prior / evidence, $P(H \mid E) = P(E \mid H)P(H)/P(E)$ . With a small prior, a good test can still give mostly false positives (base-rate effect).
Odds form: posterior odds = prior odds × likelihood ratio $P(E \mid H)/P(E \mid H^c)$ . Conditionally independent pieces of evidence multiply their likelihood ratios; dependent ones do not.
Independence: $P(A \cap B) = P(A)P(B)$ . Disjoint events with positive probabilities are dependent. Pairwise independence does not imply mutual independence (XOR).
Conditional independence and independence do not imply each other: a common cause makes conditionally independent events dependent, and a common effect (collider) makes independent events dependent (explaining away, selection bias).
Monty Hall: switching wins with probability 2/3 because the host's choice depends on the car. Simpson's paradox: pooled rates are weighted averages, and different weights can reverse a comparison.

02 Random variables and expectation

A random variable is a function $X : \Omega \to \mathbb{R}$ . Its CDF $F_X(x) = P(X \le x)$ describes its distribution, and $P(a < X \le b) = F_X(b) - F_X(a)$ .
A PMF gives probabilities of single values (bars); a PDF gives probability per unit length (areas are probabilities, heights can exceed 1, and $P(X = x) = 0$ ).
The expectation is the balance point: $\mathbb{E}[X] = \sum_x x\,p(x)$ or $\int x\,p(x)\,dx$ . LOTUS: $\mathbb{E}[g(X)] = \sum_x g(x)\,p(x)$ , without the distribution of $g(X)$ .
Linearity $\mathbb{E}[aX + bY + c] = a\mathbb{E}[X] + b\mathbb{E}[Y] + c$ needs no independence; with indicators ( $\mathbb{E}[I_A] = P(A)$ ) it gives the mean of any count.
$\operatorname{Var}(X) = \mathbb{E}[(X - \mu)^2] = \mathbb{E}[X^2] - \mu^2$ and $\operatorname{Var}(aX + b) = a^2\operatorname{Var}(X)$ ; in general $\mathbb{E}[g(X)] \ne g(\mathbb{E}[X])$ .
For a strictly monotone $g$ , $Y = g(X)$ has density $p_Y(y) = p_X(g^{-1}(y))\,|dg^{-1}(y)/dy|$ (probability is conserved); for a non-monotone $g$ , use the CDF method.
Inverse-CDF sampling: $F^{-1}(U)$ with $U \sim U(0, 1)$ has CDF $F$ (with the generalised inverse also for discrete $F$ ). PIT: $F(X) \sim U(0, 1)$ for a continuous $F$ , so PIT histograms check calibration (U-shape = too narrow).

03 Common distributions

Bernoulli → binomial (successes in $n$ trials), geometric (trials to the first success) and categorical ( $K$ classes); binomial with large $n$ , small $p$ and $np = \lambda$ → Poisson, with mean = variance = $\lambda$ .
Le Cam: a sum of independent $\text{Bernoulli}(p_i)$ is within total variation distance $\sum_i p_i^2$ of $\text{Poisson}(\sum_i p_i)$ ; for $\text{Bin}(n, p)$ the bound is $np^2$ .
$\text{Exp}(\lambda)$ (rate $\lambda$ , mean $1/\lambda$ ) is memoryless: $P(X > s + t \mid X > s) = P(X > t)$ . It and the geometric are the only memoryless distributions. The minimum of independent exponentials is exponential with the sum of the rates.
The MGF $M_X(t) = \mathbb{E}[e^{tX}]$ gives moments ( $M^{(n)}(0) = \mathbb{E}[X^n]$ ), multiplies for independent sums, and determines the distribution when it is finite near 0.
A sum of $n$ i.i.d. $\text{Exp}(\lambda)$ is $\text{Gamma}(n, \lambda)$ ; gammas with the same rate add their shapes.
$X/(X + Y)$ with independent $\text{Gamma}(a, 1)$ and $\text{Gamma}(b, 1)$ is $\text{Beta}(a, b)$ ; normalised independent gammas give a Dirichlet, whose marginals are $\text{Beta}(\alpha_i, \alpha_0 - \alpha_i)$ .
Poisson process with rate $\lambda$ : i.i.d. $\text{Exp}(\lambda)$ gaps, or independent $\text{Poisson}(\lambda t)$ counts in disjoint windows. Thinning with probability $q$ gives rate $q\lambda$ ; superposition adds rates.

04 Joint distributions and covariance

From a joint PMF or PDF: the marginal is a sum (or integral) over the other variable; the conditional is $p(x \mid y) = p(x, y)/p_Y(y)$ ; and $p(x, y) = p(x \mid y)\,p_Y(y)$ .
$X$ and $Y$ are independent when $p(x, y) = p_X(x)\,p_Y(y)$ everywhere, including where it is 0; a joint table is independent exactly when it has rank 1.
$\mathbb{E}[X \mid Y]$ is a random variable (a function of $Y$ ); the tower rule says $\mathbb{E}[\mathbb{E}[X \mid Y]] = \mathbb{E}[X]$ .
Law of total variance: $\operatorname{Var}(X) = \mathbb{E}[\operatorname{Var}(X \mid Y)] + \operatorname{Var}(\mathbb{E}[X \mid Y])$ , within plus between.
$\operatorname{Cov}(X, Y) = \mathbb{E}[XY] - \mathbb{E}[X]\,\mathbb{E}[Y]$ and $|\rho| \le 1$ ; independent implies uncorrelated, but not the reverse ( $X$ and $X^2$ ).
$\operatorname{Var}(\sum_i X_i) = \sum_i \operatorname{Var}(X_i) + 2\sum_{i<j}\operatorname{Cov}(X_i, X_j)$ , and $\operatorname{Cov}(AX + b) = A\Sigma A^\top$ .
Every covariance matrix is symmetric PSD, because $a^\top\Sigma a = \operatorname{Var}(a^\top X) \ge 0$ ; its eigenvectors are the PCA directions and its eigenvalues the variances along them.
The sample variance with $1/(n - 1)$ is unbiased, because $\mathbb{E}\sum_i (X_i - \bar X)^2 = (n - 1)\sigma^2$ .

05 The multivariate Gaussian

Change of variables: $p_Y(y) = p_X(g^{-1}(y))\,|\det J_{g^{-1}}(y)| = p_X(x)/|\det J_g(x)|$ ; mass is kept, so where the map stretches area the density drops.
Box–Muller: in polar coordinates two i.i.d. $\mathcal N(0, 1)$ give an independent uniform angle and $R^2 \sim \text{Exp}(1/2)$ ; run backwards, $\sqrt{-2\log(1 - U_1)}\,(\cos 2\pi U_2, \sin 2\pi U_2)$ gives two normals.
$x = \mu + Lz$ with $z \sim \mathcal N(0, I)$ and $LL^\top = \Sigma$ has the density $(2\pi)^{-d/2}(\det\Sigma)^{-1/2}\exp(-\tfrac12 (x - \mu)^\top\Sigma^{-1}(x - \mu))$ ; its contours are ellipses of constant Mahalanobis distance.
Affine maps of a Gaussian are Gaussian: $Ax + b \sim \mathcal N(A\mu + b, A\Sigma A^\top)$ ; a marginal keeps the matching blocks of $\mu$ and $\Sigma$ .
Uncorrelated jointly Gaussian variables are independent; Gaussian marginals alone are not enough ( $Y = WX$ with a random sign $W$ ).
With Cholesky $\Sigma = LL^\top$ : sample $\mu + Lz$ , $\log\det\Sigma = 2\sum_i \log L_{ii}$ , and the quadratic form is $\|L^{-1}(x - \mu)\|^2$ ; add jitter if the factorisation fails.
Gaussian conditioning: $x_a \mid x_b \sim \mathcal N\big(\mu_a + \Sigma_{ab}\Sigma_{bb}^{-1}(x_b - \mu_b),\ \Sigma_{aa} - \Sigma_{ab}\Sigma_{bb}^{-1}\Sigma_{ba}\big)$ , or in precision form covariance $\Lambda_{aa}^{-1}$ ; the conditional covariance does not depend on the observed $x_b$ .

06 LLN, CLT and confidence intervals

The sample mean $\bar X_n$ is a random variable with $\mathbb{E}[\bar X_n] = \mu$ and standard error $\sigma/\sqrt n$ (for uncorrelated data).
Markov: $P(X \ge a) \le \mathbb{E}[X]/a$ for $X \ge 0$ . Chebyshev: $P(|X - \mu| \ge \varepsilon) \le \sigma^2/\varepsilon^2$ .
Weak law of large numbers: $\bar X_n \to \mu$ in probability, proved by Chebyshev with $\operatorname{Var}(\bar X_n) = \sigma^2/n$ ; without a mean (Cauchy) there is no LLN.
Central limit theorem: for i.i.d. data with finite variance, $\sqrt n(\bar X_n - \mu)/\sigma \to \mathcal N(0, 1)$ in distribution; the error of this approximation is $O(1/\sqrt n)$ (Berry–Esseen).
A 95% confidence interval for a mean is $\bar x \pm 1.96\,\sigma/\sqrt n$ (known $\sigma$ ) or $\bar x \pm t_{n-1,\,0.975}\,s/\sqrt n$ (unknown $\sigma$ ); "95%" is the coverage of the procedure over repeated experiments.
The Wald interval for a proportion fails for small $n$ or rare events ( $n = 50$ , $p = 0.02$ : coverage 0.635); the Wilson interval fixes this.
A Monte Carlo estimate has the error bar $1.96\,s_f/\sqrt n$ in any dimension; the delta method gives $\operatorname{SE}(g(\bar X)) \approx |g'(\mu)|\,\sigma/\sqrt n$ .

07 Concentration inequalities

Chernoff bound: $P(S \ge t) \le \inf_{\lambda > 0} e^{-\lambda t}\,\mathbb{E}[e^{\lambda S}]$ , from Markov applied to $e^{\lambda S}$ ; for i.i.d. sums it decays exponentially in $n$ .
For Gaussian data the Chernoff bound is $e^{-nt^2/(2\sigma^2)}$ ; for Bernoulli( $p$ ) it is $e^{-n\,\mathrm{KL}(p + t \,\|\, p)}$ .
Hoeffding's inequality: for independent $X_i \in [a, b]$ , the mean satisfies $P(|\bar X_n - \mu| \ge \varepsilon) \le 2e^{-2n\varepsilon^2/(b - a)^2}$ .
A variable is sub-Gaussian with proxy $\sigma^2$ if $\mathbb{E}e^{\lambda(X - \mu)} \le e^{\lambda^2\sigma^2/2}$ for all $\lambda$ ; bounded and Gaussian variables are, exponential variables are not.
For values in $[0, 1]$ , Hoeffding guarantees error $< \varepsilon$ with probability $\ge 1 - \delta$ once $n \ge \ln(2/\delta)/(2\varepsilon^2)$ ; $n$ grows like $\ln(1/\delta)$ , Chebyshev's like $1/\delta$ .
Union bound: $P(\cup_j A_j) \le \sum_j P(A_j)$ ; for $k$ models at once Hoeffding needs only $\ln(2k/\delta)$ instead of $\ln(2/\delta)$ .
Finite-class generalisation bound: with probability $\ge 1 - \delta$ , $|R(h) - \hat R(h)| \le \sqrt{\ln(2k/\delta)/(2n)}$ for all $k$ models; choosing the best of $k$ on one test set inflates its score by up to this amount.

08 Maximum likelihood

The likelihood $L(\theta) = \prod_i p(x_i \mid \theta)$ is the probability of the observed data as a function of $\theta$ ; it is not a density over $\theta$ . Work with the log-likelihood $\ell(\theta) = \sum_i \log p(x_i \mid \theta)$ .
MLEs: Bernoulli $\hat p = \bar x$ ; Poisson $\hat\lambda = \bar x$ ; exponential $\hat\lambda = 1/\bar x$ ; Gaussian $\hat\mu = \bar x$ , $\hat\sigma^2 = \frac1n\sum_i (x_i - \bar x)^2$ .
Invariance: the MLE of $g(\theta)$ is $g(\hat\theta)$ for any one-to-one $g$ .
An estimator is a random variable: $\text{MSE} = \text{bias}^2 + \text{variance}$ . Unbiased is not the same as best; $D/n$ and $D/(n+1)$ beat $S^2 = D/(n-1)$ in MSE for Gaussian data.
The $1/n$ variance estimate has mean $\frac{n-1}{n}\sigma^2$ ; $1/\bar x$ for an exponential rate has mean $\frac{n}{n-1}\lambda$ .
Consistency: $\hat\theta_n \to \theta$ in probability; $\text{MSE} \to 0$ is enough. The MLE is consistent under regularity conditions.
Maximum likelihood minimises $\mathrm{KL}(p_{\text{data}} \,\|\, p_\theta)$ , the cross-entropy; mean squared error and cross-entropy losses are Gaussian and Bernoulli/categorical NLLs.
Exponential family $h(x)\exp(\eta^\top T(x) - A(\eta))$ : $\nabla A = \mathbb{E}[T]$ , $\nabla^2 A = \operatorname{Cov}[T]$ , the log-likelihood is concave in $\eta$ , and the MLE matches $\mathbb{E}_{\hat\eta}[T]$ to the sample average of $T$ .

09 Fisher information

The score $s(\theta; x) = \partial_\theta \log p(x \mid \theta)$ has mean 0 at the true $\theta$ (under regularity conditions).
Fisher information: $I(\theta) = \mathbb{E}[s^2] = \operatorname{Var}(s) = -\mathbb{E}[\partial_\theta^2 \log p]$ , the expected curvature of the log-likelihood; $n$ i.i.d. observations carry $nI(\theta)$ .
Examples: Bernoulli $1/(p(1-p))$ , Poisson $1/\lambda$ , exponential rate $1/\lambda^2$ , Gaussian mean $1/\sigma^2$ .
Cramér–Rao: an unbiased $T$ has $\operatorname{Var}(T) \ge 1/(nI(\theta))$ ; proof by $\operatorname{Cov}(T, s_n) = 1$ and Cauchy–Schwarz. Biased estimators can go below it.
The MLE is asymptotically normal and efficient: $\sqrt n(\hat\theta - \theta) \to \mathcal N(0, I(\theta)^{-1})$ .
Wald interval: $\hat\theta \pm 1.96/\sqrt{nI(\hat\theta)}$ , or with the observed information $-\ell_n''(\hat\theta)$ ; for several parameters use $\sqrt{[H^{-1}]_{kk}}$ .
The theory needs a support that does not depend on $\theta$ , an interior $\theta$ and a large enough $n$ ; Uniform( $0, \theta$ ) has error of order $1/n$ , and Wald intervals fail near $p = 0$ .

10 Bayesian inference

Bayes' rule for a parameter: posterior $\propto$ likelihood $\times$ prior, $p(\theta \mid \mathcal D) = p(\mathcal D \mid \theta)\,p(\theta)/p(\mathcal D)$ ; the evidence $p(\mathcal D)$ only normalises.
Beta–Bernoulli: prior Beta( $a, b$ ) and $k$ successes in $n$ trials give Beta( $a + k, b + n - k$ ); the prior acts as $a + b$ pseudo-observations, and sequential updates equal one batch update.
Gaussian–Gaussian (known $\sigma^2$ ): precisions add, $1/\tau_n^2 = 1/\tau_0^2 + n/\sigma^2$ , and the posterior mean is the precision-weighted average of $\mu_0$ and $\bar x$ .
MAP = regularised MLE: $\arg\min[-\ell(\theta) - \log p(\theta)]$ ; an L2 penalty is a Gaussian prior ( $\lambda = \sigma^2/\sigma_w^2$ ), an L1 penalty a Laplace prior. The MAP is not invariant under reparametrisation.
Shrinkage trades bias for variance: the Gaussian posterior mean beats $\bar x$ in MSE exactly when $(\mu - \mu_0)^2 < \sigma^2/n + 2\tau_0^2$ .
Every exponential family has a conjugate prior; Gamma( $a, b$ ) with Poisson counts gives Gamma( $a + \sum x_i, b + n$ ).
The posterior predictive $\int p(x^* \mid \theta)\,p(\theta \mid \mathcal D)\,d\theta$ adds parameter uncertainty to the noise, so it is wider than a plug-in; Gaussian: $\mathcal N(\mu_n, \sigma^2 + \tau_n^2)$ .
A credible interval is a probability statement about $\theta$ given the data and the prior; a confidence interval is a statement about the procedure. For large $n$ in regular models they agree (Bernstein–von Mises).

11 Hypothesis testing

A test fixes a null hypothesis $H_0$ , a statistic $T$ and a level $\alpha$ ; it rejects when the p-value $p = P_{H_0}(T \text{ at least as extreme as observed})$ is at most $\alpha$ .
Type I error = rejecting a true $H_0$ (probability at most $\alpha$ ); type II error = missing a real effect (probability $\beta_{\text{II}}$ ); power $= 1 - \beta_{\text{II}}$ .
For a z-test, power $= \Phi(\sqrt n\,\delta/\sigma - z_{1-\alpha})$ (one-sided): it depends only on $\lambda = \sqrt n\,\delta/\sigma$ , so twice the effect needs a quarter of the data.
Under a continuous $H_0$ the p-value is Uniform(0, 1); it is not $P(H_0 \mid \text{data})$ and not an effect size.
The t statistic $(\bar X_n - \mu_0)/(S/\sqrt n)$ has the $t_{n-1}$ distribution for Gaussian data; use a paired test for paired data and Welch's test for two independent groups.
For a $2 \times 2$ table, Pearson's $X^2$ equals the square of the pooled two-proportion $z$ .
Duality: a $(1 - \alpha)$ confidence interval is the set of null values that the level- $\alpha$ test does not reject.
For $m$ tests, Bonferroni ( $p_i \le \alpha/m$ ) controls the FWER under any dependence; Benjamini–Hochberg (largest $k$ with $p_{(k)} \le kq/m$ ) controls the FDR at $q\,m_0/m$ for independent tests.

12 Bootstrap and A/B tests

A permutation test shuffles the group labels; under exchangeability it is exact, and the Monte Carlo p-value $(1 + C)/(1 + B)$ satisfies $P(p \le \alpha) \le \alpha$ for every $B$ .
The bootstrap resamples the data with replacement to approximate the sampling distribution of a statistic; a resample contains about $1 - 1/e = 63\%$ of the distinct points.
The bootstrap standard error is the SD of the replicates, and the percentile interval uses their $\alpha/2$ and $1 - \alpha/2$ quantiles; check coverage by simulation.
The bootstrap fails for extremes (the maximum: coverage 0), heavy tails, dependent data (use a block or cluster bootstrap) and very small $n$ .
Users per arm of an A/B test: $n = 2\sigma^2(z_{1-\alpha/2} + z_{1-\beta_{\text{II}}})^2/\delta^2 \approx 16\sigma^2/\delta^2$ for $\alpha = 0.05$ and power 0.8.
Peeking at the p-value and stopping at the first $p < 0.05$ inflates the type I error (about 0.25 with 20 looks); fix $n$ in advance or use a sequential design.
A sample-ratio mismatch (a χ² test of the assignment counts) signals a bug; do not trust the result until it is explained.
CUPED, $Y - \theta(X - \bar X)$ with a pre-experiment covariate $X$ , reduces the variance by the factor $1 - \rho^2$ and stays unbiased; a covariate that the treatment can change biases it.

13 Regression: the statistics view

The linear model is $y = X\beta + \varepsilon$ ; OLS gives $\hat\beta = (X^\top X)^{-1}X^\top y$ (compute it with QR), and it is the Gaussian MLE.
$\hat\beta$ is random: $\mathbb{E}[\hat\beta] = \beta$ and $\operatorname{Cov}(\hat\beta) = \sigma^2(X^\top X)^{-1}$ ; in simple regression $\operatorname{Var}(\hat\beta_1) = \sigma^2/S_{xx}$ , so spread-out $x$ values give precise slopes.
$s^2 = \text{RSS}/(n - p)$ is unbiased for $\sigma^2$ , because $\operatorname{tr}(I - H) = n - p$ ; with Gaussian errors $(\hat\beta_j - \beta_j)/\operatorname{SE}_j \sim t_{n-p}$ .
The confidence interval for the mean response at $x_0$ uses $s\sqrt{h_0}$ and shrinks to 0 with more data; the prediction interval for a new $y$ uses $s\sqrt{1 + h_0}$ and tends to $\pm 1.96\sigma$ .
$R^2 = 1 - \text{RSS}/\text{TSS}$ never decreases when features are added; nested models are compared with the $F$ -test, and $F = t^2$ for one coefficient.
Gauss–Markov: under uncorrelated, equal-variance errors, OLS has the smallest covariance among linear unbiased estimators (BLUE).
Residual plots check the assumptions: curvature biases $\hat\beta$ , heteroscedastic or correlated errors make the classical standard errors wrong, and high-leverage points can hide in the residuals.
Bayesian linear regression with prior $\mathcal N(0, \alpha^{-1}I)$ has posterior $\mathcal N(m_N, S_N)$ with $S_N^{-1} = \alpha I + X^\top X/\sigma^2$ ; $m_N$ is ridge with $\lambda = \alpha\sigma^2$ , and the predictive variance is $\sigma^2 + x_0^\top S_N x_0$ .

14 Monte Carlo and importance sampling

Monte Carlo estimates $\mu = \mathbb{E}_p[f(X)]$ by $\frac1n\sum_i f(X_i)$ , with standard error $\sigma_f/\sqrt n$ in any dimension; report $\hat\mu \pm 2s_f/\sqrt n$ .
For a rare event of probability $\mu$ , the relative error is about $1/\sqrt{n\mu}$ : you need about $100/\mu$ draws for 10% accuracy.
Rejection sampling with an envelope $\tilde p \le Mq$ gives exact draws from $p = \tilde p/Z$ with acceptance rate $Z/M$ ; it needs a proposal with tails at least as heavy as the target, and fails in high dimension.
Importance sampling: $\mu = \mathbb{E}_q[f\,p/q]$ ; the estimator is unbiased, its variance is $\frac1n\big(\int f^2p^2/q - \mu^2\big)$ , and the best proposal is $q \propto |f|p$ .
A proposal lighter-tailed than the target gives infinite-variance weights: unbiased, but with no valid error bar, and the failure is silent.
Self-normalised IS divides by $\sum_i w_i$ , needs $p$ only up to a constant, is biased by $O(1/n)$ but consistent; $\frac1n\sum_i w_i$ is an unbiased estimate of $Z$ . Compute it with log-weights.
The effective sample size $(\sum_i w_i)^2/\sum_i w_i^2$ diagnoses weight collapse; a small value proves trouble, a large value does not prove success.
A control variate $g$ with known mean reduces the variance by the factor $1 - \rho^2$ at $\beta^* = \operatorname{Cov}(f, g)/\operatorname{Var}(g)$ ; antithetic pairs give the factor $1 + \rho(h(U), h(1 - U))$ and help for monotone $h$ .

15 Markov chains and MCMC

A finite Markov chain evolves as $\mu_{t+1} = \mu_t P$ ; a stationary distribution solves $\pi P = \pi$ , $\sum_i \pi_i = 1$ (a left eigenvector of $P$ for eigenvalue 1).
If the chain is irreducible and aperiodic, $\mu_0 P^t \to \pi$ from any start, with error roughly $|\lambda_2|^t$ ; irreducibility alone gives the ergodic theorem, $\frac1n\sum_t f(X_t) \to \mathbb{E}_\pi[f]$ .
Detailed balance $\pi_i P_{ij} = \pi_j P_{ji}$ implies $\pi P = \pi$ ; it is sufficient, not necessary.
Metropolis–Hastings accepts $x'$ with probability $\min\big(1, \frac{\tilde\pi(x')\,q(x \mid x')}{\tilde\pi(x)\,q(x' \mid x)}\big)$ and records $x$ again on a rejection; it satisfies detailed balance and needs only ratios of $\tilde\pi$ , so the normalising constant is never needed.
Gibbs sampling draws each coordinate from its full conditional; it is Metropolis–Hastings with acceptance 1, and for a bivariate Gaussian with correlation $\rho$ each coordinate is AR(1) with coefficient $\rho^2$ .
HMC adds a momentum $p \sim \mathcal N(0, I)$ , follows $H = -\log\tilde\pi(x) + \tfrac12\|p\|^2$ with leapfrog steps (volume preserving, reversible, energy error $O(\varepsilon^2)$ ), and accepts with probability $\min(1, e^{-\Delta H})$ .
ESS $= n/\tau$ with $\tau = 1 + 2\sum_{t \ge 1}\rho_t$ ; for AR(1) with coefficient $\varphi$ , $\text{ESS}/n = (1 - \varphi)/(1 + \varphi)$ .
Split- $\hat R$ compares between-half-chain and within-half-chain variance over at least 4 chains; trust results only with $\hat R < 1.01$ , total ESS above 400 and no divergences, and remember that chains stuck in the same mode still agree.