Causal Inference: Quick recap

01 Correlation vs causation

An association ( $P(y \mid x)$ depends on $x$ ) can come from causation, reverse causation, a common cause (confounding), selection on a common effect (collider, Berkson), or chance. One scatter plot cannot tell them apart.
A confounder causes both the treatment and the outcome. In $T := aZ + \varepsilon_T$ , $Y := bT + cZ + \varepsilon_Y$ the pooled slope is $b + ca/(a^2 + 1)$ ; adjusting for $Z$ gives $b$ .
A pooled rate is a weighted average of the group rates, $p_A = (1 - s_A)a_s + s_A a_\ell$ , with weights given by the case mix.
Simpson's paradox: a comparison has one direction in every group and the opposite direction pooled. It needs a different case mix in the compared groups.
In the kidney-stone data A wins for small (93.1% vs 86.7%) and large stones (73.0% vs 68.8%), B wins pooled (82.6% vs 78.0%); standardized to the same stone mix, A wins (83.3% vs 77.9%).
The data cannot tell which table to trust. If the grouping variable is a cause of the treatment (confounder), use the groups; if it is caused by the treatment (mediator), use the pooled data.
Pearl's ladder: association $P(y \mid x)$ , intervention $P(y \mid \mathrm{do}(x))$ , counterfactual $P(y_x \mid x', y')$ . Higher rungs need more than data from lower rungs.

02 Potential outcomes and experiments

Each unit has potential outcomes $Y_i(1)$ and $Y_i(0)$ ; the individual effect is $\tau_i = Y_i(1) - Y_i(0)$ . Estimands: ATE $\mathbb{E}[Y(1) - Y(0)]$ , ATT (given $T = 1$ ), ATC (given $T = 0$ ), CATE $\tau(x)$ .
Fundamental problem of causal inference: we observe only $Y_i = T_i Y_i(1) + (1 - T_i) Y_i(0)$ , so $\tau_i$ is never observed. Causal inference is a missing-data problem.
SUTVA = consistency (a well-defined treatment; $T_i = t \Rightarrow Y_i = Y_i(t)$ ) + no interference between units.
Naive difference $=$ ATT $+$ selection bias $\mathbb{E}[Y(0) \mid T{=}1] - \mathbb{E}[Y(0) \mid T{=}0]$ . Tutoring example: $-21 = 9 + (-30)$ .
Under consistency, exchangeability $(Y(0), Y(1)) \perp\!\!\!\perp T$ and positivity, $\mathbb{E}[Y(t)] = \mathbb{E}[Y \mid T = t]$ . Randomization makes exchangeability true by design.
Under complete randomization the difference in means is unbiased for the ATE, with variance $S_1^2/n_1 + S_0^2/n_0 - S_\tau^2/n$ ; the usual estimate drops the last term and is conservative.
Fisher's randomization test assumes the sharp null $\tau_i = 0$ for all $i$ , fills the table with the observed outcomes, and gives an exact p-value over all possible assignments.
Blocking and regression adjustment on pre-treatment covariates reduce variance; never adjust for post-treatment variables. Check sample-ratio mismatch and interference in A/B tests.

03 Causal graphs and d-separation

A structural causal model assigns each variable from its parents and an independent noise, $V_j := f_j(\mathrm{pa}_j, U_j)$ ; its causal graph is a DAG with an arrow from each parent to its child.
A missing arrow claims no direct effect, and a missing common cause claims no unmeasured confounding; these claims are what make a graph testable.
Markov factorization: $p(v) = \prod_j p(v_j \mid \mathrm{pa}_j)$ ; equivalently, each node is independent of its non-descendants given its parents.
Chain and fork: dependent, but independent given the middle node. Collider: independent, but dependent given the collider or any of its descendants.
A path is blocked by $Z$ if it has a non-collider in $Z$ or a collider with neither itself nor a descendant in $Z$ ; d-separation means every path is blocked.
Global Markov property: d-separation implies conditional independence in every distribution that factorizes over the graph; faithfulness (the converse) is an extra assumption that fails when paths cancel.
Selecting data on a common effect (admission, survival, approval) is conditioning on a collider and creates associations that do not exist in the population (Berkson's paradox, M-bias).

04 The do-operator and adjustment

$\mathrm{do}(T = t)$ replaces the assignment of $T$ by the constant $t$ and removes the arrows into $T$ ; in general $P(y \mid t) \ne P(y \mid \mathrm{do}(t))$ .
Truncated factorization: $p(v \mid \mathrm{do}(t)) = \prod_{j:\, V_j \ne T} p(v_j \mid \mathrm{pa}_j)$ at $T = t$ ; and $P(Y(t) = y) = P(y \mid \mathrm{do}(t))$ .
A causal quantity is identified if every SCM with the graph and the same observational distribution gives the same value; identification needs positivity.
Backdoor criterion: $Z$ has no descendant of $T$ and blocks every path from $T$ to $Y$ that starts with an arrow into $T$ ; then $P(y \mid \mathrm{do}(t)) = \sum_z P(y \mid t, z)\,P(z)$ .
Good controls: confounders (remove bias) and causes of $Y$ only (reduce variance). Bad controls: mediators, colliders, descendants of $Y$ , M-bias nodes; instruments add variance and amplify bias.
Frontdoor: if $M$ carries all of the effect, has no open backdoor path from $T$ , and its backdoor paths to $Y$ are blocked by $T$ , then $P(y \mid \mathrm{do}(t)) = \sum_m P(m \mid t)\sum_{t'} P(y \mid m, t')P(t')$ .
Do-calculus has three rules (insert/delete observations, exchange action and observation, insert/delete actions); it is complete, so an effect the rules cannot identify is not identified by any method without new assumptions.

05 Matching and propensity scores

Under consistency, conditional exchangeability $(Y(0), Y(1)) \perp\!\!\!\perp T \mid X$ and positivity $0 < e(x) < 1$ , the ATE is identified by the g-formula $\tau_{\mathrm{ATE}} = \mathbb{E}[\mu_1(X) - \mu_0(X)]$ ; regression, matching and weighting are different estimators of this one estimand.
Outcome regression predicts both potential outcomes for every unit; with heterogeneous effects, the single OLS coefficient of $T$ is a variance-weighted average of effects, not the ATE.
Nearest-neighbor matching of treated units estimates the ATT; a caliper drops unmatched units and changes the estimand; imperfect matches cause bias that grows with the number of covariates.
The propensity score $e(x) = P(T = 1 \mid X = x)$ balances $X$ : $T \perp\!\!\!\perp X \mid e(X)$ , and exchangeability given $X$ implies exchangeability given $e(X)$ . Judge its model by balance ( $|\mathrm{SMD}| < 0.1$ ), not by AUC.
IPW weights treated units by $1/e$ and controls by $1/(1 - e)$ (ATT: controls by $e/(1 - e)$ ); it is importance sampling, unbiased with the true $e$ , and the Hájek (normalized) form is preferred.
Extreme weights come from poor overlap; check the largest weight and the effective sample size $(\sum w)^2/\sum w^2$ ; trimming lowers the variance but changes the estimand.
AIPW $= \frac1n\sum[\hat\mu_1 - \hat\mu_0 + T(Y - \hat\mu_1)/\hat e - (1 - T)(Y - \hat\mu_0)/(1 - \hat e)]$ is consistent if either the outcome model or the propensity model is correct; its bias is a product of the two errors.
No adjustment removes an unmeasured confounder; report how strong one would need to be: omitted-variable bias $\gamma\delta$ , the robustness value, or the E-value $\mathrm{RR} + \sqrt{\mathrm{RR}(\mathrm{RR} - 1)}$ .

06 Instrumental variables

An instrument $Z$ satisfies relevance ( $Z$ moves $T$ ), exclusion ( $Z$ changes $Y$ only through $T$ ) and independence ( $Z$ is as good as random). Only relevance can be tested.
In a linear model, $\tau = \operatorname{Cov}(Z, Y)/\operatorname{Cov}(Z, T)$ . For binary $Z$ this is the Wald estimator: reduced form divided by first stage.
With noncompliance, the units are compliers, always-takers, never-takers and defiers. The first stage equals the share of compliers when there are no defiers.
Under independence, exclusion, relevance and monotonicity (no defiers), the Wald ratio equals the LATE: the average effect for compliers, not the ATE.
With one-sided noncompliance (no one can get the treatment without $Z$ ), the LATE equals the effect on the treated.
2SLS: $\hat\beta = (\mathbf X^\top P_{\mathbf Z}\mathbf X)^{-1}\mathbf X^\top P_{\mathbf Z}\mathbf y$ . Residuals for the standard error must use the real $T$ , not $\hat T$ .
A weak instrument (small first-stage $F$ ) pulls 2SLS towards OLS, gives heavy tails and intervals that cover the truth too rarely; $F > 10$ is a minimum, not a guarantee.

07 RDD, DiD and synthetic control

A sharp RDD identifies $\tau_c = \lim_{r \downarrow c}\mathbb{E}[Y \mid R = r] - \lim_{r \uparrow c}\mathbb{E}[Y \mid R = r]$ , the effect at the cutoff, if the potential-outcome regressions are continuous at $c$ .
Estimate the RDD jump with local linear regression on each side (triangular kernel, bandwidth $h$ ); a larger $h$ lowers variance and raises bias. Avoid high-order global polynomials.
RDD checks: no bunching of the running variable at the cutoff, no jumps in pre-treatment covariates, no jumps at placebo cutoffs. A fuzzy RDD is IV at the cutoff: jump in $Y$ divided by jump in $P(T = 1)$ .
Two-period DiD: $(\bar Y_{\text{treated, post}} - \bar Y_{\text{treated, pre}}) - (\bar Y_{\text{control, post}} - \bar Y_{\text{control, pre}})$ identifies the ATT under parallel trends in $Y(0)$ , no anticipation and no spillovers.
Card and Krueger (1994): New Jersey $20.44 \to 21.03$ , Pennsylvania $23.33 \to 21.17$ , DiD $\approx 2.75$ FTE per store (2.76 in the paper).
An event study estimates one DiD per period; flat pre-period coefficients support parallel trends but cannot prove it for the post-period.
With staggered adoption and effects that change over time, TWFE uses already-treated units as controls and can put negative weights on effects; use group-time estimators.
Synthetic control: a convex combination of donors fitted on the pre-period; the post-period gap estimates the effect, and placebo-in-space tests give inference.

08 Causal ML

The CATE $\tau(x) = \mathbb{E}[Y(1) - Y(0) \mid X = x]$ equals $\mu_1(x) - \mu_0(x)$ under consistency, conditional exchangeability and positivity; ML helps estimate it but cannot fix a missing confounder.
Meta-learners turn any regression method into a CATE estimator: the S-learner shrinks effects toward 0, the T-learner is noisy when one group is small, the X-learner uses the large group's model, the DR-learner regresses AIPW pseudo-outcomes.
Double ML for $Y = \theta T + g(X) + \varepsilon$ : cross-fit $\hat\ell(X) \approx \mathbb{E}[Y \mid X]$ and $\hat m(X) \approx \mathbb{E}[T \mid X]$ , then $\hat\theta = \sum \tilde T_i \tilde Y_i / \sum \tilde T_i^2$ (residual on residual, the FWL idea).
Neyman orthogonality makes nuisance errors enter only as products; cross-fitting removes the bias from overfitting. Rate condition: $\|\hat m - m\|(\|\hat m - m\| + \|\hat \ell - \ell\|) = o(n^{-1/2})$ .
With heterogeneous effects, the partially linear $\theta$ is an average weighted by $e(1 - e)$ ; AIPW with cross-fitting is the DML estimator of the ATE.
Honest causal trees choose splits on one subsample and estimate leaf effects on another; causal forests average them and give confidence intervals for $\tau(x)$ .
Individual effects are never observed: evaluate CATE models on randomized held-out data with the transformed outcome ( $\mathbb{E}[Y^* \mid X] = \tau(X)$ ), uplift or Qini curves, and group average effects.

09 Counterfactuals and mediation

A counterfactual $Y_x(u)$ is the value of $Y$ in the modified model $\mathcal M_x$ for a unit with background $u$ ; it is about the observed units, while $P(y \mid \mathrm{do}(x))$ is about the population.
Counterfactuals are computed in three steps: abduction ( $P(u) \to P(u \mid e)$ ), action (replace the equation of $X$ by $X := x$ ), prediction (run $\mathcal M_x$ with the posterior noise).
Rung 3 is not identified by rungs 1–2: $Y := X \oplus U$ and $Y := U$ give the same experiments but opposite counterfactuals; extra assumptions or bounds are needed.
Probability of necessity $\mathrm{PN} = P(Y_{x'} = 0 \mid x, y)$ ; under exogeneity $\max(0, (p_1 - p_0)/p_1) \le \mathrm{PN} \le \min(1, (1 - p_0)/p_1)$ , and under monotonicity $\mathrm{PN} = 1 - 1/\mathrm{RR}$ .
Mediation: $\mathrm{NDE} = \mathbb{E}[Y(1, M(0)) - Y(0, M(0))]$ , $\mathrm{NIE} = \mathbb{E}[Y(1, M(1)) - Y(1, M(0))]$ , $\mathrm{TE} = \mathrm{NDE} + \mathrm{NIE}$ ; $\mathrm{CDE}(m)$ fixes the mediator for everybody.
In a linear model with interaction $dTM$ : $\mathrm{NDE} = c + d\,\mathbb{E}[M(0)]$ and $\mathrm{NIE} = a(b + d)$ ; the product of coefficients $ab$ is right only when $d = 0$ .
Natural effects are identified by the mediation formula under no unmeasured $T$ – $Y$ , $M$ – $Y$ , $T$ – $M$ confounding and the untestable cross-world assumption; randomizing $T$ is not enough.
Counterfactual fairness asks whether a decision would change if the protected attribute and its descendants changed; counterfactual explanations of a classifier are not causal, and recourse needs an SCM of the features.

10 Causal discovery

Causal discovery needs causal Markov, faithfulness, causal sufficiency and acyclicity; then the conditional independences of the data are exactly the d-separations of the true graph.
Two DAGs are Markov equivalent (same independences) if and only if they have the same skeleton and the same v-structures (Verma and Pearl 1990); on 3 nodes, 25 DAGs fall into 11 classes.
A CPDAG draws a class: arrows are shared by every DAG in the class, lines can point either way; observational independences cannot orient the lines.
PC removes edges with tests on growing conditioning sets, orients $i \to k \leftarrow j$ when $k$ is not in the separating set of $i, j$ , and applies Meek's rules; with correct tests it returns the CPDAG.
On data, PC treats "not rejected" as independent: $\alpha$ controls sparsity, and one wrong test propagates to other edges.
NOTEARS replaces the search over DAGs by the smooth constraint $h(W) = \operatorname{tr}(e^{W \circ W}) - d = 0$ , which holds exactly for acyclic graphs; on simulated data, check for varsortability by standardizing.
Non-Gaussian noise (LiNGAM) or nonlinear additive noise breaks the symmetry: only in the causal direction is the residual independent of the regressor.
An intervention on a node orients every edge at that node; $d - 1$ single-node experiments always suffice.

11 Causality in ML practice

A predictor trained by minimizing loss on one distribution uses every predictive association there, including spurious ones; more data from the same source does not remove them. Report worst-group accuracy.
A distribution shift is an intervention on one mechanism of the SCM: covariate shift ( $P(x)$ changes, $P(y \mid x)$ fixed) is natural when $X \to Y$ ; label shift ( $P(y)$ changes, $P(x \mid y)$ fixed) is natural when $Y \to X$ .
Under label shift, $P'(y \mid x) \propto P(y \mid x)\,P'(y)/P(y)$ : the old classifier can be corrected without new labels.
The causal parents $S^*$ of $Y$ give the same $P(y \mid x_{S^*})$ in every environment that does not intervene on $Y$ ; ICP returns the intersection of invariant sets and covers $S^*$ with probability $\ge 1 - \alpha$ .
IRM, group DRO, reweighting, last-layer retraining and augmentation fight spurious features; careful ERM is a strong baseline, and IRM needs many environments.
Logged data are selected by the old policy; decisions need effects (uplift), not risk scores.
Off-policy evaluation: $\hat V_{\mathrm{IPS}} = \frac1n \sum_i \frac{\pi(a_i \mid x_i)}{\pi_0(a_i \mid x_i)} r_i$ is unbiased with logged propensities and overlap; SNIPS normalizes the weights; DR adds a reward model and stays unbiased if either part is right.
Large importance weights mean a small effective sample size $(\sum w)^2/\sum w^2$ and noisy estimates; exploration in the logging policy is the real fix.