Counterfactuals and mediation: questions about single cases and causal paths

Before this lesson: potential outcomes (lesson 02); structural causal models and graphs (lesson 03); the do-operator and graph surgery (lesson 04); conditional distributions and Bayes' rule (probability 10 helps for abduction). Time: about 90 minutes, plus the notebook. The folded blocks can be skipped on a first reading.

---

Why this matters #

• Law and responsibility. Courts ask "but for" questions: would the harm have happened without the exposure? The probability of necessity is the formal version, and it explains the rule "a risk ratio above 2 makes causation more probable than not".
• Medicine and public health. Mediation analysis asks how a treatment works: does a drug lower heart attacks through blood pressure or through another path? Does a smoking ban work through fewer smokers or through less second-hand smoke?
• Fair machine learning. Counterfactual fairness (Kusner et al. 2017) asks whether a decision would stay the same if a person's protected attribute were different and everything it causes changed with it. Path-specific effects separate fair and unfair paths.
• Explanations and recourse. "You would have got the loan if your income were 500 higher" is a counterfactual statement about a model. It is useful only if acting on it works in the real world, which needs a causal model of the features (Karimi, Schölkopf and Valera 2021).
• Data augmentation. Some NLP and vision pipelines create counterfactual examples (swap a gender word, change a background) so that a model cannot rely on the changed attribute.

1. Predict first #

2. The picture #

2.1 Three rungs, three kinds of question #

Lesson 01 introduced Pearl's ladder of causation. Rung 1 is association: $P(y \mid x)$, "what do we expect to see?". Rung 2 is intervention: $P(y \mid \mathrm{do}(x))$, "what happens if we act?". Rung 3 is the counterfactual: "given what actually happened, what would have happened if…?".

The difference between rung 2 and rung 3 is the population. An interventional question is about everybody (or a group): "what is the average recovery if all patients get the drug?". A counterfactual question is about the units that we observed, with what we know about them: "would Ann, who took the drug and recovered, have recovered without it?". The evidence about Ann ($X = 1$, $Y = 1$) and the hypothetical action ($X = 0$) contradict each other. They live in two different worlds.

In this lesson, as in Pearl's books, $X$ in sections 2–3.3 is the variable we change (a cause), not a vector of covariates. A structural causal model (SCM, lesson 03) gives every variable an equation $V_j := f_j(\mathrm{pa}_j, U_j)$ with an exogenous noise $U_j$. For one unit, the noises $U = u$ fix everything: they are the unit's background, the things that make it special. A counterfactual keeps the unit's $u$ and changes the equations.

Take the model $X := U_X$, $Y := 2X + U_Y$ with $\mathbb{E}[U_Y] = 0$, and one unit with $X = 1$, $Y = 3$:

1. Abduction. Solve the equation of $Y$ for the noise: $U_Y = Y - 2X = 3 - 2 = 1$. This unit has a large positive noise: its $Y$ is 1 above the average of units with the same $X$.
2. Action. Replace the equation of $X$ by $X := 0$ (graph surgery, as for $\mathrm{do}$).
3. Prediction. $Y_{X=0} = 2 \cdot 0 + U_Y = 0 + 1 = 1$.

The counterfactual $Y_{X=0}$ ("the value $Y$ would have had if $X$ had been 0") is 1 for this unit. The interventional mean is $\mathbb{E}[Y \mid \mathrm{do}(X = 0)] = 2 \cdot 0 + \mathbb{E}[U_Y] = 0$. The two answers differ because the counterfactual uses what we learned about this unit. That answers question 1. ($Y_{X=0}$ is Pearl's notation; in the notation of lesson 02 it is the potential outcome $Y(0)$ of this unit.)

Interactive figure: loading…

Why can the graph alone not answer rung 3? The graph $X \to Y$ says only that $X$ may affect $Y$. Two models with the same graph and the same distributions can disagree about single units (section 3.2). The counterfactual depends on how the noise enters the equation, for example whether the effect of $X$ is the same for every unit.

2.2 Who was helped, who would have been fine anyway #

For a binary cause $X$ and a binary outcome $Y$, each unit has two potential outcomes $(Y_{X=0}, Y_{X=1})$, so it is one of four response types: never $(0, 0)$, always $(1, 1)$, caused $(0, 1)$: the outcome happens only with the exposure, and prevented $(1, 0)$. Ann was exposed and had the outcome, so she is an "always" or a "caused" unit. The probability of necessity (PN) asks how likely she is "caused": would the outcome have been absent without the exposure?

A randomized trial gives two numbers: $p_1 = P(y \mid \mathrm{do}(x)) = \text{always} + \text{caused}$ and $p_0 = P(y \mid \mathrm{do}(x')) = \text{always} + \text{prevented}$. There are four shares and only two equations (and the shares add up to 1), so one number stays free.

Interactive figure: loading…

The free number is the share of "prevented" units. If we can argue that the exposure never prevents the outcome (monotonicity), it is 0 and PN is fixed. Section 3.3 gives the formulas.

2.3 Direct and indirect paths #

Now take a treatment $T$, a mediator $M$ (a variable on a causal path from $T$ to $Y$) and an outcome $Y$, with $T \to M \to Y$ and $T \to Y$. The total effect of $T$ mixes two paths. Mediation analysis splits it. The natural indirect effect asks: keep the treatment at 1, and move only the mediator from the value it would have without treatment to the value it has with treatment. The natural direct effect asks: change the treatment, but keep the mediator at the value it would have without treatment.

Interactive figure: loading…

3. Definitions and main results #

3.1 Counterfactuals in an SCM #

Definition. The counterfactual "$Y$ had $X$ been $x$" for a unit with noise $u$ is $Y_x(u)$, the solution for $Y$ of the modified model $\mathcal M_x$ with $U = u$. Given evidence $E = e$, the counterfactual distribution is

Theorem (three steps; Pearl 2009, Theorem 7.1.7). $P(Y_x \mid e)$ is computed by

1. Abduction: replace $P(u)$ by the posterior $P(u \mid e)$;
2. Action: replace the equation of $X$ by $X := x$, giving $\mathcal M_x$;
3. Prediction: compute the distribution of $Y$ in $\mathcal M_x$ with the noise distribution $P(u \mid e)$.

This is the definition written as a recipe: the sum over $u$ is step 3, the weights $P(u \mid e)$ are step 1, and $Y_x(u)$ is computed in the model of step 2.

Special cases.

• With no evidence, $P(Y_x = y) = P(y \mid \mathrm{do}(x))$: rung 3 contains rung 2.
• Consistency: if the evidence includes $X = x$ and $Y = y$, then $Y_x = y$. The modified model with the same $u$ reproduces the factual world.
• In an additive-noise model $V_j := f_j(\mathrm{pa}_j) + U_j$, abduction for fully observed variables is exact: $U_j = V_j - f_j(\mathrm{pa}_j)$. Then every counterfactual is a single number.
• Variables that are not descendants of $X$ keep their factual values in $\mathcal M_x$. Descendants change, through their equations, with their own abducted noise.

A binary example. Let $X := U_X$ and $Y := (X \wedge U_1) \vee U_2$ with independent $U_1 \sim \mathrm{Bernoulli}(0.5)$, $U_2 \sim \mathrm{Bernoulli}(0.2)$. In words: the drug ($X = 1$) works for people with $U_1 = 1$, and people with $U_2 = 1$ recover anyway. Ann has $X = 1$, $Y = 1$. Would she have recovered without the drug?

1. Abduction: $Y = 1$ with $X = 1$ means $U_1 \vee U_2 = 1$, which has probability $1 - 0.5 \cdot 0.8 = 0.6$. So $P(U_2 = 1 \mid X = 1, Y = 1) = 0.2/0.6 = 1/3$.
2. Action: $X := 0$, so $Y := U_2$.
3. Prediction: $P(Y_{X=0} = 1 \mid X = 1, Y = 1) = 1/3$.

With probability $2/3$ Ann would not have recovered without the drug: the drug was necessary for her with probability $2/3$. This number is her probability of necessity (section 3.3).

3.2 Rung 3 is not identified by rungs 1 and 2 #

Theorem (two models, one experiment). Two SCMs can give the same observational and interventional distributions but different counterfactuals.

Proof by example. $X := U_X$ with $U_X \sim \mathrm{Bernoulli}(0.5)$, and $U \sim \mathrm{Bernoulli}(0.5)$ independent of $U_X$.

• Model A: $Y := X \oplus U$ (exclusive or). Half the units are "caused" ($U = 0$: $Y = X$), half are "prevented" ($U = 1$: $Y = 1 - X$).
• Model B: $Y := U$. Half the units are "always", half are "never".

In both models $P(Y = 1 \mid \mathrm{do}(X = x)) = 0.5$ for $x = 0, 1$, and $X$ is independent of $Y$ with $P(Y = 1) = 0.5$: every rung-1 and rung-2 quantity is equal. Now take a unit with $X = 1$, $Y = 1$. In model A abduction gives $U = 0$, so $Y_{X=0} = 0$: the exposure was necessary (PN $= 1$). In model B, $Y_{X=0} = U = 1$: it was not (PN $= 0$). $\blacksquare$

So no amount of experimental data alone fixes a counterfactual. We need extra assumptions about the functions (for example monotonicity or additive noise), or we must report bounds.

Twin networks

Balke and Pearl (1994) draw a counterfactual query as one graph. Copy every endogenous node: one copy for the factual world, one for the counterfactual world. The two copies share the exogenous noises $U$. In the counterfactual copy, cut the arrows into $X$ and set $X := x$. Now "$Y_x \mid X = x', Y = y$" is an ordinary conditional probability in this larger network, and d-separation in the twin network tells us which counterfactual independences hold. For example, in $X \to Y$ with independent $U_X, U_Y$, the counterfactual $Y_x$ (in the second copy) is d-separated from the factual $X$ (in the first copy), which gives the potential-outcome condition $Y_x \perp\!\!\!\perp X$ of a randomized experiment. Shpitser and Pearl (2007) extend this to "counterfactual graphs" and give a complete identification algorithm for counterfactual queries.

3.3 Probability of necessity and sufficiency #

Definitions (Pearl 1999; Tian and Pearl 2000), for binary $X$ and $Y$:

• Probability of necessity: $\mathrm{PN} = P(Y_{x'} = 0 \mid X = 1, Y = 1)$. Among the exposed who had the outcome, the share who would not have had it without the exposure. ("But for" causation.)
• Probability of sufficiency: $\mathrm{PS} = P(Y_{x} = 1 \mid X = 0, Y = 0)$. Among the unexposed without the outcome, the share who would have had it with the exposure.
• Probability of necessity and sufficiency: $\mathrm{PNS} = P(Y_x = 1, Y_{x'} = 0)$, the share of "caused" units in the population.

Assumptions. Exogeneity: $X$ is independent of the response type, as in a randomized trial (then $P(y \mid \mathrm{do}(x)) = P(y \mid x)$). Monotonicity: $Y_x \ge Y_{x'}$ for every unit; no unit is "prevented". Exogeneity can be secured by design; monotonicity is a claim about the science and cannot be tested with these data.

Theorem (bounds under exogeneity; Tian and Pearl 2000). With $p_1 = P(y \mid \mathrm{do}(x))$ and $p_0 = P(y \mid \mathrm{do}(x'))$,

Under monotonicity PN equals the lower bound,

the excess risk ratio; also $\mathrm{PNS} = p_1 - p_0$.

Proof. Let the type shares be $\alpha$ (always), $\kappa$ (caused), $\pi$ (prevented), $\nu$ (never).

1. Exogeneity: the types have the same shares among the exposed. Exposed units with $Y = 1$ are always or caused, so $\mathrm{PN} = \kappa/(\alpha + \kappa) = \kappa/p_1$.
2. The data give $p_1 = \alpha + \kappa$ and $p_0 = \alpha + \pi$, so $\kappa = p_1 - p_0 + \pi$.
3. The free share $\pi$ must keep every share in $[0, 1]$: $\kappa \ge 0$ gives $\pi \ge p_0 - p_1$; $\alpha \ge 0$ gives $\pi \le p_0$; $\nu = 1 - p_1 - \pi \ge 0$ gives $\pi \le 1 - p_1$.
4. $\mathrm{PN} = (p_1 - p_0 + \pi)/p_1$ grows with $\pi$. At the smallest $\pi$, $\max(0, p_0 - p_1)$, it is $\max(0, (p_1 - p_0)/p_1)$. At the largest $\pi$, $\min(p_0, 1 - p_1)$, it is $\min(1, (1 - p_0)/p_1)$.
5. Monotonicity means $\pi = 0$, which gives the lower bound and $\mathrm{PNS} = \kappa = p_1 - p_0$. $\blacksquare$

Now question 2 of section 1: $p_1 = 0.4$, $p_0 = 0.2$, $\mathrm{RR} = 2$. The data alone give $0.5 \le \mathrm{PN} \le \min(1, 0.8/0.4) = 1$. Under monotonicity, $\mathrm{PN} = 1 - 1/2 = 0.5$. So the honest answer is "at least 50%"; it is exactly 50% only if the drug never prevents recovery. This is the reason behind the legal rule of thumb: when $\mathrm{RR} > 2$, $\mathrm{PN} > 0.5$ under monotonicity, so causation is "more probable than not". Check the binary example of 3.1: $p_1 = 0.6$, $p_0 = 0.2$, and that model is monotone, so $\mathrm{PN} = 0.4/0.6 = 2/3$, as computed by the three steps.

Combining experimental and observational data

With confounding, $P(y \mid x) \ne P(y \mid \mathrm{do}(x))$. Tian and Pearl (2000) show that the two kinds of data together can narrow PN:

$$\max\Big(0, \frac{P(y) - P(y \mid \mathrm{do}(x'))}{P(x, y)}\Big) \le \mathrm{PN} \le \min\Big(1, \frac{P(y' \mid \mathrm{do}(x')) - P(x', y')}{P(x, y)}\Big).$$

Here $P(x, y) = P(X = 1, Y = 1)$ and $P(x', y') = P(X = 0, Y = 0)$ come from observational data. Under monotonicity PN equals the lower bound. Sometimes the bounds even collapse to one value. (We checked these bounds against a linear program over the eight cells "exposure × response type" for 2000 random models; the notebook shows the method.)

3.4 Mediation: direct and indirect effects #

Let $T$ be binary, $M$ a mediator, $Y$ the outcome. Write $M(t)$ for the mediator under treatment $t$ and $Y(t, m)$ for the outcome when $T$ is set to $t$ and $M$ to $m$.

Definitions.

• Total effect: $\mathrm{TE} = \mathbb{E}[Y(1, M(1)) - Y(0, M(0))]$, the usual ATE.
• Controlled direct effect: $\mathrm{CDE}(m) = \mathbb{E}[Y(1, m) - Y(0, m)]$. The mediator is fixed at $m$ for everybody. It is a rung-2 quantity: $\mathrm{do}(T = t, M = m)$.
• Natural direct effect: $\mathrm{NDE} = \mathbb{E}[Y(1, M(0)) - Y(0, M(0))]$. Change $T$, but keep each unit's mediator at its natural value without treatment.
• Natural indirect effect: $\mathrm{NIE} = \mathbb{E}[Y(1, M(1)) - Y(1, M(0))]$. Keep $T = 1$, and move each unit's mediator from $M(0)$ to $M(1)$.

Theorem (decomposition). $\mathrm{TE} = \mathrm{NDE} + \mathrm{NIE}$.

Proof. Add and subtract $Y(1, M(0))$ inside the expectation: $Y(1, M(1)) - Y(0, M(0)) = [Y(1, M(1)) - Y(1, M(0))] + [Y(1, M(0)) - Y(0, M(0))]$. Take expectations. $\blacksquare$

The term $Y(1, M(0))$ is a cross-world quantity: the treatment is 1, but the mediator takes the value it would have under treatment 0. No experiment can ever observe it, not even a randomized one, because a unit cannot be treated and untreated at the same time.

Theorem (linear models). Let $M := \alpha_M + aT + \varepsilon_M$ and $Y := \alpha_Y + cT + bM + dTM + \varepsilon_Y$, with noises of mean 0. Then

Without the interaction ($d = 0$): $\mathrm{NDE} = \mathrm{CDE}(m) = c$ for every $m$, and $\mathrm{NIE} = ab$, the product of coefficients (Baron and Kenny 1986), with $\mathrm{TE} = c + ab$.

Proof.

1. $Y(t, m) = \alpha_Y + ct + bm + dtm + \varepsilon_Y$, so $Y(1, m) - Y(0, m) = c + dm$. Take expectations: $\mathrm{CDE}(m) = c + dm$.
2. Put $m = M(0)$: $Y(1, M(0)) - Y(0, M(0)) = c + d\,M(0)$, so $\mathrm{NDE} = c + d\,\mathbb{E}[M(0)]$.
3. $Y(1, M(1)) - Y(1, M(0)) = (b + d)(M(1) - M(0)) = (b + d)\,a$, because $M(1) - M(0) = a$ for every unit. So $\mathrm{NIE} = a(b + d)$.
4. With $d = 0$ steps 1–3 give $c$, $c$ and $ab$. $\blacksquare$

Now question 3 of section 1: with $a = 2$, $b = 1$, $d = 0.5$, the indirect effect is $a(b + d) = 3$, not $ab = 2$. The product rule fails as soon as the effect of the mediator depends on the treatment. (With $\mathbb{E}[M(0)] = 1$ and $c = 0.5$, $\mathrm{NDE} = 1$ and $\mathrm{TE} = 4$: the first figure's default.)

3.5 Identification: the mediation formula #

Assumptions (Pearl 2001; Imai, Keele and Tingley 2010), possibly given pre-treatment covariates $X$:

1. No unmeasured confounding of $T \to Y$.
2. No unmeasured confounding of $M \to Y$, given $T$.
3. No unmeasured confounding of $T \to M$.
4. Cross-world assumption: no confounder of $M \to Y$ is itself affected by $T$. Formally $Y(t, m) \perp\!\!\!\perp M(t') \mid X$ for all $t, t', m$.

If $T$ is randomized, 1 and 3 hold by design. Assumptions 2 and 4 do not: the mediator is not randomized. Assumption 4 involves two worlds ($t$ and $t'$) at once, so no experiment can test it.

Theorem (mediation formula; Pearl 2001). Under 1–4 (shown without $X$; with $X$, apply it within each stratum and average over $P(x)$), for a discrete $M$:

Proof sketch for the NIE. $\mathbb{E}[Y(1, M(t'))] = \sum_m \mathbb{E}[Y(1, m) \mid M(t') = m]\,P(M(t') = m)$. By assumption 4, $\mathbb{E}[Y(1, m) \mid M(t') = m] = \mathbb{E}[Y(1, m)]$, and by 1–2 this equals $\mathbb{E}[Y \mid T = 1, M = m]$. By 3, $P(M(t') = m) = P(m \mid T = t')$. Subtract the cases $t' = 1$ and $t' = 0$. The NDE is the same computation. $\blacksquare$

3.6 Counterfactuals in machine learning #

Counterfactual fairness (Kusner, Loftus, Russell and Silva 2017). Let $A$ be a protected attribute in an SCM. A predictor $\hat Y$ is counterfactually fair if for every individual (every $u$ consistent with their data) and every value $a'$,

"Would the decision be the same if this person's attribute were different, and everything it causes changed with it?" Kusner et al. show a simple sufficient condition: a predictor that uses only non-descendants of $A$ (and abducted noises) is counterfactually fair. A model that uses a descendant of $A$, such as income in a world where $A$ affects income, is not, even if it never sees $A$. This is different from "fairness through unawareness" (dropping $A$), which ignores the paths through other features.

Counterfactual explanations (Wachter, Mittelstadt and Russell 2017). For a classifier $f$ and an input $x$ with an unwanted decision, a counterfactual explanation is the closest point $x^*$ with the wanted decision: $x^* = \arg\min_{x'} d(x, x')$ subject to $f(x') = \text{wanted}$. This is a statement about the model, and it treats every feature as an independent knob. It is not a causal counterfactual: it uses no SCM of the features. If a person follows it, the features they change move other features through the real causal links.

Algorithmic recourse (Karimi, Schölkopf and Valera 2021) asks the causal question: which action $\mathrm{do}(\cdot)$ on the features, at the smallest cost, leads to the wanted decision, when the other features respond through the SCM? The person's own noise is kept (abduction), so recourse is a counterfactual computation.

Interactive figure: loading…

Counterfactual data augmentation. In NLP, Kaushik, Hovy and Lipton (2020) asked people to edit texts minimally so that the label flips; models trained with these pairs rely less on spurious words. Automatic versions swap attribute words ("he" ↔ "she") to reduce gender bias. These are counterfactuals only in a loose sense: the edit changes the attribute but usually does not model what the attribute causes.

4. Worked example #

We do one counterfactual in a three-variable linear model by hand, and then one mediation analysis from a table.

(a) Joe's homework (after Pearl, Glymour and Jewell, Primer, section 4.2). $X$ is time in an after-school program, $H$ is homework time, $Y$ is the exam score (standardized units):

We observe Joe: $X = 0.5$, $H = 1$, $Y = 1.5$. What would his score have been had he done $H = 2$?

1. Abduction. $U_X = X = 0.5$. $U_H = H - 0.5X = 1 - 0.25 = 0.75$. $U_Y = Y - 0.7X - 0.4H = 1.5 - 0.35 - 0.4 = 0.75$.
2. Action. Replace the equation of $H$ by $H := 2$. $X$ is not a descendant of $H$, so $X = 0.5$ stays.
3. Prediction. $Y_{H=2} = 0.7 \cdot 0.5 + 0.4 \cdot 2 + 0.75 = 0.35 + 0.8 + 0.75 = 1.9$.

The counterfactual rises by $0.4 \cdot (2 - 1) = 0.4$, the coefficient times the change. Now a different question: what would his score have been had he not been in the program ($X = 0$)?

1. Abduction: the same noises $(0.5, 0.75, 0.75)$.
2. Action: $X := 0$.
3. Prediction: $H$ is a descendant of $X$, so it changes: $H_{X=0} = 0.5 \cdot 0 + 0.75 = 0.75$. Then $Y_{X=0} = 0.7 \cdot 0 + 0.4 \cdot 0.75 + 0.75 = 0.3 + 0.75 = 1.05$.

The score falls by $1.5 - 1.05 = 0.45$: the direct part $0.7 \cdot 0.5 = 0.35$ plus the indirect part $0.4 \cdot 0.5 \cdot 0.5 = 0.1$ through homework.

(b) A mediation table. A randomized program $T$, a binary mediator $M$ (a certificate) and a binary outcome $Y$ (employed after a year). Assume 1–4 of section 3.5 hold. The data give

	$P(M = 1 \mid T)$	$\mathbb{E}[Y \mid T, M = 0]$	$\mathbb{E}[Y \mid T, M = 1]$
$T = 0$	0.4	0.2	0.4
$T = 1$	0.8	0.3	0.7

1. Total effect. $\mathbb{E}[Y \mid T = 1] = 0.2 \cdot 0.3 + 0.8 \cdot 0.7 = 0.06 + 0.56 = 0.62$. $\mathbb{E}[Y \mid T = 0] = 0.6 \cdot 0.2 + 0.4 \cdot 0.4 = 0.12 + 0.16 = 0.28$. $\mathrm{TE} = 0.34$.
2. Controlled direct effects. $\mathrm{CDE}(0) = 0.3 - 0.2 = 0.1$ and $\mathrm{CDE}(1) = 0.7 - 0.4 = 0.3$. They differ, so there is an interaction.
3. NDE: weight the CDEs by the mediator distribution without treatment: $0.1 \cdot 0.6 + 0.3 \cdot 0.4 = 0.06 + 0.12 = 0.18$.
4. NIE: with $T = 1$, the mediator moves from $P(M{=}1) = 0.4$ to $0.8$: $0.3 \cdot (0.2 - 0.6) + 0.7 \cdot (0.8 - 0.4) = -0.12 + 0.28 = 0.16$.
5. Check: $\mathrm{NDE} + \mathrm{NIE} = 0.18 + 0.16 = 0.34 = \mathrm{TE}$.

About half of the effect ($0.16/0.34 \approx 47\%$) goes through the certificate. With the other convention (indirect effect at $T = 0$) the indirect part would be $0.2 \cdot (-0.4) + 0.4 \cdot 0.4 = 0.08$: the reference level matters when there is an interaction.

5. In code #

NumPy checks the worked examples, the binary counterfactual and the linear mediation formulas.

import numpy as np

# (a) Joe: X := U_X, H := 0.5 X + U_H, Y := 0.7 X + 0.4 H + U_Y
def f_H(x, uH): return 0.5 * x + uH
def f_Y(x, h, uY): return 0.7 * x + 0.4 * h + uY

X, H, Y = 0.5, 1.0, 1.5
uX, uH, uY = X, H - 0.5 * X, Y - 0.7 * X - 0.4 * H           # abduction
print(uX, uH, round(uY, 3))                                  # 0.5 0.75 0.75
print(round(f_Y(X, 2.0, uY), 3))                             # 1.9   (Y had H been 2)
h0 = f_H(0.0, uH)                                            # H changes: it is a descendant of X
print(h0, round(f_Y(0.0, h0, uY), 3))                        # 0.75 1.05  (Y had X been 0)

# binary counterfactual: Y := (X and U1) or U2, by enumerating the noises
pU1, pU2 = 0.5, 0.2
num = den = 0.0
for u1 in (0, 1):
    for u2 in (0, 1):
        p = (pU1 if u1 else 1 - pU1) * (pU2 if u2 else 1 - pU2)
        if (1 and u1) or u2:                     # evidence X = 1, Y = 1
            den += p
            num += p * (u2 == 0)                 # Y_{X=0} = U2 = 0: the drug was necessary
print(round(num / den, 4))                       # 0.6667 = PN
p1, p0 = 1 - (1 - pU1) * (1 - pU2), pU2
print(round(p1, 3), p0, round((p1 - p0) / p1, 4))  # 0.6 0.2 0.6667 (excess risk ratio)

# (b) mediation formula on the table
PM = {0: 0.4, 1: 0.8}                                        # P(M = 1 | T)
EY = {(0, 0): 0.2, (0, 1): 0.4, (1, 0): 0.3, (1, 1): 0.7}    # E[Y | T, M]
pm = lambda m, t: PM[t] if m else 1 - PM[t]
TE  = sum(EY[1, m] * pm(m, 1) - EY[0, m] * pm(m, 0) for m in (0, 1))
NDE = sum((EY[1, m] - EY[0, m]) * pm(m, 0) for m in (0, 1))
NIE = sum(EY[1, m] * (pm(m, 1) - pm(m, 0)) for m in (0, 1))
print(round(TE, 3), round(NDE, 3), round(NIE, 3))            # 0.34 0.18 0.16

# linear mediation with interaction: simulate the cross-world quantity Y(1, M(0))
rng = np.random.default_rng(1)
n, a, b, c, d = 10**6, 2.0, 1.0, 0.5, 0.5
eM, eY = rng.normal(size=n), rng.normal(size=n)
M = lambda t: 1 + a * t + eM
Yf = lambda t, m: c * t + b * m + d * t * m + eY
print(round(np.mean(Yf(1, M(0)) - Yf(0, M(0))), 3))          # 1.0  = c + d E[M(0)]
print(round(np.mean(Yf(1, M(1)) - Yf(1, M(0))), 3))          # 3.0  = a (b + d), not ab = 2

The simulation can compute $Y(1, M(0))$ only because we wrote the model and kept the noises. In real data this quantity is never observed; the mediation formula replaces it under assumptions 1–4.

6. Where it shows up #

• Epidemiology and medicine. Mediation analysis is standard: how much of the effect of education on heart disease goes through smoking or blood pressure; how much of a vaccine's effect goes through antibodies. VanderWeele's book (2015) is the main reference.
• Law. "But for" causation in toxic-tort cases uses the excess risk ratio $1 - 1/\mathrm{RR}$ and the rule $\mathrm{RR} > 2$; the PN bounds show what the epidemiological data really support.
• Fairness in ML. Counterfactual fairness and path-specific fairness (Chiappa 2019) decide which causal paths from a protected attribute to a decision are allowed; for example, a path through a job-relevant skill may be allowed, a direct path may not.
• Explainability and recourse. Counterfactual explanations are implemented in many explainability libraries; recourse work adds the causal model and feasibility constraints (age cannot go down).
• Root-cause analysis. "Would the outage have happened without this deployment?" is a PN-type question; causal root-cause tools in operations and in manufacturing compute counterfactuals in fitted SCMs.
• ML for science. In climate science, probabilistic event attribution compares the probability of an extreme event with and without human forcing, $p_1$ and $p_0$; the reported "fraction of attributable risk" $1 - p_0/p_1$ is the PN under monotonicity. In genetics, mediation analysis asks how much of a variant's effect on a disease goes through gene expression.

7. Common mistakes #

• Answering a rung-3 question with a rung-2 number. "The drug raised recovery from 20% to 40%, so it saved Ann with probability 20%" is wrong. Compute PN or its bounds; state monotonicity if you use it.
• Treating a counterfactual as identified by the graph. Counterfactuals depend on the functional form and the noise. Say which assumptions (additive noise, monotonicity) fix them, or report bounds.
• Keeping the descendants fixed. In the action step, only the non-descendants of the changed variable keep their factual values. Descendants change through their equations with the abducted noises (Joe's homework in section 4).
• Product of coefficients with an interaction or a nonlinear model. $ab$ is the NIE only in linear models without a $T \times M$ term. Fit the interaction and use the mediation formula; check by simulation.
• Mediation analysis that ignores mediator–outcome confounding. Randomizing $T$ does not randomize $M$. Adjust for measured $M$–$Y$ confounders, never for ones affected by $T$, and run a sensitivity analysis.
• Mixing up CDE and NDE. The CDE fixes the mediator at one value for everybody; the NDE keeps each unit's natural value. They are equal only without interaction.
• Calling a counterfactual explanation causal. It describes the model's decision boundary. Following it can fail when features cause each other; use recourse with a causal model of the features.
• "Fairness through unawareness". Removing the protected attribute does not remove its effect through descendants such as income or zip code.

8. Check yourself #

1. In the model $X := U_X$, $Y := 3X + U_Y$, we observe $X = 2$, $Y = 5$. Compute $Y_{X=1}$ and $\mathbb{E}[Y \mid \mathrm{do}(X = 1)]$ (with $\mathbb{E}[U_Y] = 0$).
   Answer

   Abduction: $U_Y = 5 - 6 = -1$. Action $X := 1$. Prediction: $Y_{X=1} = 3 - 1 = 2$. The interventional mean is $3 \cdot 1 + 0 = 3$. The unit has a negative noise, so its counterfactual is 1 below the population mean.
2. Why is $\mathrm{PN} = 1 - 1/\mathrm{RR}$ only a lower bound without monotonicity?
   Answer

   The trial fixes $p_1 = \alpha + \kappa$ and $p_0 = \alpha + \pi$. If some units are "prevented" ($\pi > 0$), then for the same $p_0$ there are fewer "always" units and more "caused" ones: $\kappa = p_1 - p_0 + \pi$. So $\mathrm{PN} = \kappa/p_1$ grows with $\pi$. Monotonicity sets $\pi = 0$, the smallest value.
3. A linear SCM has $M := 0.5T + \varepsilon_M$, $Y := 1.0T + 2M + \varepsilon_Y$. Give TE, NDE, NIE and $\mathrm{CDE}(3)$.
   Answer

   No interaction: $\mathrm{NDE} = \mathrm{CDE}(m) = 1$ for every $m$, so $\mathrm{CDE}(3) = 1$; $\mathrm{NIE} = ab = 0.5 \cdot 2 = 1$; $\mathrm{TE} = c + ab = 2$.
4. Why can no randomized experiment, even one that randomizes both $T$ and $M$, observe $\mathbb{E}[Y(1, M(0))]$?
   Answer

   It needs, for each unit, the mediator value under $T = 0$ and the outcome under $T = 1$ with that mediator value. A unit is either treated or not; we never see $M(0)$ for a treated unit. Randomizing $M$ gives $\mathbb{E}[Y(1, m)]$ (CDEs), not the unit's own $M(0)$. This is why the cross-world assumption 4 is needed.
5. A lending model does not use race, but uses zip code and income, which race affects in the SCM. Is it counterfactually fair? What predictor would be?
   Answer

   Usually not: changing race in the counterfactual world changes zip code and income (descendants), so the prediction changes. A predictor that uses only non-descendants of race, or the abducted noise terms of the features (the parts not caused by race), is counterfactually fair (Kusner et al. 2017).
6. In the recourse figure with $\beta = 0.5$, the applicant is at $(1.5, 1.5)$. Verify that the explanation costs 2.30 and the recourse 1.64.
   Answer

   Score $1.5 + 0.8 \cdot 1.5 = 2.7$, gap $5 - 2.7 = 2.3$. The explanation changes $x_1$ only: $\delta = 2.3/1 = 2.30$. Acting on $x_1$ also raises $x_2$ by $0.5\delta$, so the score rises by $(1 + 0.8 \cdot 0.5)\delta = 1.4\delta$; $\delta = 2.3/1.4 = 1.64$. Acting on $x_2$ alone gains $0.8$ per unit and would cost $2.875$.

Recall cardsAnswer in your head first, then reveal. Add the cards to your review deck to see them again just before you would forget.

1. Name the three steps for computing a counterfactual in an SCM.

   Show answer

2. Why does a counterfactual need more than the causal graph and $P(y \mid \mathrm{do}(x))$?

   Show answer

3. In $X := U_X$, $Y := 2X + U_Y$, a unit has $(X, Y) = (1, 3)$. Compute $Y_{X=0}$ and $\mathbb{E}[Y \mid \mathrm{do}(X{=}0)]$.

   Show answer

4. What is a twin network?

   Show answer

5. Define the probability of necessity.

   Show answer

6. Give PN under exogeneity and monotonicity.

   Show answer

7. Define the natural direct and natural indirect effects.

   Show answer

8. Why is $\mathrm{TE} = \mathrm{NDE} + \mathrm{NIE}$?

   Show answer

9. When does the product of coefficients $ab$ equal the NIE, and what is the NIE with an interaction $dTM$?

   Show answer

10. Define counterfactual fairness.

    Show answer

Add these cards to my reviewOpen the review →

9. Interview angle #

• "What is the difference between an interventional and a counterfactual question?" — Interventional: the effect of an action on a population, $P(y \mid \mathrm{do}(x))$, identifiable from experiments. Counterfactual: what would have happened to observed units in another world, $P(y_{x'} \mid x, y)$; needs the functions and noise of an SCM or bounds.
• "A trial shows risk ratio 3. What is the probability that the drug caused this patient's side effect?" — Under monotonicity (the drug never prevents the side effect) and no confounding, $\mathrm{PN} = 1 - 1/3 = 2/3$. Without monotonicity, only $\mathrm{PN} \ge 2/3$.
• "How would you measure how much of a feature launch's effect on revenue goes through engagement?" — Randomize the launch (TE), define NDE/NIE, use the mediation formula with engagement as mediator; say that engagement is not randomized, so engagement–revenue confounders must be measured; run a sensitivity analysis.
• "Is the product of coefficients a good estimate of the indirect effect?" — Only in linear models without interaction and without unmeasured mediator–outcome confounding.
• "Is a counterfactual explanation the same as a causal counterfactual?" — No. It is the nearest input with another prediction, ignoring causal links between features. Recourse uses an SCM of the features.

10. Practice #

The notebook for this lesson has four exercises, each with a check cell and a folded solution:

1. Write counterfactual(scm, evidence, intervention) for additive-noise SCMs (abduction by solving for the noise); check the examples of this lesson and Joe's homework.
2. Build two binary SCMs with the same interventional distribution but different counterfactuals, and compute PN in both by enumerating the noise; compare with the Tian–Pearl bounds.
3. Mediation in a linear SCM: estimate NDE and NIE by regression (product of coefficients and mediation formula) and by simulating $Y(1, M(0))$ from the true model; add an interaction and see the product rule fail.
4. Counterfactual fairness on a simulated lending SCM: change the protected attribute, propagate to the descendants with each person's own noise, and compare a model that uses descendants with one that does not.

11. Watch and read #

14 - Counterfactuals and MediationBrady Neal · Introduction to Causal Inference, lecture 14 · 35 min

The whole lecture: the definition of counterfactuals in SCMs, the three steps, and natural direct and indirect effects.

14.3 - MediationBrady Neal · Introduction to Causal Inference, lecture 14 · 16 min

Controlled and natural direct effects, the indirect effect, and why the cross-world assumption is needed.

Counterfactuals: Causal Inference BootcampMod•U · 5 min

A short, plain introduction to counterfactual thinking.

Counterfactual Fairness: Matt Kusner, The Alan Turing InstituteThe Alan Turing Institute · 46 min

One of the authors explains counterfactual fairness and why using descendants of a protected attribute is unfair.

• Pearl, Glymour and Jewell, Causal Inference in Statistics: A Primer (2016), chapter 4 (companion site): counterfactuals, the three steps, PN and mediation.
• Peters, Janzing and Schölkopf, Elements of Causal Inference (2017, open access), chapter 3 and section 6.4: counterfactuals in SCMs.
• Tian and Pearl (2000), Probabilities of causation: bounds and identification.
• Pearl (2001), Direct and indirect effects.
• Imai, Keele and Tingley (2010), "A general approach to causal mediation analysis", Psychological Methods.
• Kusner, Loftus, Russell and Silva (2017), Counterfactual fairness.
• Wachter, Mittelstadt and Russell (2017), Counterfactual explanations without opening the black box.
• Karimi, Schölkopf and Valera (2021), Algorithmic recourse: from counterfactual explanations to interventions.
• Brady Neal, Introduction to Causal Inference (free course notes), chapter 14.