Causal Inference

Why you can’t get causation “for free” from correlation

In everyday data analysis, you often estimate something like P(Y | X=x): the distribution of outcomes among units where X happens to equal x. That is an observational quantity. It answers: “Among the units I observed with X=x, what does Y look like?”

A causal question is different. It asks: “If I were to force X to be x (possibly contrary to what it would naturally be), what would Y look like?” That is P(Y | do(X=x)). The do-operator is not a stylistic choice; it marks a different data-generating regime.

A classic trap:

• •Observational: People who take a certain drug recover more often.
• •Causal claim: Taking the drug causes recovery.

These are not equivalent because of confounding: some variable Z (e.g., disease severity) might influence both drug-taking and recovery. Then P(Y | X) mixes multiple mechanisms.

The core objects: interventions, DAGs, and identification

Causal inference (in the Pearl/DAG framework) revolves around three atomic ideas:

1) Intervention (do-operator)

• •P(Y | do(X=x)) is defined as the distribution of Y when an external action sets X to x.
• •This “cuts” the normal causes of X; X is no longer generated by its usual parents.

2) Causal directed acyclic graph (DAG)

• •Nodes are variables; directed edges encode direct causal influence.
• •The graph implies conditional independences and supports reasoning about which paths create spurious associations.

3) Identification (often via do-calculus)

• •A causal effect is identified if it can be expressed purely in terms of observational distributions (things you can estimate from passive data) given the DAG.
• •Do-calculus provides formal rules to transform expressions with do(·) into expressions without do(·) when warranted.

Observing vs doing: same symbols, different worlds

It’s tempting to read P(Y | X=x) and P(Y | do(X=x)) as “conditional probabilities with different notation.” But their meanings differ:

• •P(Y | X=x): you condition on an event in the observed world.
• •P(Y | do(X=x)): you modify the data-generating process and then ask about Y.

A helpful mental model:

• •Conditioning is like filtering your dataset to rows where X=x.
• •Intervention is like editing reality so that everyone’s X is set to x, then sampling outcomes.

Structural causal model intuition (without getting lost)

A common semantics for DAGs is a set of structural assignments:

• •X := fₓ(Parents(X), Uₓ)
• •Y := fᵧ(Parents(Y), Uᵧ)

where Uₓ, Uᵧ are exogenous “noise” terms.

Intervening do(X=x) replaces the equation for X with:

• •X := x

Everything else stays the same. This formalizes “cutting incoming arrows into X.”

Why topological order matters (a prerequisite connection)

Because a causal graph is acyclic, it admits a topological order. That’s not just an algorithmic convenience: it reflects a generative ordering—causes upstream, effects downstream. When we factor a joint distribution according to a DAG,

P(V₁, …, Vₙ) = ∏ᵢ P(Vᵢ | Pa(Vᵢ))

we are implicitly using that ordering. Under intervention, we alter exactly one (or more) of those factors.

What causal inference promises—and what it requires

Causal inference can answer “what if we change X?” questions from observational data, but only if you are willing to:

• •encode assumptions (via a DAG or equivalent),
• •check testable implications where possible,
• •accept that some causal questions are not identifiable without additional data or assumptions.

This is a feature, not a bug: it forces you to separate what the data say from what your causal assumptions add.

Why DAGs are the right abstraction

If you only use probability tables, every dependence can be “explained” by many stories. DAGs provide a language of mechanisms: they constrain which variables can directly influence which others. That constraint lets you reason about which associations are causal and which are spurious.

A DAG is a directed acyclic graph where:

• •An arrow X → Y means X is a direct cause of Y (in your model).
• •Absence of an arrow means “no direct causal influence” (again, in your model).

The DAG does not automatically tell you the effect size; it tells you what adjustments are needed to estimate causal effects from observational data.

Three fundamental causal motifs

Most causal reasoning problems are combinations of three motifs:

1) Chain (mediation)

X → M → Y

• •X affects Y through mediator M.
• •If you condition on M, you may block part (or all) of X’s effect on Y.

2) Fork (confounding)

X ← Z → Y

• •Z is a common cause of X and Y.
• •Without adjusting for Z, X and Y are associated even if X has no causal effect on Y.

3) Collider (selection)

X → C ← Y

• •C is a common effect of X and Y.
• •Unconditioned, X and Y can be independent.
• •Conditioning on C (or a descendant of C) can create a spurious association between X and Y.

This collider rule is one of the most counterintuitive pieces of causal inference and is responsible for many real-world errors (e.g., selection bias, Berkson’s paradox).

d-separation: reading independences from a DAG

A DAG implies conditional independence constraints via d-separation. The idea is to determine whether all paths between two sets of variables are blocked by a conditioning set.

A path is blocked if it contains:

• •a non-collider that is conditioned on, or
• •a collider that is not conditioned on and has no conditioned descendants.

While the full formal definition can be dense, the practical payoff is clear:

• •d-separation tells you which conditional independences must hold if your DAG is correct.
• •those independences can sometimes be tested against data (not always, especially with latent variables).

Backdoor paths: the key to confounding

When estimating the causal effect of X on Y, you worry about paths that create association between X and Y that is not due to the directed causal influence X → … → Y.

A backdoor path from X to Y is any path that begins with an arrow into X.

Example:

• •X ← Z → Y is a backdoor path.
• •X ← Z ← W → Y is also a backdoor path (it still starts with an arrow into X).

Backdoor paths matter because they transmit non-causal association. If you can block all backdoor paths (without introducing new bias), then the remaining association corresponds to the causal effect.

Adjustment sets (backdoor criterion)

A set of variables S satisfies the backdoor criterion relative to (X, Y) if:

1) No node in S is a descendant of X.

2) S blocks every backdoor path from X to Y.

If such an S exists, then the causal effect is identified via adjustment:

P(Y | do(X=x)) = ∑ₛ P(Y | X=x, S=s) P(S=s)

This looks like “standardize over S,” but the why is causal: conditioning on S blocks spurious paths, and averaging over S restores the overall population distribution.

Confounding vs selection: two different dangers

It’s easy to say “we’ll control for variables.” But which variables you control for matters.

• •Confounder control (forks): often necessary.
• •Collider control (selection): often harmful.

A compact comparison:

Latent variables and hidden confounding

Real systems often include unobserved variables U (e.g., genetics, socioeconomic factors). If U causes both X and Y, you have an unblocked backdoor path X ← U → Y, but you can’t condition on U.

Then adjustment may fail even if you “control for everything you measured.” This is where identification becomes subtle: you may still identify effects via alternative structures (e.g., frontdoor), instruments, or additional assumptions.

A note on notation: vectors vs scalar variables

You may see causal effects with covariate vectors Z. The adjustment formula generalizes:

P(Y | do(X=x)) = ∑_z P(Y | X=x, Z=z) P(Z=z)

(For continuous variables, replace ∑ with ∫.)

The important part is conceptual: you’re averaging the conditional outcome model over the marginal distribution of confounders, not the conditioned-on distribution within a particular X group.

Why we need identification rules

A DAG plus observational data gives you P(V) and conditional distributions like P(Y | X, Z). Your causal target, however, is interventional: P(Y | do(X)).

Identification asks:

• •Can we express P(Y | do(X)) using only observational quantities implied by P(V) (and the DAG)?
• •If yes, what formula computes it?
• •If no, what extra data/assumptions would be required?

This matters because randomized experiments directly approximate do(X) by design, but observational studies do not. Identification is the bridge.

The truncated factorization (intervention) formula

If the joint distribution factorizes along the DAG as:

P(V) = ∏ᵢ P(Vᵢ | Pa(Vᵢ))

then intervening do(X=x) yields:

P(V | do(X=x)) = 𝟙[X=x] ∏_{Vᵢ ≠ X} P(Vᵢ | Pa(Vᵢ))

Equivalently, for the post-intervention distribution over non-intervened variables:

P(V\{X} | do(X=x)) = ∏_{Vᵢ ≠ X} P(Vᵢ | Pa(Vᵢ))\big|_{X=x}

This is the formal “cut incoming arrows into X” idea.

Backdoor and frontdoor as special identification results

Two major identification patterns appear so often they deserve names.

Backdoor adjustment (confounding control)

If S satisfies the backdoor criterion, then:

P(Y | do(X=x)) = ∑ₛ P(Y | X=x, S=s) P(S=s)

Frontdoor adjustment (mediated identification under hidden confounding)

Frontdoor applies when:

• •There is unobserved confounding between X and Y,
• •but X affects Y only through a measured mediator M,
• •and there is no unobserved confounding between X and M, nor between M and Y conditional on X.

Then:

P(Y | do(X=x)) = ∑ₘ P(M=m | X=x) \; ∑_{x'} P(Y | M=m, X=x') P(X=x')

Intuition:

• •First, learn how X changes M (observable).
• •Then, learn how M relates to Y by averaging over natural variation in X (observable), carefully avoiding the confounding path.
• •Combine them to simulate intervening on X.

Do-calculus: the general tool

Backdoor/frontdoor are consequences of do-calculus. Do-calculus provides rules for transforming expressions like P(Y | do(X), Z) into forms where the do(·) can be removed.

You don’t need to memorize all details to use causal inference effectively, but you do need to understand what the rules do:

• •They allow you to replace interventions with observations when certain graph-separation conditions hold in manipulated graphs.
• •They are sound and (together with standard probability) complete for identification in causal DAG models.

The three rules (high-level)

Let X, Y, Z, W be disjoint sets of nodes. Let G be the causal DAG.

We define modified graphs:

• •G\bar{X}: remove all incoming arrows into X (representing do(X)).
• •G\underline{Z}: remove all outgoing arrows from Z (representing conditioning vs intervention manipulations).

Then the do-calculus rules (informally) are:

1) Insertion/deletion of observations

If Y is d-separated from Z given X and W in G\bar{X}, then:

P(Y | do(X), Z, W) = P(Y | do(X), W)

2) Action/observation exchange

If Y is d-separated from Z given X and W in G\bar{X}\underline{Z}, then:

P(Y | do(X), do(Z), W) = P(Y | do(X), Z, W)

3) Insertion/deletion of actions

If Y is d-separated from Z given X and W in G\bar{X}, where Z has no ancestors in X after modification (more precisely via graph surgery conditions), then:

P(Y | do(X), do(Z), W) = P(Y | do(X), W)

These statements can look intimidating because of the graph modifications, but the core idea is consistent:

• •If intervening/conditioning on something doesn’t open a path to Y (given what you already control), you can add or remove it.

Identification workflow (practical)

When handed a causal query, a DAG, and observational data, a practical workflow is:

1) State the target: e.g., P(Y | do(X=x)).

2) List candidate adjustment variables using backdoor (if possible).

3) If backdoor fails due to unobserved confounding, check frontdoor conditions.

4) If neither applies, use general do-calculus / algorithmic tools (e.g., ID algorithm) to test identification.

5) If not identifiable, redesign: collect more variables, use instruments, exploit experiments, or accept partial identification bounds.

A careful derivation: why adjustment works

Suppose S blocks all backdoor paths from X to Y and contains no descendants of X.

We want to show:

P(Y | do(X=x)) = ∑ₛ P(Y | X=x, S=s) P(S=s)

Sketch with “show your work” steps (conceptual algebra):

1) Start with law of total probability under intervention:

P(Y | do(X=x))

= ∑ₛ P(Y, S=s | do(X=x))

= ∑ₛ P(Y | S=s, do(X=x)) P(S=s | do(X=x))

2) Because S has no descendants of X and we only intervene on X, distribution of S is unchanged:

P(S=s | do(X=x)) = P(S=s)

3) Because S blocks backdoor paths from X to Y, Y is conditionally independent of the intervention once we condition on (X, S):

P(Y | S=s, do(X=x)) = P(Y | X=x, S=s)

4) Substitute back:

P(Y | do(X=x))

= ∑ₛ P(Y | X=x, S=s) P(S=s)

The DAG is doing the heavy lifting in steps (2) and (3). The formula is “statistics,” but the justification is “causality.”

Estimation vs identification (don’t conflate them)

Identification is symbolic: it tells you what expression equals the causal effect in the population.

Estimation is numerical: given finite data, how do you estimate that expression?

Even if a causal effect is identifiable, estimation can still be hard due to:

• •high-dimensional Z (curse of dimensionality),
• •model misspecification,
• •limited overlap/positivity (some X levels rarely occur for some covariates),
• •measurement error.

This lesson focuses on identification logic, but keep the distinction clear: do-calculus answers “can we,” not “how well with this dataset.”

Why causal inference shows up everywhere

Modern ML and data science frequently optimize predictive accuracy: estimate P(Y | X). But decision-making needs causal quantities: what happens to Y if we change X?

Examples:

• •Policy: effect of a training program on income.
• •Medicine: effect of a treatment on survival.
• •Product: effect of a feature change on retention.
• •ML fairness: effect of using a sensitive attribute (or a proxy) on outcomes.

In each case, you are implicitly asking about P(Y | do(X)).

Turning a vague question into a causal estimand

A strong habit: translate English into an estimand.

• •“Does the ad campaign work?”
• •Target: average treatment effect (ATE) in some population: E[Y | do(X=1)] − E[Y | do(X=0)]

• •“What if we set the price to $p?”
• •Target: E[Sales | do(Price=p)]

• •“What would the conversion rate be if everyone saw the new UI?”
• •Target: P(Y=1 | do(X=1))

Once you have the estimand, you can ask: is it identifiable from available data and assumptions?

Choosing an identification strategy: a decision table

Example connection: Bayesian inference as estimation machinery

You already know Bayesian inference. Once identification gives you an estimand like:

P(Y | do(X=x)) = ∑ₛ P(Y | X=x, S=s) P(S=s)

you can estimate the components using Bayesian models:

• •Put priors on parameters of P(Y | X, S) and P(S).
• •Compute posterior predictive quantities.

Causal inference tells you what to estimate; Bayesian inference tells you how to estimate with uncertainty.

Causal inference and ML: where the DAG matters

Some common ML pitfalls become clearer with DAGs:

1) Target leakage

If a feature is a descendant of the label (Y → Feature), the model will “predict” using consequences of Y. That’s not causally meaningful.

2) Controlling for post-treatment variables

If you adjust for a mediator M (X → M → Y) while trying to estimate the total effect of X, you will generally underestimate it.

3) Selection on engagement

Analyzing “active users only” can create collider bias: Feature change (X) affects engagement (C), and user satisfaction (Y) affects engagement; conditioning on active users selects on C.

A realistic end-to-end analysis plan (template)

When you face a causal question in practice:

1) Specify variables: treatment X, outcome Y, potential confounders Z, mediators M, selection variables S.

2) Draw a DAG: even if imperfect, it makes assumptions explicit.

3) Decide estimand: ATE, conditional effect, policy value, etc.

4) Identify using backdoor/frontdoor/do-calculus.

5) Assess assumptions:

• •Are confounders measured?
• •Any colliders being conditioned on by your sampling strategy?
• •Positivity/overlap plausible?

6) Estimate with appropriate methods (regression adjustment, matching, IPW, doubly robust, Bayesian models).

7) Sensitivity analysis for unmeasured confounding.

The conceptual leap is steps (2)–(4): that is what causal inference adds beyond standard statistics.