← The Nature of Knowledge

I: is the quantity recoverable at all

Whether the quantity a claim is about can be recovered from the evidence at all, independent of how carefully the claim itself is stated. This axis has seven rungs of its own, running from bare association up to a validated mechanism, and no rung of U buys a single step up it.

The seven rungs

  1. I0, Association

    P(Y | X) as observed. Prediction under the status quo policy only.

  2. I1, Adjusted association

    P(Y | X, Z). Still zero interventional content; adjustment is not identification.

  3. I2, Ignorability asserted

    A causal reading conditional on an untestable assumption you named. Gate on a sensitivity witness.

  4. I3, Partially identified causal

    A SET containing the causal effect, under assumptions weak enough to defend.

  5. I4, Point-identified, local

    E[Y|do(X)] on a subpopulation you may not be able to name (LATE compliers, RDD at the cutoff, DiD treated units).

  6. I5, Point-identified, target population

    E[Y|do(X)] on the population the decision is about.

  7. I6, Mechanism / counterfactual

    Rung-3 queries: attribution, mediation, "would this unit have converted anyway". Needs a validated SCM, not a DAG you drew.

Why refinement moves up, never right

U and I are non-substitutable, and the reason is definitional, not a theorem. U types the shape of a report’s assertion; I types whether the quantity it is about can be recovered at all. Sharpening a report’s shape cannot change which question it answers, so refining a claim along U moves it up the grid below and never right.

Pearl’s Causal Hierarchy Theorem supplies the empirical bite: a rung-k quantity is not determined by rung-(k−1) data alone, absent causal assumptions, except on a measure-zero set of structural models. Drop that qualifier and the theorem forbids this ladder’s own I2 through I5 - Ignorability asserted, Partially identified causal, Point-identified, local, and Point-identified, target population - each of which recovers a causal quantity from observational data by importing an assumption the data cannot check. The theorem prices the assumption; it does not forbid the move, and the assumption is itself a claim with its own coordinates on this grid.

C1 - U0 Point, I0 assoc: "Elasticity is -1.8". A point estimate of an observational slope. Two axes at zero.C2 - U3 Calibrated, I0 assoc: "Calibrated elasticity band". THE TRAP. Beautifully calibrated posterior on a meaningless quantity. Every internal diagnostic passes - calibration is estimand-relative by construction. And more data makes it WORSE: variance goes to zero, bias stays, so any trust metric that decreases in posterior variance is anti-correlated with truth under confounding.C3 - U6 Residual-bounded, I1 adj: "Residual-bounded observational forecast". Top of the uncertainty ladder, still rung 1. Refinement along U never moves you right. This is the Causal Hierarchy Theorem, not a modelling shortfall.C4 - U1 Interval, I5 target: "RCT point estimate + CI". Licenses the decision. Under-committed on U, which is a cheap fix: report the posterior.C5 - U3 Calibrated, I5 target: "Randomized price ladder, calibrated". The target state for an in-support decision.C6 - U5 Partial ID, I3 bounds: "Manski bounds off-support". Off support no point is identified, however sharp the design is at its own cutoff, so the honest column is the partially-identified one even for a study that point-identifies locally on support. Set-valued and non-shrinking is the correct type here, not a failure to converge.C7 - U6 Residual-bounded, I5 target: "Randomized + named unmodeled terms". The only cell where an autonomous agent should be allowed to recommend a price move whose expected gain is smaller than the named residual bound - and even here it should refuse.C8 - U2 Distribution, I2 ignor: "Matched cohort posterior". Causal reading conditional on an untestable assumption. Admissible only with a sensitivity witness attached.I0 assocI1 adjI2 ignorI3 boundsI4 localI5 targetI6 ctfU0 PointU1 IntervalU2 DistributionU3 CalibratedU4 Credal setU5 Partial IDU6 Residual-boundedNOT IDENTIFIEDSOLID: REFINING U, ALWAYS AVAILABLE · DASHED ROSE + X: CLIMBING U NEVER MOVES A CLAIM RIGHT

Eight worked claims on the U x I grid (rows run U0 at the bottom to U6 at the top; columns run I0 through I6 left to right). The two leftmost columns are the dead zone, shaded and hatched in rose: I0 (Association) and I1 (Adjusted association) license no causal reading no matter how far up U a claim climbs. The two diamonds are the traps, both inside that dead zone. Hover a dot for the claim and why it sits where it does.

Pricing figures throughout are synthetic and chosen to make the type of the claim legible; none of them are measurements.

Climbing U cannot rescue a confounded estimand, and the failure is sharper than merely no improvement. Trap "Calibrated elasticity band" above is the worked instance, in the claim record’s own words:

THE TRAP. Beautifully calibrated posterior on a meaningless quantity. Every internal diagnostic passes - calibration is estimand-relative by construction. And more data makes it WORSE: variance goes to zero, bias stays, so any trust metric that decreases in posterior variance is anti-correlated with truth under confounding.