← The Nature of Knowledge

The catalog, and what it refuses to order

33 evidence kinds, 5 scored axes each (I, S, rho, sigma, T), ordered by componentwise dominance - the one ordering that names no decision and therefore costs nothing to justify. Most pairs below are not comparable under it. That is not a gap in the data; it is the argument this page makes.

The partial order, drawn as a Hasse diagram

One evidence kind dominates another only if it scores at least as high on every one of the five axes and strictly higher on at least one. The diagram below draws the cover relation - the transitive reduction of that order, so an edge means “nothing sits between these two,” not “these two are related somehow.” Levels run bottom to top by longest chain below; two nodes at the same level with no path between them are genuinely incomparable, not merely undrawn.

RCT on the target population (experimental, empirical domain) - I5 S5 rho5 sigma4 T5Adversarial collaboration, joint prereg (experimental, empirical domain) - I5 S5 rho4 sigma6 T4A/B test, valid sequential analysis (experimental, empirical domain) - I5 S5 rho5 sigma3 T4Regression discontinuity (quasi-experimental, empirical domain) - I4 S3 rho4 sigma3 T2Natural experiment / exogenous shock (quasi-experimental, empirical domain) - I4 S3 rho4 sigma3 T3Instrumental variables / 2SLS (quasi-experimental, empirical domain) - I4 S3 rho3 sigma3 T2Diff-in-diff / synthetic control (quasi-experimental, empirical domain) - I3 S2 rho3 sigma3 T3Matching / propensity / TMLE + sensitivity (observational, empirical domain) - I2 S2 rho3 sigma3 T3Cross-sectional regression (observational, empirical domain) - I1 S1 rho2 sigma1 T2Production telemetry (observational, empirical domain) - I0 S1 rho4 sigma1 T5Third-party panel / scraped data (observational, empirical domain) - I0 S1 rho1 sigma1 T2Backtest, unpurged (computational, empirical domain) - I0 S1 rho2 sigma0 T1Purged / walk-forward CV (computational, empirical domain) - I1 S2 rho4 sigma2 T3Held-out test set, first look (computational, empirical domain) - I1 S4 rho5 sigma3 T3Shadow deployment / canary (computational, empirical domain) - I1 S4 rho5 sigma3 T5Property / metamorphic testing, fuzzing (formal, formal domain) - I5 S3 rho6 sigma4 T4Machine-checked proof (formal, formal domain) - I6 S2 rho6 sigma4 T5Example-based test suite (formal, formal domain) - I3 S1 rho6 sigma2 T3Simulation / Monte Carlo (computational, synthetic domain) - I6 S1 rho6 sigma1 T0Audited financials / regulatory filing (testimonial, empirical domain) - I0 S3 rho5 sigma5 T4Calibrated forecaster track record (testimonial, empirical domain) - I0 S4 rho4 sigma4 T3Structured elicitation (SHELF / Cooke) (testimonial, empirical domain) - I0 S2 rho3 sigma3 T3Unstructured expert judgment (testimonial, empirical domain) - I0 S1 rho1 sigma1 T3Cross-provider LLM adversary, measured ρ (machine, empirical domain) - I0 S3 rho4 sigma4 T2Retrieval-grounded entailment check (machine, empirical domain) - I0 S2 rho4 sigma3 T2Single LLM judge, unmeasured ρ (machine, empirical domain) - I0 S1 rho0 sigma1 T1LLM-generated citation or specific (machine, empirical domain) - I0 S0 rho0 sigma0 T0Single anecdote / demo (testimonial, empirical domain) - I0 S1 rho3 sigma1 T2Reproduced counterexample (formal, any domain) - I0 S6 rho6 sigma2 T3Negative control / regime-change probe (experimental, empirical domain) - I2 S5 rho5 sigma3 T4Cryptographic hash comparison (formal, formal domain) - I0 S6 rho6 sigma0 T0Hostile short-seller report, no stated method (testimonial, empirical domain) - I0 S1 rho1 sigma5 T2Preregistered, blinded, powered observational slope study (observational, empirical domain) - I0 S4 rho5 sigma4 T3RCTAdv. collabA/B (seq)RDDNat. experimentIV / 2SLSDiD / synthMatching / TMLECross-sectionTelemetry3P panelBacktestPurged CVHoldoutShadow / canaryProperty testsProofUnit testsSimulationAudited filingTrack recordSHELF elicitExpert opinionX-provider LLMEntailment checkSingle LLM judgeLLM citationAnecdoteCounterexampleNegative controlHashShort-sellerPrereg slope262 COMPARABLE + 266 INCOMPARABLE = 528 PAIRS TOTAL - 49.6% OF THE ORDER DECIDABLE WITHOUT NAMING A DECISION

Hover a node for its full record. The catalog’s comparability census - printed inside the figure and re-derived from the data on every render - is 262 comparable pairs and 266 incomparable pairs out of 528 total, which is 49.6% of the order decidable without naming a decision.

Pricing figures throughout are synthetic and chosen to make the type of the claim legible; none of them are measurements.

What four records can do to that number

Over every one of the C(33,4) = 40,920 possible four-record removals from this catalog, the comparable fraction ranges from 36.9% to 60.8%. Which four records you drop decides where in a 23.9-point band this statistic lands. Whatever 49.6% describes, it is not a property of the five axes - it describes this exemplar set at least as much, and treating it as a property of the frame is an over-read.

Blackwell has no completion for credal evidence. Once information is a set of measures rather than a measure, "is E more informative than F" has no settled definition. Two adjacent honesty notes belong with it: componentwise dominance on hand-assigned rungs is a PROXY for Blackwell's order rather than that order, since no garbling kernel is constructed; and the intersection-of-decision-orders claim holds over the full weight simplex, not over the four decisions shipped here.

None known. Stated on the page rather than papered over.

Dominance is also not the only decision-free ordering available - it is simply the coarsest one. Mutual information, any single f-divergence, and Le Cam deficiency each order experiments without naming a decision too, and each disagrees with dominance about some pair. Blackwell’s order is the reference point here because it refuses to decide the most pairs, not because it is the unique way to avoid naming a loss function.

The construct-domain gate

Every kind above carries a construct domain - empirical, formal, synthetic, or any - and every decision below declares which domains it admits. The gate applies before any weighting, because a domain mismatch is not a discount to be priced in: a machine-checked proof about your code carries exactly zero bits about a demand curve, however high its reliability. synthetic is admitted nowhere, for the reason Simulation / Monte Carlo’s own record already states - A deduction with extra steps. Upgrades your understanding of your model, never your model. Monte Carlo error is driven to zero by buying compute, which is why reporting it as uncertainty is a category error.

The corrected renderer partitions on this gate rather than discounting through it. The first draft instead multiplied a gated kind’s score by 0.16 and left it in the ranked column - unsound on its own terms, and a divergence from the estate severity mechanism it was modelled on, which rejects on a domain mismatch rather than discounting through one. That draft’s pricing projection ranked a machine-checked proof third and a Monte Carlo simulation eighth: a domain mismatch, weighted down but not removed, still outscoring evidence that actually reaches the decision. Live, not asserted: under all 4 scored decisions below, Simulation is refused with the identical reason - synthetic evidence, not admitted here

What each decision admits, and refuses, and why

Pick a decision. The left-hand column re-ranks by that decision’s own weights; the right-hand column is not a lower tier of the same ranking, it is a different mechanism entirely - a domain mismatch, a failed axis gate, or a meet below the floor, and the reason named is exactly which one. A refused kind has no score to show, because scoring it would imply a ranking this decision never joined.

The only decision-free ordering is componentwise dominance. Most pairs are genuinely incomparable, and that is a theorem, not a gap in the data.

0 of 33 admitted - no decision named, so nothing is scorable

    Refused - no score, because there is no ranking to join

    1. 3P panelno decision named, so there is no ranking to join
    2. A/B (seq)no decision named, so there is no ranking to join
    3. Adv. collabno decision named, so there is no ranking to join
    4. Anecdoteno decision named, so there is no ranking to join
    5. Audited filingno decision named, so there is no ranking to join
    6. Backtestno decision named, so there is no ranking to join
    7. Counterexampleno decision named, so there is no ranking to join
    8. Cross-sectionno decision named, so there is no ranking to join
    9. DiD / synthno decision named, so there is no ranking to join
    10. Entailment checkno decision named, so there is no ranking to join
    11. Expert opinionno decision named, so there is no ranking to join
    12. Hashno decision named, so there is no ranking to join
    13. Holdoutno decision named, so there is no ranking to join
    14. IV / 2SLSno decision named, so there is no ranking to join
    15. LLM citationno decision named, so there is no ranking to join
    16. Matching / TMLEno decision named, so there is no ranking to join
    17. Nat. experimentno decision named, so there is no ranking to join
    18. Negative controlno decision named, so there is no ranking to join
    19. Prereg slopeno decision named, so there is no ranking to join
    20. Proofno decision named, so there is no ranking to join
    21. Property testsno decision named, so there is no ranking to join
    22. Purged CVno decision named, so there is no ranking to join
    23. RCTno decision named, so there is no ranking to join
    24. RDDno decision named, so there is no ranking to join
    25. Shadow / canaryno decision named, so there is no ranking to join
    26. SHELF elicitno decision named, so there is no ranking to join
    27. Short-sellerno decision named, so there is no ranking to join
    28. Simulationno decision named, so there is no ranking to join
    29. Single LLM judgeno decision named, so there is no ranking to join
    30. Telemetryno decision named, so there is no ranking to join
    31. Track recordno decision named, so there is no ranking to join
    32. Unit testsno decision named, so there is no ranking to join
    33. X-provider LLMno decision named, so there is no ranking to join

    Pricing figures throughout are synthetic and chosen to make the type of the claim legible; none of them are measurements.