Dollarized Confusion Matrix
A confusion matrix where counts are replaced with costs. The optimal threshold follows: theta* = C_FP / (C_FP + C_FN). Costs drive thresholds, thresholds drive autonomy levels.
Why It Exists
Teams commonly default to 0.5 or an intuitive cutoff. The optimal threshold is a calculation, and the inputs are the actual dollar cost of being wrong in each direction.
Rosetta Stone
Four circles, four readings of the same object. Each role reads the artifact through its own lens.
The cost of being wrong, direction by direction. Thresholds follow from costs, not convention.
The answer to "how confident should the AI be before it acts?" - not a feeling, a calculation. Plug the cost of each error and get the threshold.
How much does each kind of mistake cost us? That question, formalized.
Bayes risk matrix with asymmetric loss. Optimal theta* = C_FP / (C_FP + C_FN). Textbook decision theory, rarely used.
Related Terms
Verification Trap - A task that is easy to generate but hard to verify.
Autonomy State Machine - A graduated trust system for AI deployments with three states: Disabled, HITL (human verifies every output), and Autonomous (spot-check only).