Experiment
When can a smaller model take over?
Agreement with a frontier model measures agreement with a fallible teacher. This experiment examines how the teacher's own errors affect the decision to hand work to a local model.
Read the experiment
Andrew Templeton
I work on the decisions around software: what to build, what a mistake costs, and when a system has earned more independence.
What does it cost
when the model is wrong?
A false alarm and a missed case rarely cost the same. The Dollarized Confusion Matrix turns those costs into a decision threshold.
Watch the decision play outTry a hypothetical decision
Flag above this probability
Assumes calibrated probabilities and fixed error costs. Correct decisions have zero assigned value in this example.
Change either cost to see why accuracy alone cannot tell you where to set it.
Open the full calculatorSame cases. Different consequences.
200 hypothetical cases. Move the cutoff to trade false alarms for misses.
Correctly passed
$075 true negatives
Actual: noFalse alarms
$25025 false positives
Actual: noMissed cases
$2,50025 false negatives
Actual: yesCaught cases
$075 true positives
Actual: yes Actual no Actual yes1 dot = 1 case
Net cost / 200 cases$2,750
At a 50% cutoff$2,750
Same cost as the 50% cutoff. 50 errors; 75% accuracy.
How this example works
200 constructed cases, not customer data or a forecast. Ten groups of 20 have probabilities of 5%, 15%, up to 95%; each has exactly the corresponding fraction of actual positives. Half the cases are positive overall. Your model may have a very different mix.
We flag scores at or above the cutoff. Dots keep their actual outcome as they move. Dollar totals count false alarms and misses, minus any assigned value of correct decisions. Negative net cost means net benefit.
The cost-based cutoff uses calibrated probabilities and fixed per-case costs. Finite groups make the counts change in steps; several cutoffs can produce the same decisions. This is an illustration of the trade-off, not evidence of savings on real work.
Selected work
All toolsEssay and interactive figures
What does the evidence let you conclude?
A regression and a randomized experiment can report the same elasticity. They support different decisions. The Nature of Knowledge follows that distinction through the claims a model can justify.
Explore the essayMore writing
All essaysThe Eighth Ledger: When Intellectual Labor Becomes a Capital Asset
Published draft, still being revised
What earlier technology transitions suggest about turning recurring knowledge work into owned software pipelines.
Any Sufficiently Advanced Technology: A Taxonomy of Magic
Every sufficiently advanced technology collapses one friction and reveals the new scarcity behind it. LLMs collapsed the cost of language, putting commodity intellectual output in the blast radius. A 6-step protocol, calibrated on 700 years of transitions from the printing press to the internet, for finding the new scarcity and moving first.
What AI Actually Made Scarce
AI made production cheap and revealed the constraint it was hiding: selection. Green dashboards stay green while quality quietly decays. A 30-minute protocol to find your new scarcity.
Work with me
I lead a technology organization spanning 13 retail brands. If you're interested in working together, write to me.
Frameworks
Concept graphs
All of this is one graph. The curated layer up top is hand-built; the dense graphs below are the working substrate. They cross-reference. Eventually they live in one searchable space.
Curated
Frameworks →Nine strategic models for finding, pricing, and allocating operating valueTools →Five interactive calculators for pricing operating instruments - diagnose, calibrate, invest, value, screenLexicon →37 defined terms - each one invented to name something that existed but had no wordPositions →Four overlapping circles - Allocator, Operator, Builder, Scientist - and the intersections where the value livesThesis →The full lattice - every framework, tool, and term on one page, organized by allocation stageI Have This Problem →Problem-to-solution index - find the right framework, tool, or term for your AI deployment challengeKnowledge graph →Interactive map of how 101 frameworks, tools, terms, positions, and posts connect
Dense
Larger working substrates. Each node is a lesson; edges are prerequisites and cross-graph references. Less polished than the curated layer - this is how I keep my own map.
Business →377 concepts from P&L mechanics to operating doctrine, each a full lesson with prerequisites and worked examplesMoney →52 personal-finance decisions structured as a dependency tree - the why behind the standard flowchartMath + ML →201 concepts from arithmetic to transformers, ordered by dependency with interactive visualizations