The $95 Education of an LLM Cache
Pre-writing 6,000 field notes with an LLM cost more than the plan said and taught more than expected, including which model quietly narrates its own prompt.
Why a Cache at All
Emic writes a plain-language field note for every concept in every ontology, about 6,000 of them. Generating those live, on demand, would mean an API key, a per-visit cost, and a public site that breaks the moment the bill runs out. So the notes are pre-written once, in a batch, and shipped as static files. (There is a live-generation tier, but the public site never touches it; it’s read-only by design.)
A cache turns a recurring cost and a runtime dependency into a one-time bill and a folder of JSON.
The One-Time Bill Wasn’t What the Plan Said
The plan budgeted about $25 to seed CIM on Sonnet. CIM actually cost $38.63, written on Opus, with more tokens per note than estimated. Across all six ontologies the corpus came to roughly $95:
- CIM: $38.63 (Opus)
- D3FEND: $54.79 (Sonnet)
- SPDX: $0.52
- BFO: $0.83
- FOAF: $0.13
- SSN: $0.04
Two ontologies account for almost the entire bill, because two ontologies hold almost all the concepts. The rest is rounding error.
Seeding cost scales with concept count, not ambition. The expensive ontologies are simply the big ones.
The Audit That Earned Its Keep
A cache you generate once is a cache you should check once. So I ran an audit over all 6,016 notes, and it paid for itself immediately.
The first finding was mechanical. 24% of the notes (1,469 of 6,016) leaked the scaffolding: phrases like “the graph context” or “the provided context,” the model narrating its own prompt instead of just describing the concept. A field note is supposed to read like a field note, not like a model talking to itself.
The interesting part was where the leak concentrated.
The same defect ran at 34% in D3FEND and near-zero in CIM. The only thing that changed was the model.
D3FEND was seeded on Sonnet and CIM on Opus, with the same prompt and the same task. The gap is a measurable, after-the-fact signal of model quality, surfaced entirely by accident.
The Part Where the AI Just Made Things Up
A closer qualitative pass turned up the worse category: genuine fabrications. One note invented a Last Accessed property on a class that has no such thing. Another described “Product” endpoints that simply don’t exist in the schema. Plausible, confident, wrong: exactly the failure mode you’d expect from a system that optimizes for plausibility rather than truth.
The field notes are aids, not authorities, and the audit is what lets me say that honestly instead of hopefully.
Fixing It Two Ways
The cleanup matched the two kinds of defect:
- The scaffolding leaks were mechanical, so the fix was deterministic: clause-removal that strips the meta-reference while preserving the surrounding paragraph. 1,328 notes cleaned this way.
- The fabrications needed judgment, so the 16 confirmed cases were regenerated by an LLM under a tighter prompt.
Together, that took the mechanical defect rate from 24% down to 2%.
The Takeaway
The Takeaway: Pre-writing knowledge with an LLM is cheap enough to be worth doing and unreliable enough that you have to check your work. The whole corpus cost about $95; the audit is what made it trustworthy. Both the bill and the error rate belong in the open.