The Cost of a Token

The demo cost nothing anyone noticed. Ten requests, a few cents, applause. Then the feature shipped, ten requests became ten thousand a day, and the invoice arrived with a name on it. Someone now owns the AI bill. It has become a line item, a budget, a small project of its own.

The reflex is to blame the model's price. It is almost never the price. It is how you spend tokens.

The cache you already have

Every call re-sends its context: the system prompt, the instructions, the retrieved documents. Anthropic caches that context, so you pay full freight once and a fraction after. A cache read costs one tenth of a normal input token.

But you do not switch it on. You earn it. The cache lives for five minutes, its TTL, and that clock resets every time you hit it. Keep working and it stays warm at a tenth of the price. Walk away for five minutes and it expires; the next call pays a write again, at one and a quarter times the base.

Cache hit10%

Base input100%

Cold cache write125%

So the cheapest conversation is the one that stays warm and stays small.

One conversation, one job

A sprawling, all-day session is the expensive shape. It idles between turns, so the cache keeps going cold. It accumulates history, so every turn re-sends more than the last. And a bloated context makes the answers worse, not better.

The cure is the instinct that already makes good code: one thing, done well. A method does one job. In an agent fleet each subagent gets one mission and a hook guards one tool call. A conversation deserves the same discipline: short, intense, dedicated to a single task, then closed. Small context, warm cache, sharper answer. Cheaper and better turn out to be the same move. And when a session has to keep going, compacting it, replacing the transcript with a summary, buys back a small context for the price of one cache rewrite.

Keep the knowledge outside the window

The other half is not carrying everything you know in the prompt. Your whole codebase does not belong in context, and neither does every past decision. Retrieve what the task needs and leave the rest on disk.

That is what these tools are for. Claude Context indexes a codebase into a vector database and pulls only the relevant snippets into the window. MemPalace keeps memory across sessions, so each new one starts small instead of replaying the last. Both shrink the context you pay for on every single call. Retrieval is a cost lever before it is an accuracy one.

The bill is a design decision

Two levers finish the picture. Send the easy work to a small, cheap model and keep the expensive one for the hard cases. And for your own daily work, a flat subscription beats metering yourself by the token; per-token pricing is for the product, not the person.

None of this turns anything off. The feature stays as capable as the demo. What changes is that the bill stops being a fixed cost you dread and becomes a number you shape. Spend tokens the way you spend attention: on one thing at a time.