Your Context Window Has a Standing Charge, and Almost Nobody Has Counted Theirs

Ask an engineer how big their context window is and you get an instant answer. Ask what their agent spends before the user has typed anything and you get a pause, then a guess. Your tool definitions are re-sent on every single request, before the model does anything you asked for. That is not a one-off. It is a standing charge, and it grows every time somebody wires in another server. In this session I walk through a real setup on screen. Three ordinary MCP servers. Twelve tools. About 3,000 tokens gone before the user says hello. The tool schemas cost roughly five times what the system prompt does, and the system prompt is the part people edit. Then a teammate adds one more tool. It goes on the end of the tools array and it is never called. Most people expect the tools cache to die and the rest to hold. It does not work like that. Tools render at position zero, so the end of your array is the front of the request, and every cache breakpoint after it dies too. Nothing errors. The only symptom is the bill. The rest is what to do about it: what caching is actually worth across 1, 10, 50 and 200 turns, why accuracy drops as the window fills, and why most production systems should be lots of small models and ordinary code rather than one big model holding everything. What you will learn: - What your tool definitions really cost on every request - Why a context window is a per-turn budget, not a size - Why "the end of the array" is the front of the request - What prompt caching is worth across 1, 10, 50 and 200 turns - Semantic caching, and when it is worth building - Why models lose the middle of a long context - How to count tokens at every step, for every agent you run - Why you rarely need a frontier model for the whole job