The Model Is a Dependency That Fails Without an Error Code

Every senior engineer already knows how to design around an unreliable dependency. You put a timeout on it. You wrap it in a circuit breaker so a slow failure does not become a queued failure. You add a bulkhead so its bad day is not your bad day. None of that is new, and none of it is hard once you have done it a few times.

All of those patterns share an assumption that nobody states out loud, because until recently it was always true: the dependency tells you when it has failed. A timeout fires. A connection resets. A 500 comes back. The failure is a discrete event that ordinary code can branch on.

The language model is the first dependency most of us have integrated that breaks that assumption. It returns 200. The response is well-formed, schema-valid, syntactically perfect, and the content is wrong. There is no signal in the transport layer, no signal in the type system, and often no signal in the text itself, because plausibility is exactly what the thing is optimised for. This is the whole reason the gap exists between an agent that demos beautifully and one that holds up under real traffic, and it is why the agents that do hold up turn out, on inspection, to be mostly deterministic software that calls a model at two or three deliberately chosen points.

The arithmetic nobody runs before building

Start with the simplest agent, because it is the one everyone builds first and it is genuinely appealing. The model gets a context, returns a structured decision about the next step, your code executes that step, appends the result, and hands everything back. Loop until the model says it is done. You describe a goal, supply some tools, and the model works out the ordering. Very little branching logic to write.

Now do the multiplication. Suppose each individual decision is correct 95% of the time, which most people would call reliable. Chain twenty of them and the probability that the whole run is clean is 0.95²⁰, which is roughly a third. Nothing failed loudly. The per-step accuracy you would have shipped without hesitation just produced a system that gets the task wrong twice out of three attempts.

The number itself is illustrative, not measured, and your real per-step accuracy varies wildly by task. The shape is what matters. Reliability compounds downward with chain length, so every additional step the model is allowed to take on its own is a multiplication you are agreeing to. That single fact explains most of what production teams do to their agents, and it reframes the design question from "how autonomous can I make this" to "how few multiplications can I get away with".

There is a second, nastier property. Because the failure is silent, a wrong step does not stop the chain. It gets appended to the context as though it were true, and every subsequent decision is now conditioned on it. A retry does not help, because the failure is not transient. A circuit breaker does not help, because nothing tripped. The resilience patterns you already trust were built for infrastructure failure and are completely blind to this one.

Shrink the decision, not the model

The most useful change I have made to agents I own is not switching models or rewriting prompts. It is narrowing what the model is being asked to decide on any individual call.

There is a large difference between asking a model "what should happen next" and asking it "which of these three things applies here". The first has an unbounded output space and the failure mode is invention. The second is a classification with three outcomes, where two thirds of the wrong answers are still inside a set your code already knows how to handle. Same model, same context, wildly different blast radius.

This is why structured output matters more than it appears to. It is easy to read a JSON schema as a serialisation convenience, a way to avoid parsing prose. It is really a constraint on the decision surface. An enum of four values cannot hallucinate a fifth. A parameter typed as a date cannot come back as a paragraph explaining why the date is uncertain. Every field you can narrow from free text to a closed type is a category of silent failure you have converted into either a valid answer or a schema violation your code can actually catch, which is the closest thing to an error code you are going to get.

The same logic applies upward to tool design. A tool is a function the model can request, and the description plus the schema is the entire interface it has to reason about. When two of your tools overlap enough that a competent engineer would have to guess between them, the model will guess too, and it will guess silently. If you cannot state in one sentence which tool applies in a given situation, no amount of prompting fixes that, because the ambiguity is in the surface you designed rather than in the model reading it.

The loop belongs to your code

Once you accept that each model call is a probabilistic step, the question of who decides how many steps to take stops being a stylistic preference.

In an agent that survives contact with real traffic, the control flow lives in ordinary code. A sequence of steps, some conditionals, calls to external systems, and at two or three points where the problem genuinely requires open-ended judgement, an invocation of the model. Everywhere else, deterministic logic does the work, because it is cheaper, predictable, and testable in the way the rest of your system is testable.

Two things make this hold. The loop needs a hard stop that does not depend on the model agreeing to stop: an iteration cap, a wall-clock timeout, an explicit completion condition, ideally all three. A loop whose only termination signal comes from the same component that might be confused is not bounded in any meaningful sense, and the bill arrives before the bug report does.

The other is that model calls should be deliberate rather than default. It is worth being suspicious of any step where you invoked the model because it was easier than writing the branch. That step costs tokens, adds latency, and multiplies into the reliability arithmetic above, and if the logic could have been specified in advance then you paid all three for nothing.

Intercom's Fin is a good public example of what this looks like at scale rather than in a diagram. Their procedures let the agent reason in natural language while the surrounding system supplies conditional steps, deterministic code snippets for the parts that must be reproducible, and checkpoints where the run pauses for human approval before anything sensitive happens. Klarna's assistant is the same idea from the traffic side: they reported roughly 2.3 million conversations in the first month, and the overwhelming majority of those were predictable paths like refund status and order tracking, which structured logic handles perfectly well without asking a model to improvise.

Keep the memory outside the model

If the control flow is yours, the state should be too.

The model starts every call fresh. It reads the context you hand it, produces one decision, and retains nothing. People treat that as a limitation to work around, when it is the most useful property available: it means the entire memory of the system can live in storage you control, and the context becomes something you construct on each call rather than something that accumulates by default.

The practical payoffs are the ones you would expect from any stateless service. A run that crashes halfway can be reloaded and resumed, because the plan and the progress are rows rather than tokens. A long-running task can pause for a human and pick up two days later. Any instance behind the load balancer can serve any request, because nothing is pinned to the process that started the work.

There is a subtler payoff that matters more for correctness. When the state lives in your database, you have a ground truth to reconcile the model's view against. The model believes the invoice was sent; your records say it was not. Keeping those two in agreement, and treating your record as authoritative when they disagree, catches a class of compounding error before it propagates. I have written about the storage and retrieval side of this in agent memory, and the pattern is the same one you would apply to any supervisor architecture: the coordinator owns the truth, the workers are disposable.

The cleanest way to think about the model in this design is as a pure function from current state and new event to next state and action. Same inputs, same shape of output, no hidden memory. That is a component you can test.

Blast radius is a design parameter

All of the above reduces how often the agent is wrong. None of it makes the agent right. So the last question is what a wrong answer is allowed to touch.

The public failures are instructive precisely because they are boring. Cursor's support bot told customers about a one-device-per-subscription rule that the company later confirmed did not exist, and cancellations followed. Air Canada was held liable by a tribunal for a bereavement fare policy its chatbot invented, and was ordered to honour it. Neither of those was an exotic model failure. Both were ordinary hallucinations with an unmediated path to a customer, which is a systems decision rather than a model decision.

So separate your actions by reversibility and gate them differently. Reading a record, drafting a reply, searching a knowledge base: cheap to be wrong, let the model drive. Sending an email, issuing a refund, changing an entitlement, telling a customer what your policy is: expensive and often irreversible, so those get a confirmation step, a policy check in code, or a human.

Which makes the human handoff a first-class path rather than a failure branch. It is reached on purpose when confidence is low, when the action is sensitive, or when someone asks for a person, and it works properly only if you designed the state to brief that person with. An escalation that dumps a raw transcript on a support agent is a handoff in name only. This is also the honest answer to the scope question: a narrow agent with one well-defined job and a small surface, composed under deterministic orchestration with other narrow agents, is easier to test, easier to reason about, and easier to bound than one broad agent holding a long list of responsibilities.

Where the arguments actually are

Two things here are genuinely unsettled and I would rather say so than present a tidy consensus.

The first is single versus multiple agents. In June 2025 the Cognition team argued against multi-agent designs on the grounds that parallel sub-agents make independent decisions that then conflict, and reconciling those conflicts costs more than it saves. A day later Anthropic published their multi-agent research system, which burned something like fifteen times the tokens of a single-agent approach and scored substantially better on their internal research evaluations. Both were right about their own workload. The pattern that has settled out since is an orchestrator that owns the full context and spawns short-lived, isolated sub-agents that each complete one task and return a summary, which is close to what I described in multi-agent topologies. Sub-agents that talk directly to each other are where it stops working. The token bill is real and worth modelling before you commit, which I have gone into separately in token economics.

The second is Sutton's bitter lesson, which says general methods that scale with computation beat handcrafted techniques over time. Applied here, it is a warning that a good portion of this scaffolding exists to compensate for today's models and will be dead weight in eighteen months. That is partly true and it is the strongest argument against over-engineering your harness. It is not the whole story, though, because several of these constraints do not come from model weakness. A finite context window, the need to pause and resume safely, the need to reconcile the model's belief with your database, the requirement that an irreversible action be authorised by something accountable: those survive better models, because they are properties of the system rather than of the reasoning inside it.

What I would actually do, and what I would push back on in a design review, is the same either way. Build the simplest version that solves the problem, which usually means more deterministic code and fewer model calls than the first design proposes. Measure where it falls short with something you can score, even crudely. Then grant autonomy at the specific points where the measurement says judgement is worth more than predictability, and nowhere else. Autonomy is not a feature you add because the framework supports it. It is a multiplication you accept because the alternative was worse.

I write about system design and the senior-to-staff transition every week in Monday BY Gazar on Substack, and I break down architecture and engineering decisions on Gazar Breakpoint on YouTube.

If you are putting an agent in front of real traffic and want a second opinion on where the model gets control, I do 1:1 mentorship.