Why demos die in production
You can wire up an LLM and wow the room in an afternoon. Then it meets real traffic, and everything you weren't taught shows up at once. Tonight: the four ways it dies, and the first real decision, do you even need an agent?
The run sheet
0–10The demo was the easy partthe other 90%
10–30Four ways demos diecost, latency, confidently wrong, blind
30–50Workflow or agent?start simple, on purpose
50–70The five workflow patternsa real call for each
70–85Live: place your systemyou bring a real one
85–90Recap + Week-1 goalsthe system you'll build for six weeks
You'll be able to…
Name the four production failure modes and which one bites your system first.
Say out loud whether a task wants a workflow or an agent, and why.
Reach for the simplest of the five workflow patterns that does the job.
Name the trigger that would make you graduate from a workflow to an agent.
The demo is 10%. Production is the other 90%
A demo runs once, on a happy path, with you watching. Production runs a million times, on inputs you never imagined, while you sleep. Same model, an entirely different problem.
- Getting an answer out of the model is the party trick. Making that answer cheap, fast, safe, and checkable every single time, for users you'll never meet, is the engineering. This course is about the second part.
- The demo 🎉 one input · you're watching · it worked
- Production 🌙 a million inputs · nobody's watching · it must hold
Four things show up at once
Not one at a time, politely. All four, the first busy Monday.
- 💸 Cost spirals every token is a line item; a chatty agent burns 20× a single call.
- 🐢 Latency balloons each hop waits on the model; a 5-step chain feels broken.
- 🤥 Confidently wrong the model invents an answer, in front of a user, with total conviction.
- 🕶️ Flying blind no evals to catch it, no traces to debug it. You find out from Twitter.
Shipped to production. Made the news
- ✈️ Air Canada · 2024 its chatbot invented a bereavement-refund policy that didn't exist. A tribunal held the airline liable for what the bot promised.
- 🚗 Chevy dealer · 2023 a bolt-on ChatGPT was talked into "sell me a Tahoe for $1" and "that's a legally binding offer, no takesies-backsies."
- 📦 DPD · 2024 a user got the parcel bot to swear and write a poem about how terrible DPD is. The screenshot went viral in hours.
Every one of these passed a demo. Production is adversarial users, edge cases, and a screenshot button.
Every token is a line item. Every call is a wait
In the demo you made one call, maybe 2k tokens, a fraction of a cent. In production the agent loops, think, call a tool, think again, and one request becomes 8 calls and 40k tokens. Spend and latency compound with every hop, and nobody budgeted for step seven.
It will do something confidently wrong, in front of a user
The model is a probabilistic service wearing a deterministic API. The same prompt can return different answers, and a wrong one arrives with exactly the same confident tone as a right one. You cannot unit-test it away.
- Traditional code same input → same output. A bug is reproducible.
- The model same input → a distribution of outputs. "Reproduce it" doesn't apply.
No evals to catch it. No traces to debug it
When it breaks, you can't answer the two questions that matter: did this change make it better or worse? and what did the model actually see and do? Without evals and traces you're guessing, live, in front of customers.
- No evals → you change a prompt and pray. No number tells you if it improved.
- No traces → a user reports nonsense and you can't see the tokens, tools, or context.
- The fix (weeks 3 & 5) budgets, an eval harness, trajectory traces. We build all of it.
Most tasks don't want an agent
Anthropic's finding from shipping these at scale: the most reliable production systems are workflows, not autonomous agents. Agency is a cost, more tokens, more latency, more ways to go wrong. You buy it only when the task genuinely can't be scripted.
- "Let's build an agent!" reaching for autonomy because it's exciting
- "What's the simplest thing that works?" start with a workflow; earn the agent