The one idea: control and observability over systems you do not own
Every decision in this course comes back to one uncomfortable fact. The most important part of your system is one you did not write, cannot read, and cannot pin down. Everything else follows from taking that seriously.
You own less of this system than you are used to
In a normal service you can read every line that runs. In an LLM feature, the part doing the interesting work is a remote service behind an API. It is probabilistic, which means the same input can give you a different answer twice. And it sits underneath a stack of other things you also did not write.
Only the top box is yours. Everything below it can change without a version number changing. The dotted line coming back up is untrusted input, no matter how confident it sounds.
The instinct this should produce is not fear. It is the instinct you already have about a third-party payment provider, or a service owned by another team. Put a boundary around it. Budget for it. Watch it. Have a plan for when it misbehaves.
Control means deciding what the model is allowed to decide
Control is not about writing a better prompt. It is about drawing a line between what your code decides and what the model decides, then keeping the model on its side of that line.
| Decision | Give it to code when | Give it to the model when |
|---|---|---|
| Which path a request takes | The categories are known and stable | The categories are open-ended and fuzzy |
| Whether an action is permitted | Always. Permission is never the model's job | Never |
| What text to produce | Never, that is why you are here | Always |
| How many times to retry | Always. A loop the model controls is a loop with no limit | Never |
| Which of five known tools to call | You can decide from the input alone | The choice genuinely depends on reading the content |
The pattern in that table is the whole of module 1: the model produces judgment, your code keeps authority. If the model can decide that it is allowed to do something, you have no security boundary at all.
Observability means being able to answer "why did it do that?"
The second half is being able to rebuild the story of any single request after the fact. Not logs of your own code, which you already have. The whole chain.
| Question you will be asked | What has to exist for you to answer it |
|---|---|
| Why did it say that? | The exact text that went in, not the template it came from |
| Why did that cost so much? | Token counts per call, tied to a request and a feature |
| Why was it slow? | A timing record per hop, so you can see which one waited |
| Why is it worse than last month? | Scored runs over a fixed set of examples, from before and after |
| Which tool call did that? | The trajectory: every step the agent took, in order |
Not one of those is answerable from the model's response alone. They all need something you decided to capture at the time. That is the point. Observability is a design decision you make before the incident, not a debugging trick you reach for during one.
// The minimum record that makes all five questions answerable later.
// Emit one per model call; everything in this course reads from it.
interface LlmSpan {
traceId: string; // ties every hop of one user request together
step: number; // position in the agent's trajectory
model: string; // the exact model string sent, not "the small one"
promptHash: string; // which prompt version produced this
inputTokens: number;
outputTokens: number;
costCents: number; // computed at emit time; prices change
ms: number;
toolName?: string; // set when this step called a tool
outcome: "ok" | "refused" | "invalid_output" | "tool_error" | "timeout";
}Every later module is, in some form, a use of this record. Cost budgets add it up. Fallback ladders read the outcome. Evals replay it. The runbook you write in the final module is a set of queries over it.
Two failure modes this prevents
Teams that skip this end up in one of two places. Both are expensive.
| Failure mode | What it looks like | What it costs |
|---|---|---|
| No control | The agent has a tool that can refund money, and nothing sits between the model's decision and the refund | One convincing user, or one poisoned document, and it acts |
| No observability | Quality drops, and nobody can say when, why, or by how much | Weeks of guessing, and a rollback you cannot justify |
Neither is caused by a weak model. Both are caused by shipping a system whose boundaries and signals were never designed.
Recap
- You own the top box only. The framework, the gateway and the model can all change under you.
- Control is a boundary, not a better prompt. The model supplies judgment, your code keeps authority over actions, retries and permissions.
- Observability is a design decision made in advance, and one record per model call is what makes the awkward questions answerable.
- Next: lesson 0.4, what you will be able to build and defend by the end.