Start Here

The one idea: control and observability over systems you do not own

Every decision in this course comes back to one uncomfortable fact. The most important component in your system is one you did not write, cannot read, and cannot pin. Everything else follows from taking that seriously.

You own less of this system than you are used to

In a normal service you can read every line that runs. In an LLM feature, the part doing the interesting work is a remote, probabilistic service behind an API, and it sits underneath a stack of things you also did not write.

Only the top box is yours. Everything below it can change without a version bump, and the dotted line back is untrusted input, no matter how confident it sounds.

The instinct this should produce is not fear. It is the same instinct you already have about a third-party payment provider or an upstream service owned by another team: put a boundary around it, budget for it, watch it, and have a plan for when it misbehaves.

Control means deciding what the model is allowed to decide

Control is not about constraining the prompt. It is about drawing a line between what your code decides and what the model decides, and then keeping the model on its side of it.

Decision	Give it to code when	Give it to the model when
Which path a request takes	The categories are known and stable	The categories are open-ended and fuzzy
Whether an action is permitted	Always. Authorisation is never the model's job	Never
What text to produce	Never, that is why you are here	Always
How many times to retry	Always. A loop the model controls is a loop with no bound	Never
Which of five known tools to call	You can decide from the input alone	The choice genuinely depends on reading the content

The pattern in that table is the whole of module 1: the model produces judgment, your code keeps authority. A system where the model can decide it is allowed to do something is a system with no security boundary at all.

Observability means being able to answer "why did it do that?"

The second half is being able to reconstruct any single request after the fact. Not logs of your own code, which you already have. The whole chain.

Question you will be asked	What has to exist for you to answer it
Why did it say that?	The exact context that went in, not the template
Why did that cost so much?	Per-call token counts, attributed to a request and a feature
Why was it slow?	A span per hop, so you can see which one waited
Why is it worse than last month?	Scored runs over a fixed set, from before and after
Which tool call did that?	The trajectory, every step the agent took in order

Notice that not one of those is answerable from the model's response alone. They all need something you decided to capture at the time. That is the point: observability is a design decision made before the incident, not a debugging technique applied during one.

Typescript

// The minimum record that makes all five questions answerable later.
// Emit one per model call; everything in this course reads from it.
interface LlmSpan {
  traceId: string;        // ties every hop of one user request together
  step: number;           // position in the agent's trajectory
  model: string;          // the exact model string sent, not "the small one"
  promptHash: string;     // which prompt version produced this
  inputTokens: number;
  outputTokens: number;
  costCents: number;      // computed at emit time; prices change
  ms: number;
  toolName?: string;      // set when this step called a tool
  outcome: "ok" | "refused" | "invalid_output" | "tool_error" | "timeout";
}

Every later module is, in some form, a use of this record. Cost budgets sum it, fallback ladders read the outcome, evals replay it, and the runbook you write in the final module is a set of queries over it.

Two failure modes this prevents

Teams that skip this end up in one of two places, and both are expensive.

Failure mode	What it looks like	What it costs
No control	The agent has a tool that can refund money and nothing between the model's decision and the refund	One convincing user, or one poisoned document, and it acts
No observability	Quality drops, nobody can say when, why, or by how much	Weeks of guessing, and a rollback you cannot justify

Neither is caused by a weak model. Both are caused by shipping a system whose boundaries and signals were never designed.

Recap

You own the top box only. The framework, the gateway and the model can all change under you.
Control is a boundary, not a better prompt: the model supplies judgment, your code keeps authority over actions, retries and permissions.
Observability is a design decision made in advance, and one span record per model call is what makes the awkward questions answerable.
Next: lesson 0.4, what you will be able to build and defend by the end.