Start Here

What you will be able to design by the end

Before the work begins, the payoff. By the last module you will have designed six production LLM systems end to end, and, far more useful, you will run one repeatable method on any "add AI to this" you are handed.

Not "I have used the API." "I can design the system."

There is a gap between calling a model successfully and being able to stand at a whiteboard and design the system around it: what the model decides, what it costs, how it degrades, what stops it, and how you would know it got worse. This course closes that gap.

This is the shape of the thing you will be able to draw and defend by the end.

Every production LLM system is a variation on this skeleton. What changes is which box carries the hard part.

Six systems, six signature hard parts

Each worked design in the capstone exists because of one thing it forces you to solve. The system is the vehicle. The hard part is the cargo.

System	Signature hard part	What it forces you to solve
Support triage and refunds agent	Authority boundaries	What the model may do itself, what it may only propose, and where a human sits
Doc-QA over a private corpus	Retrieval quality and freshness	Chunking, hybrid search, re-ranking, and what happens the day a document changes
Repo-aware coding agent	Long horizons and blast radius	Context that outlives the window, and bounding what a wrong step can damage
Bulk document extraction	Cost at scale	Turning a per-request price into a per-million-documents budget that survives review
Real-time voice agent	Latency budget	Spending a fixed number of milliseconds across every hop, and what gets cut first
Multi-tenant internal gateway	Isolation and provider drift	Quotas per team, and detecting the day the model changed underneath everyone

Authority, retrieval, horizon, cost, latency, isolation. That list covers most of what you will ever be asked to build.

The goal is not to memorise the six

If all you leave with is six cached answers, you will freeze on the seventh. The six are vehicles. What you are extracting is the method that produced them.

Move	What it gets you	Where it breaks
Copy a reference architecture	A diagram for the system you copied	The moment a constraint differs, and one always does
Own the method	A place to put every decision on any prompt	Nowhere. It degrades to "here is how I would find out"

Seven steps you run on every LLM design

The method is the same every time. What changes is the numbers and the signature hard part.

Step	What you produce	The question it answers
1. Task and boundary	What the model decides, what code decides	Who is driving, and where does the model's authority stop?
2. Context plan	What goes in the window, and how it gets found	What does the model need to see, and where does it come from?
3. Cost and latency budget	A per-request number for both	What does this cost, and how slow is it at p95?
4. Failure ladder	Degradation path, per failure mode	What happens when the model, the tool, or the retrieval fails?
5. Threat model	Trust boundaries and what crosses them	What can this be talked into doing, and what stops it?
6. Eval plan	The set, the scorers, the gate	How do I know it is good, and that it stayed good?
7. Operability	Signals, traces, runbook	How would I know at 3am, and what would I do?

The steps run in order, but the dotted arrows are the honest part. A budget you cannot meet sends you back to narrow the task, and an eval failure usually turns out to be a context problem, not a model problem.

Every step ends in a decision, and a decision is only worth writing down if it carries its price and its trigger.

Typescript

type Step =
  | "boundary"
  | "context"
  | "budget"
  | "failure"
  | "threat"
  | "eval"
  | "operability";

interface DesignNote {
  step: Step;
  decision: string;
  cost: string;
  revisitTrigger: string;
}

const refundAuthority: DesignNote = {
  step: "boundary",
  decision: "The agent may issue refunds under 50 dollars; above that it drafts and a human approves.",
  cost: "Roughly 30 percent of refunds still need a person, so the savings are capped.",
  revisitTrigger: "Approval agreement above 98 percent for a full quarter, then raise the threshold.",
};

Four fields: which step, what you chose, what it cost, and what would make you change your mind. A finished design is a handful of these, in order. That is exactly what the capstone project asks you to produce.

Recap

Six worked designs, six hard parts: authority, retrieval, horizon, cost, latency, isolation.
One seven-step method: boundary, context, budget, failure, threat, evals, operability, run on every design.
The real skill is transfer, not the six answers but the method that handles a seventh you have never seen.
Next: you have finished Start Here. Set your goals in the pre-course project, then module 1, where the first real decision is whether you need an agent at all.