Prefer listening over reading? 🎧 Brief (01:25 min) for the key ideas, or Deep-dive (27:03 min) for the full discussion. (AI-Generated Audio Versions)
Design AI Workflows for Reliability
When I discuss the reliability of an AI workflow, the point usually raised is this:
Suppose each step in an agentic workflow is right 99% of the time. In a workflow of 10 steps, the chance that everything is right drops to about 90%, and in a workflow of 50 steps it drops to about 60%. So, the “1% error rate” turns into a failure rate that you can feel in production.
While this is true, I believe it makes the problem sound like a verdict on AI (e.g. “the agents are unreliable”) when it is really a statement about structure. Two teams can build on the same model with the same per-step accuracy and ship very different products. One lets small mistakes travel quietly through the whole chain and surface as a confident wrong answer, while the other decides where a mistake is allowed to travel and where it is stopped.
Why errors compound (and when they don’t)
Errors usually compound when three conditions hold at once:
- The steps run in sequence
- Each step trusts the output of the one before it
- Nothing in between can tell a good output from a bad one
If we remove any of the three, the math changes. When I design an AI workflow, there are multiple areas I discuss with my teams. For example:
Sequential versus parallel
When agents work on independent pieces in parallel, mistakes add up roughly in line with the amount of work, but each one stays contained. When agents are chained, each step builds on the last, so one early mistake becomes the foundation for everything after it.
Loud versus silent
A crash, a malformed output or an empty field may be a cheap failure, because someone notices at the step where it happens. The expensive failure is the plausible wrong answer, such as a classification that looks reasonable, a number that is slightly off or a policy that sounds right but does not exist. This failure passes through every later step as if it were true.
Wrong in a way that matters
The “1%” needs a definition that everyone is aligned with. For example, a reply that is slightly stiff may be wrong in a way no one cares about. But a refund sent to the wrong customer may be wrong in a way that matters a great deal. The compounding math applies to the second kind, not the first, so a team that counts every imperfection will overestimate its problem and a team that counts none will underestimate it.
Why “add a reviewer agent” is a weaker fix than it sounds
Adding a reviewer agent is a common first move, and its value depends on how independent the reviewer really is. A reviewer built on the same model, with a similar prompt, can share the blind spots of the agent it reviews. When both agents are wrong in the same way, the check agrees with the mistake. Cross-checking will work far better on independent errors than on correlated ones. A check earns its keep when it is different in kind from the step it guards, such as plain code, a schema, a test, a lookup against a source of truth or a person with context the agent does not have.
The good news is that, because the end-to-end result depends on structure, you can change it without waiting for a better model.
Three questions to ask at every step
While the instruction “check the AI’s output at every step” may sound right, it is not very useful, as checking everything is slow and expensive, and defeats much of the point of automation. In addition, when people do the checking, attention usually fades and review turns into approval by habit. A better instruction is to decide deliberately, for each step, where a check goes and what kind it is. What has helped me make that decision concrete in the past is asking these questions:
What does a wrong output look like here, and would anyone notice?
This is about detectability. If a wrong output is loud, the workflow will usually take care of it. If it is plausible and silent, it needs a check or a redesign.
What does it cost if the error flows downstream?
This is about blast radius and reversibility. A draft that a person will read is cheap to get wrong. A payment that is made or a record that is overwritten is not. Put the strongest checks in front of the actions you cannot take back.
What is the cheapest check that is different in kind from the step itself?
Different in kind is the important part. Prefer a rule, a schema, a test or a lookup against a source of truth. Use a second model only when the first two are impossible, and use a human where judgement or context is truly needed.
I usually discuss these three questions for each step of the workflow, write the answers down, and the workflow stops being a black box. What we usually find out is that some steps need a strong check, while others need a light one, and a few need none (that is allowed, as long as it is a choice we made and can explain).
Examples
Let’s walk through the framework using two examples.
A support-ticket workflow
Assume a support workflow with three agent steps:
- Classify the incoming ticket
- Draft a reply based on company policy
- Issue a refund when the policy allows it
The risk is not spread evenly across the three steps, and the checks should not be either.
| Step | What a wrong output looks like | Check that is different in kind | Cost of the check |
|---|---|---|---|
| 1. Classify the ticket | A billing dispute tagged “general question”. Silent, and it routes everything after it wrongly. | Code rules that force high-risk words (chargeback, legal, cancel) to the right queue regardless of the agent. A small weekly sample audit by a person. | Low, e.g. a rules file and a small weekly sample review. |
| 2. Draft a reply from policy | A refund policy that sounds right but does not exist. | Require the draft to cite the policy clause, then confirm in code that the cited text exists in the real policy document. | Low, e.g. a string match, no second agent. |
| 3. Issue the refund | The wrong amount or the wrong customer. Irreversible. | Hard limits in code (amount cap, order belongs to the customer). A human approves only above a threshold. | Highest, but it fires only on the small share of cases above the threshold. |
Three things are worth mentioning.
- The error in the first step is the dangerous one even though it looks harmless, because nobody sees it and every later step inherits it.
- The policy check works because it compares the agent’s claim to a real document. A reviewer agent reading the same draft could easily accept a convincing invention.
- The human sits only where an error cannot be undone, not on every ticket.
In this example, the tone of replies on low-risk tickets can be deliberately unchecked. A slightly stiff reply costs almost nothing, and checking it would cost more than it saves.
A document-generation workflow
Now, assume a workflow that generates a client proposal. An agent extracts figures and facts from source files, drafts the sections, assembles the document, and sends it to the client.
| Step | What a wrong output looks like | Check that is different in kind | Cost of the check |
|---|---|---|---|
| 1. Extract figures and facts | A transposed number or a wrong date. Plausible and baked into everything after. | Validate against a schema, reconcile totals in code, spot-check a sample against the source files. | Low to medium. This is where most of the checking budget should go. |
| 2. Draft the sections | Mostly harmless variation in wording, except where the text restates a number or a commitment. | Automatically confirm that every number and date in the draft matches the extracted data. | Low, e.g. a comparison script. |
| 3. Assemble the document | Leftover placeholders, a missing section, totals that do not match. | A checklist in code. Nothing here needs an agent. | Near zero. |
| 4. Send to the client | A wrong price or commitment, sent outside the company. It cannot be recalled. | A person reviews a short list of risky items (prices, dates, commitments) pulled out of the document, not the whole document. | Human time, kept small by narrowing what the person looks at. |
In these two examples the pattern is the same: the strongest checks sit at the start of the workflow, where facts enter, and at the end, where an irreversible action happens. The middle stays light, because mistakes there are either harmless or caught by the checks on either side. Notice also that most of the checks are ordinary code. The agents do the work that needs flexibility, and deterministic checks guard the places where flexibility would be dangerous.
As before, we have left deliberately unchecked the stylistic phrasing in the drafted sections. Two reasonable ways of writing the same paragraph are not an error, and reviewing them would burn the attention that the final risky-items review needs.
Make mistakes cheap to undo
While checks try to stop errors before they land, in many cases reversibility reduces what the error costs after it lands, and it is often the cheaper fix. For example:
- Hold before acting. Delay sends, refunds and publishes by a short window so a bad one can be cancelled.
- Make the action undoable. Prefer soft deletes, versioned records and drafts over irreversible writes and sends.
- Limit the blast radius. Caps on amount, scope or volume mean a mistake that gets through is a small one.
- Detect after the fact. Log what each agent did, sample outputs for audit, and alert on odd patterns such as a spike in refunds. Not every error has to be caught inline if it can be found and fixed quickly.
None of these is meant to replace checks. Their aim is to change which steps need a hard check, because a step whose output can be undone cheaply needs less guarding than one that cannot.
Checks have a cost too
Every check we add has a price, such as:
- Latency. Each check adds time, and some add a wait for a person.
- Money and compute. A second model call on every step can double the cost of a workflow.
- Maintenance. Rules, schemas and test sets drift as policies, products and documents change. A stale check fails quietly.
- Attention. A person has a limited number of careful reviews in a day. Spend them on the risky ones.
A team that ignores the price of checks will either skip checks entirely or add so many that the workflow becomes slow, expensive and eventually ignored. I treat checks as a budget, not as a virtue.
When designing AI workflows and discussing what checks are required, it is worth keeping in mind:
Checks fail in their own ways
A check that raises too many false alarms teaches people to override it. A person approving hundreds of outputs a day is unlikely to stay a careful checker, because the next output looks like the last hundred. A noisy check can end up worse than none, because it adds friction while people learn to ignore it.
A rough heuristic for deciding
As a rough guide, a check earns its place when the likelihood of an error, multiplied by the cost of its impact, multiplied by how hard the error is to spot, is larger than the cost of running the check. Where that is not true, leaving the step unchecked is a decision, not an oversight. These factors are not in common units, so use the idea to structure a discussion, not as a calculation. Rough ratings of low, medium and high will separate the steps that need a hard check from the ones that do not.
Fewer steps often beat more checks
The cheapest way to remove a failure point is to remove the step, for example by merging agents or replacing one with plain code.
Write down what you chose not to check
A line in the design doc (e.g. “tone of low-risk replies: not checked, cost exceeds benefit”) turns a hidden risk into a recorded decision. It also tells the next person that the gap is on purpose, and gives you something to revisit if the stakes change.
Closing
I am not aiming to make my agentic workflows perfect. A 1% error rate is only a problem when nothing in the workflow can see it, stop it or limit what it can touch. The teams that build reliable AI products are not the ones with the most accurate model; they are the ones who decided, step by step, where mistakes are allowed to travel, who put strong checks of different kinds in front of the actions that cannot be undone, and who were honest about what they chose to leave unchecked.