I’ve sat in a review where every step of an agent workflow had its own eval set, and every step cleared 95%. Extraction, routing, the tool call, the write-back — each one individually solid. Then the workflow went to a slice of real traffic and failed noticeably more often than any single step’s number suggested it should. Nobody had done anything wrong. Nobody had skipped a step. The math was just against them from the start.
The multiplication nobody runs
A workflow that chains steps only succeeds if every step succeeds. If each step has an independent 95% success rate, the odds of all of them landing is 0.95 raised to the number of steps, not 0.95 flat.
Five steps: 0.95⁵ ≈ 77%. Ten steps: 0.95¹⁰ ≈ 60%. A workflow built entirely out of steps that look production-ready in isolation ends up flipping a coin by the time you’re ten steps deep. None of the individual steps got worse. The chain did.
This isn’t unique to LLMs — it’s how independent failure rates have always combined, the same reason a service with five nines of uptime can still have a rough month if it depends on five other services with five nines of their own. What’s different with agents is how easy it is to keep adding steps. A tool call here, a verification pass there, a re-planning loop when the first attempt looks off — each one felt free to add, and each one you added multiplied straight into your end-to-end number.
Where it actually bites
It shows up hardest in the workflows people are proudest of: the ones with the most autonomy, the most tool calls, the longest planning loops. A single-shot classification or a one-tool lookup barely feels this. A workflow that reads a ticket, searches three systems, drafts a response, checks its own draft, and files an update is stacking five-plus opportunities for a 95%-looking step to be the one that misses — and the modal failure mode isn’t one dramatic error, it’s a small miss two steps in that everything downstream quietly builds on.
It’s also why a workflow can pass its offline eval and still feel flaky to users. The eval usually scores each step, or scores the final output against a held-out set that doesn’t stress every path through the chain. Production traffic finds the path you didn’t have a test for.
Fewer steps beats better steps
The instinct when a workflow underperforms is to go find the weakest step and improve it. That helps, but it’s fighting the wrong exponent. Cutting a nine-step workflow to six steps does more for end-to-end reliability than pushing one step’s accuracy from 95% to 98%, because you’re removing a multiplication, not shaving one of the numbers being multiplied. Before adding another verification pass or another tool call, ask whether it’s paying for the reliability it costs. Sometimes the honest answer is that a workflow doing three things well beats one doing seven things adequately.
Checkpoint state so a retry doesn’t restart the whole chain
If a nine-step workflow fails at step seven, the failure is cheap only if you don’t have to redo steps one through six to recover. Persist intermediate state after each step, and let a retry resume from the point of failure instead of from scratch. This doesn’t change the underlying math, but it changes what a miss costs — a step-level retry with saved state is a few seconds and maybe a token spend; a full restart is the user waiting again and your workflow eating the same exponent twice.
Put review where verification is hardest, not where the chain ends
Teams often put the human check at the end of the workflow, on the theory that’s where you catch everything. But a review at the end can only tell you the final output looks wrong — not which of the upstream steps caused it, and by then you’ve paid for every step already. It’s usually more effective to put a check at the step where an error is hardest to catch automatically and most expensive to inherit downstream, even if that’s step three of nine, not step nine of nine.
Measure the chain, not the average of its parts
If your dashboard reports each step’s success rate but not the end-to-end rate across real sessions, you’re measuring the thing that made the workflow look fine before launch. Track completion rate for the whole chain, on real traffic, and watch it the way you’d watch any other product metric. When it drops, the multiplication tells you the culprit probably isn’t the step everyone assumes — it’s whichever one is quietly running colder than its own eval said it would.
The individual steps were never the lie. Ninety-five percent was a true number for each of them. What it can’t tell you, on its own, is what happens when you chain five or ten of those numbers together and ask the result to hold up in front of a real user.
