Track · AI in Product5 min read
100+ AI Agents Fail by Silent Drift, Not Errors: Monte Carlo Data
Data from 100+ production AI agents shows 89% alerted within the study period; error checks fired first for just 1%, while quality and token alerts led at 38% each.

Context
89% of 100+ monitored agents raised at least one alert; median time to first alert was six days.
Errors were among the first alerts for only ~1% of agents; quality and token alerts each led at 38%.
92% of token-spike days coincided with another warning sign, versus 48% of ordinary days.
Only 8% of token checks tie spend to a specific tool or step; 72% of loop alerts involve tool and SQL calls.
Governance and safety checks cover fewer than one in ten agents.
Only about 1% of production AI agents showed an error among their first warning signals, while quality and token-spend alerts led at 38% each — evidence from more than 100 monitored agents across dozens of companies that agents fail by silent drift rather than visible breakage. The seven-month analysis, run on alerting data from Monte Carlo's observability platform between April 9 and November 2, 2026, found that 89% of monitored agents raised at least one alert, with a median time to first alert of just six days.
Traditional software fails loudly: exceptions, 500s, climbing error rates. AI agents do not. An agent can keep returning confident, well-formed answers while becoming less accurate, more expensive, or slower, and none of that registers as an obvious error. That is precisely what the data showed.
What fired first, and what fired late?
Monitors watching how well and how efficiently an agent works fired early and often. 83% of agents with output quality evaluations — LLM judges scoring answers for helpfulness, groundedness or task completion — raised a quality alert, with a median of eight days to the first one. Roughly three-quarters of agents with token-spend and latency checks alerted, typically within eight or nine days. Loop checks fired almost immediately, with a median of one day.
Error checks were the outlier. Fewer than half the agents with error checks ever raised an error alert, and those that did took a median of 42 days — about six weeks. By the time an agent produces something an error check recognizes, it has usually been giving worse answers, burning more tokens, or running slower for weeks. A monitoring setup that starts with error rates watches the last thing to change.
Do agent problems come back?
Yes, persistently. For every check type, most agents that alerted did so on three or more separate days. One-off alerts were the minority: 16% for quality evaluations, 20% for token spend, and roughly a third for latency and errors. Runaway loops were the most chronic: 78% of agents with a loop alert had loop alerts on three or more days, with a median of nine separate days. A caveat — some checks re-alert on the same ongoing condition — but the practical point holds: one-time patches rarely fix agent problems.
Do cost, quality and speed move together?
Token spend proved a good test. About three-quarters of agents with a token check raised at least one anomaly, and fewer than 3% of those were drops. On 92% of spike days, at least one other warning sign appeared nearby, versus 48% of ordinary days. Quality or behavior alerts accompanied 58% of spikes against 19% of other days. Latency and error alerts were 3.4 times as likely around a spike; loop alerts 2.8 times. In a third of cases, other agents at the same company spiked simultaneously, pointing to shared causes such as traffic surges or model changes. None of this proves causation for any single spike, but it fits the drift pattern: when an agent changes behavior, cost, quality and speed move at once.
Where do loops hide?
In tool calls, where few teams look. 72% of loop alerts involved tool and SQL calls — an agent re-running the same query, retrying a failing tool, or re-planning the same step. Yet of agents with any latency or time-limit check, 84% time only the end-to-end run and just 18% time individual tool calls. Four in five agents monitored for loops have no tool-call timing at all. Token monitoring shares the blind spot: 62% of token checks watch a whole run or conversation, another 24% watch daily or hourly totals, and only 8% tie to a specific tool, step or span.
The governance layer is thinnest of all. Checks for prompt injection, PII leakage, out-of-scope requests and policy violations covered fewer than one in ten agents. These categories come from monitor names, so the figures are floors — but even allowing for that, the gap is large. Notably, 57% of customer-built monitors were written by describing what they wanted to an AI in a chat interface; LLMs default to suggesting quality, accuracy and tone checks, and rarely propose tool-call latency loops, data-freshness traps or injection detection. AI-generated monitoring replicates our own blind spots.
What should teams do Monday?
- Put quality evaluations, token-spend and latency monitors in place first; error rates are the last signal to move.
- Treat an agent's alerts together over weeks, not as isolated tickets.
- Time individual tool calls, count repeated calls per run, and tag token counts to specific steps.
- Add explicit PII, injection and scope checks, since safety checks don't appear by default.
- Audit your own alert history: if error checks fire first and quality checks never fire, your quality checks are probably too loose.
The limitations matter. An alert is a threshold breach, not a confirmed failure; quality checks outnumber error checks per agent, inflating quality figures; over half of monitors were set up by the platform or its staff, and conservative default error thresholds may make errors look quieter than they are. The sample also skews toward teams already investing in commercial monitoring, and obvious errors may be caught by existing application monitoring tools instead.
The clearest takeaway: agent failure emerges as drift — deteriorating quality, rising latency, slowly increasing cost — not a screaming error. As agent deployments multiply through 2026, the teams that instrument tool calls and step-level spend, rather than run-level totals, will be the ones that catch drift before users do.
via towardsdatascience.com (Source)
More from Nathan Brooks
Show full bio
Senior reporter covering companies, deals and strategy in product management at Roadmap File.
56 articles