Seven in Ten Enterprises Cannot Tell Which Agent Broke Production

Published: at 03:50 AMSuggest Changes

The refund rate on the storefront tripled at 03:12. The paging worked. The dashboard showed the spike in eleven seconds, the burn-rate alert fired, and by 03:20 four engineers were on the bridge with the trace explorer open. At 03:58 someone asked the question the incident could not answer: which agent did this?

The estate held six agents with write access to order state. Six invocations sat inside the window. Every span looked healthy on its own — HTTP 200s, well-formed JSON, token counts in band, no error types. The bad refund had been issued by a fulfilment agent acting on a plausible but wrong order status, produced three hops earlier by an orchestration agent that had trusted a cache entry nobody had invalidated. No span was red. The failure was in the space between spans.

Detection and attribution are different capabilities, and most enterprises have bought one of them. Detection answers whether the system is broken. Attribution answers which agent, which step, whose authority, and what now has to be reversed. The second question is the one that costs money at 03:58.

Detection was the easy half

Kore.ai’s Agent Productivity Index 2026 surveyed 408 IT and engineering leaders, all of them running agents in production, through Propeller Insights in May. Half detect a malfunction within one to four hours. A third need four to eight. Fifteen per cent rely primarily on end users reporting the problem, which means the customer finds the failure before the organisation does. [Kore.ai, June 2026]

Then the number that matters: 70% could not identify which agent in a multi-agent environment caused a failure they had detected. Seven in ten organisations had the alert and not the answer.

The cost of that gap is not abstract. In the same survey, 79% of consequential autonomous actions required manual reversal, and 93% of those reversals were described as costly and disruptive. Worse, 91% said safeguards were in place before the action that later had to be undone. Forty per cent watched a single agent failure cascade across multiple systems. The controls existed; the record that would have let anyone stop the right thing did not.

Classification compounds it. Fifty-four per cent of respondents file agent failures as IT incidents only. A quarter treat them as operational risk events. Fourteen per cent have a dedicated AI operations or governance team to intervene when an agent misbehaves. [Kore.ai] So the reversal cost lands on an engineering cost centre, the process owner never sees it, and the risk register records nothing. A control that cannot name the thing it stopped is a control nobody can improve.

The failure usually sits in the middle of the chain

Two research efforts from this year explain why naming the agent is harder than it looks.

TraJudger, accepted at USENIX ATC ‘26 and deployed for over a month at a large online retailer, examined production agent failures and found that 45.5% involved error propagation chains spanning three or more steps. The intermediate steps were not faulty. They were what the authors call conduit steps: correct in themselves, faithfully forwarding an upstream error in natural language until it surfaced as a customer-visible symptom. The paper is blunt about the prevailing diagnostic approach — LLM-as-judge methods treat trajectories holistically, misattribute cascaded failures, and fabricate plausible but ungrounded explanations. Its own approach, a dependency graph plus typed specialist detection, bought at least a 15% relative improvement at agent level and 13.3% at step level, and cut average time-to-attribution from 30 minutes to under a minute. [Zhao et al., USENIX ATC ‘26, preprint]

TraceElephant, from ACL 2026, attacks the same problem from the evidence side. Its authors annotated 220 failure traces drawn from 380 executions across three agent systems and showed that access to complete execution traces, rather than outputs alone, improves attribution accuracy by up to 76.5%. The honest half of that result is the ceiling: even with full traces, automated methods averaged 65.9% accuracy at identifying the responsible agent and 30.3% at identifying the decisive step. [Chen et al., ACL 2026]

Read the two together and the investment case changes shape. Better evidence buys a great deal. Automated attribution is a helper, not an oracle. Anyone selling agent root-cause analysis as a button is selling roughly a one-in-three hit rate on the question that determines how much you reverse.

The action record is the unit of evidence

Agent telemetry today is mostly model telemetry: latency, tokens, cost, finish reason, error type. Useful for capacity planning and useless for attribution, because it describes the call rather than the action. The record that settles an incident has to answer who acted, on what authority, against what, with what effect.

Eight fields carry most of that weight:

The OpenTelemetry conventions model the right span shapes for this: create_agent, invoke_agent, invoke_workflow, plan, execute_tool, with gen_ai.tool.call.id and gen_ai.conversation.id tying calls together. [OpenTelemetry, semantic-conventions-genai] Two caveats before you build on them. Every GenAI convention is still marked Development, and the work moved to a dedicated repository in June 2026, so pin versions, keep attribute names in one place, and expect renames. Prompts and completions also belong in a separate event rather than span attributes, because they carry personal data — which is why the attribution record and the privacy posture have to be designed together.

Attribution needs topology, not tribal knowledge

The question “which agent broke production” is only answerable if you hold the graph: which agents call which, which tools each can reach, which systems each writes to, which data each reads. Most organisations hold that in three heads and a wiki page that was accurate in March.

Build it from the release manifests you already need for promotion, and treat it as an operational asset with an owner and a freshness check. At incident time it answers the three questions that decide the response: which agent first made the wrong decision, which step made it irreversible, and which downstream consumers have already ingested the bad output. Traces answer the first. The action record answers the second. Only the graph answers the third, and the third is where the reversal scope comes from.

Keep the graph close to trace replay. Both research efforts depend on re-executing a trajectory with the exact configuration that produced it. An agent whose release is not versioned cannot be attributed, only blamed.

Name an owner and a clock

Attribution is an ownership design problem wearing a telemetry costume. Three roles, separated deliberately:

The platform team owns emission and retention — the trace pipeline, the action record schema, the graph, the replay environment. It does not own the outcome.

The process owner — whose name sits on the service level, the refund policy, the claims decision — owns the outcome and the decision to reverse. They should be the person who reads the attribution report first, not the last.

The on-call rotation owns the clock, with a runbook that starts from the trace identifier. Most agent incidents I have seen rehearsed start with “check the agent logs”, which is not a procedure.

Then measure the thing you have been avoiding: mean time to attribution, separate from detection and recovery. A tier-one workflow should name the agent, the step and the authority inside fifteen minutes. If attribution takes longer than recovery, you are running an incident process that guesses and then apologises.

Retention has a statutory floor heading your way. Article 12 of the AI Act requires high-risk systems to allow automatic logging of events over their lifetime, and Article 26(6) requires deployers to keep those logs for at least six months. Both become applicable on 2 December 2027 for Annex III systems and 2 August 2028 for Annex I, following the Omnibus deferral. [EU AI Act, Articles 12 and 26] That runway is the point: the six-month floor is longer than the thirty- and ninety-day defaults most estates run, and considerably longer than the lag between a mispriced transaction and its discovery.

The test

Take the most recent incident an agent caused. Ask for the trace identifier and start a clock. Three questions: which agent made the first wrong decision; which step turned it into something you had to reverse; and under whose authority did that step run.

If the answers arrive in minutes, you have attribution. If the trace identifier does not exist, or the answer is a dashboard screenshot and a theory about the model, you have detection and a story. At 03:58 that distinction is the whole incident.

Sources

Next Post
Raising the Autonomy Ceiling: When Can an Agent Ship Without a Human?