Raising the Autonomy Ceiling: When Can an Agent Ship Without a Human?

Published: at 03:05 AMSuggest Changes

The change board cleared the payroll agent for unattended execution on a Tuesday afternoon. The evidence pack ran to nine pages: a 96% pass rate on the monthly reconciliation suite, a signed security review, a rollback plan. Nobody asked which actions the 96% covered. The agent’s read path — pulling employee records, checking contribution rates, validating expense claims — had been exercised thousands of times. The write path, the one that releases a consolidated bank transfer, had been exercised eleven times, in a sandbox, by an engineer who knew the correct answer.

The board believed it had approved an agent. It had approved one action class and inherited the risk of another.

That distinction settles more autonomy arguments than any framework. Autonomy is a property of an action, not an agent: this action, against these records, up to this value, reversible this way. Teams that hold that line raise their ceiling faster than teams that spend a quarter debating whether an agent is “ready”.

The ceiling is rising whether you raise it or not

Capability improves on a schedule nobody in the enterprise controls. METR measured the length of tasks frontier agents complete at 50% reliability and found it doubling roughly every seven months since 2019. The same work shows the 80% reliability horizon growing at a similar rate but sitting substantially lower: a task an agent manages half the time is several times longer than one it manages four times in five. [METR, March 2025]

Vendor demonstrations live at the 50% horizon. Production lives at the 80% one, and the distance between them is where most autonomy decisions go wrong.

The decision is being taken regardless. VentureBeat’s Pulse research found that half of the enterprises surveyed had shipped an agent that passed internal evaluation and then caused a customer-facing failure, with a quarter seeing it happen more than once. Only 5% fully trust automated evaluation. Two-thirds already permit, or are engineering toward, deploying low-risk agent changes with no human in the loop. [VentureBeat Pulse Research, June 2026]

Organisations do not believe their evaluation evidence, and they are removing the human anyway. Singapore’s agentic AI framework concedes part of the reason: continuous human oversight over every agent workflow becomes impractical at scale. [IMDA, May 2026] That is true, and it is not a standard.

Two numbers set the ceiling, and neither is model quality

Consequence and reversibility. Those are the dials, and they move independently of how clever the model is.

IMDA’s framework lists the factors that raise risk: the scope of actions available to the agent, whether those actions can be undone, the agent’s level of autonomy, task complexity, and system complexity — how many agents interact and how much behaviour emerges rather than being designed. [IMDA] Reversibility carries the most weight, because it decides whether a mistake becomes an incident or a correction. An internal reminder is reversible. Emailing a customer is not.

Reliability compounds in a way that surprises people. A twenty-step workflow in which each step succeeds 99% of the time completes end to end about 82% of the time. At 98% per step, it finishes roughly two runs in three. Forty steps at 99% lands near 67%. Autonomy lengthens the workflow and shortens the reliability, which is why the ceiling belongs on the shortest action class that delivers the value.

There is a cost argument underneath. DORA’s ROI work names the verification tax: the effort of checking whether generated output is reliable, secure and architecturally sound. It sits inside the J-curve the report describes, the dip before returns arrive, and in DORA’s sample model for a 500-person engineering organisation the instability that accompanies faster output appears as a change failure rate rising from 5% to 6%. [DORA, April 2026] Review effort does not disappear when the reviewer does. It moves upstream into evaluation and monitoring, or it becomes incident cost. A team that raises the ceiling without reducing the verification work has relocated the cost and called it a saving.

The promotion file

Autonomy should be promoted the way access is granted: per action class, on evidence, with a signature and a review date. Most organisations have the first part of that and none of the rest.

Tencent’s CodeBuddy coding agent shows what per-action-class defaults look like in production. Reading files needs no approval. Editing a file needs approval valid only for that session. Running a shell command needs approval that persists per project. Network requests and external tool calls through MCP are governed by allow, ask and deny rules. [IMDA, Tencent case study]

Dayos, an enterprise automation vendor, took the same logic into IT service management. It scored every ticket type against three questions: if the agent gets this wrong, how bad is it; can the action be undone; and is human review at each step realistic? The scores produced three tiers. Sixty per cent of tickets — password resets, access requests, status enquiries — are fully automated, with a designated reviewer auditing a cross-section of the agent’s reasoning chains every fortnight. Thirty per cent, including chart-of-accounts updates and failed API diagnosis, are agent-diagnosed and human-approved. Ten per cent, covering production deployments and permission changes, the agent does not touch. [IMDA, Dayos case study]

Three questions, three tiers, one audit cadence. What the file must carry is straightforward:

That last item is where European practice is heading, deferral notwithstanding. Article 14 of the AI Act requires oversight measures “commensurate with the risks, level of autonomy and context of use”, and Article 26(2) requires deployers to assign oversight to people with the necessary competence, training, authority and support. [EU AI Act, Articles 14 and 26] The Digital Omnibus moved Annex III high-risk duties to 2 December 2027 — precisely the runway available to build this file. Authority and support are the two requirements organisations routinely fail: authority means the reviewer can overturn the decision, support means time and staffing rather than a job title.

IMDA’s guidance deserves to be a configuration requirement rather than a principle: deny action by default when the approval infrastructure fails. If the supervisor is unreachable, or the agent attempts an action with no established approval policy, the answer is no.

Audit the reviewer, not just the agent

Here is the failure mode that quietly resets the ceiling to zero. Automation bias is named in the AI Act as something oversight must address: the tendency to over-rely on output, particularly where the system has performed well before. [EU AI Act, Article 14(4)(b)] Training does not remove it, because it is a property of humans sitting in front of a system that is usually right.

IMDA sets out three measurable checks on whether oversight is real: the human override rate, where a low rate may signal rubber-stamping; response times during review, where shorter times may signal automation bias or review fatigue; and outlier analysis to identify reviewers whose decision patterns deviate from the norm. [IMDA]

I have sat in front of a review queue where almost every proposal was approved in seconds. The control existed on the organisation chart, in the workflow, and in the audit trail. It did not exist in the decision.

Two consequences follow. Track the reviewer’s numbers with the same seriousness as the agent’s. And be honest about expertise: IMDA notes that users “vibe coding” with an agent may lack the expertise to judge the robustness of what it produced.

Who signs, and what revokes it

The signature matters more than the framework. The platform team produces the evidence. The risk function sets the boundary and the residual-risk appetite. The process owner — whose name sits on the service level, the payroll run, the claims decision — signs the promotion and owns the outcome. A vendor can supply evaluation tooling. It cannot accept your residual risk.

Then treat the ceiling as reversible. Gartner predicts that 40% of enterprises will demote or decommission autonomous agents by 2027 because governance gaps surfaced only after something went wrong in production. [Gartner, May 2026] IMDA frames the same point as an iterative loop: an anomaly found in monitoring sends you back to re-assess and re-bound the risk.

Write the demotion triggers down before you need them. An override rate below the stated threshold. A new tool added to the action class. A model, prompt or policy change. An exception pattern the evaluation set never covered. An incident in an adjacent action class, because agents share context and identity.

Then accept the enforcement bill. MSD, which runs agents across a risk-calibrated governance pathway, concluded that the highest levels of agency need technical measures beyond monitoring: a programmatic runtime policy enforcement layer at the AI gateway, implemented before higher autonomy is enabled. [IMDA, MSD case study] Autonomy is bought with enforcement, not with confidence.

The test

Take the action class you moved to unattended most recently. Ask its owner for four things: the action class in fields and values, the reliability evidence at the level the decision required, the name of the person who signed, and the condition that would demote it.

If the answer is a success rate with no statement of which actions it covers, the ceiling was raised by whoever last edited the configuration. The person accountable for the outcome has not been told, and the human nominally still in the loop is there to satisfy a diagram.

Sources

Next Post
The EU AI Act Deadline That Actually Applies Now