Who Pays When the Agent Is Wrong?

Published: at 05:20 AMSuggest Changes

A finance agent approved a supplier invoice last quarter and moved the money. The invoice was fraudulent: the supplier’s bank details had been changed weeks earlier by a convincing email the agent treated as authoritative, and by the time a controller noticed, the funds had cleared. The chief financial officer asked who pays, and got four answers. The platform team pointed at the model provider, whose contract capped liability at twelve months of fees. The integrator pointed at its instructions. Legal pointed at the deployment decision. The insurer had never been told the agent could move money at all.

This is not a technology problem. Enterprises have spent two years deploying agents that act — calling tools, updating records, approving transactions — while leaving the allocation of loss, the response of their insurance and the evidence needed to defend a claim undecided. The decisions are being made regardless, in procurement paperwork, policy wordings and the logs nobody configured. They are just not deliberate.

The position is blunt. By default, liability for an agent’s wrong action lands on the organisation that deployed it. Insurance is narrowing rather than widening around that exposure. The asset that changes the outcome is a decision record showing what the agent did, who authorised its authority and what human oversight was operating. Contracts, controls and cover need fixing before the first material incident, because afterwards the enterprise negotiates from weakness.

Accountability lands on the deployer

An autonomous action travels through four parties: the enterprise that deployed the agent, the model provider whose inference layer it calls, the platform vendor that supplied the framework, and the integrator that assembled the workflow. Every incident review instinctively pushes the loss up that chain. The contracts make that difficult.

The Cloud Security Alliance’s June 2026 research note on enterprise accountability is direct about where the loss settles: deploying an agent does not transfer liability to the agent, it concentrates accountability on the deploying organisation. Under the oversight standard now emerging, the enterprise is liable unless it can demonstrate that documented and operable monitoring, auditing and safety systems were in place when the agent acted. The vendor side compounds this: model providers cap damages at a year of fees and exclude output accuracy, application vendors inherit those exclusions through flow-down clauses, and enterprises find their contracts provide no material indemnity for the harm the agent caused. [Cloud Security Alliance, June 2026]

Regulators are closing the space for deflection. California’s AB 316, effective from 1 January 2026, stops a developer, modifier or user of an AI system from arguing that the system acted autonomously as a defence against liability. “The agent did it” is no longer a legal answer in the largest US technology market. [State of California, October 2025]

The European Union has gone further on the supply chain. The revised Product Liability Directive, Directive (EU) 2024/2853, brings software — including AI systems — inside the definition of a product, so a defective agent can trigger strict liability without proof of fault, with transposition due by 9 December 2026. Its value-chain rule should concern every integrator: the AI component provider and the manufacturer of any larger system that integrates it can be jointly and severally liable, so an enterprise assembling a workflow from third-party models and tools may be unable to deflect a claim upstream. [European Union, November 2024]

The enterprise holds the decision rights, the customer relationship and the deepest pockets. That is where the loss concentrates.

Your policies were not written for this

The second misconception is that an existing programme already covers the exposure. For most enterprises it covers it ambiguously, which is worse than not covering it, because the ambiguity is resolved after the loss by whoever drafts the better argument.

The RAND Corporation’s September 2026 report, The Insurability of Artificial Intelligence, maps the market three ways: a minority of carriers writing affirmative AI cover, a growing number filing broad exclusions, and a majority staying silent. Silence is the problem. When a policy says nothing about AI, coverage turns on how the claim is characterised at the moment of loss, and two enterprises running identical agents can reach opposite outcomes depending on the carrier. RAND found AI-related losses touching at least eleven insurance lines, and single events that resemble a cyber incident, a professional error and a product defect at once. [RAND Corporation, September 2026]

The lines matter. Professional indemnity responds to a negligent act in a professional service; cyber to a security failure such as a breach; general liability to bodily injury and property damage. An agent that was not hacked, touched no personal data and hurt nobody physically but cost a counterparty money by being confidently wrong sits in the gap between all three, which is where the market is tightening.

The Lloyd’s Market Association’s survey of its members, published in January 2026, asked underwriters to rate the viability and magnitude of AI loss scenarios across lines. Professional indemnity scored the highest magnitude and overall impact; cyber the highest viability. Both were rated only “plausible” or “moderate” rather than likely — an exposure insurers see as real but not yet well enough understood to price with confidence. [Lloyd’s Market Association, January 2026]

Where the market cannot price a risk, it excludes it. The Insurance Services Office’s generative-AI endorsements for commercial general liability, form CG 40 47 among them, took effect in January 2026 and carve AI-related injury and damage out of the standard policy; several large US carriers have moved to adopt them or file their own. The critical variable in professional indemnity is human review. Where a professional used an AI tool, checked the output and approved it, the insured act is the professional’s judgment and cover may hold. Where the agent acted autonomously, with no human in the causal chain, the claim lands where the new exclusions are written.

Standalone AI liability policies now exist at Lloyd’s. For most enterprises the binding question is whether the loss scenario they fear is named as a covered peril in the policy they already hold — an answer that comes from the endorsement schedule, not the marketing summary.

The evidence trail decides who pays

The evidence trail decides a liability argument. The revised Product Liability Directive gives it teeth. A claimant who shows a plausible harm can have a court order disclosure of the technical documentation, and a manufacturer that fails to produce it, or produces it incomplete, triggers a presumption that the product was defective. Where the system is complex enough that causation is excessively difficult to prove, the causal link is presumed and the burden shifts to the manufacturer to disprove it. For an EU operator, the documentation file determines whether it carries the presumption or rebuts it. [European Union, November 2024]

The oversight standard runs the same way. Under the Cloud Security Alliance’s formulation, the enterprise is liable unless it can show documented and operable monitoring, auditing and safety systems were running when the agent acted. [Cloud Security Alliance, June 2026]

What that means for the log is specific. Technical events — an API call, a tool invocation, a model response — are necessary but not sufficient. The record that decides a claim captures business intent: which authority the agent operated under, what policy permitted the action, what data it relied on, whether a human approved it or should have, and who is accountable for the outcome. Approval records need the same discipline, and retention must be set against the limitation periods in the jurisdictions where the enterprise operates, not against storage cost.

A logging capability that cannot answer “why did the agent do this, and who allowed it to” is not an audit trail. It is telemetry.

Fix the contracts before you need them

Contracts are where liability is actually allocated, and the default allocation is unfavourable. Four points are worth auditing.

Indemnity scope. A vendor indemnity drafted for intellectual-property infringement does not reach harm caused by an autonomous action. The clause needs to name the agent’s actions and failure modes, and to survive the vendor’s own disclaimers of output accuracy.

Liability caps. A cap set at twelve months of fees, or a monthly subscription, bears no relationship to a loss an agent can cause by moving money, mispricing a product or denying a customer. Negotiate it against a plausible harm figure, and know that figure before the negotiation.

Audit rights and change control. An agent’s behaviour changes when its model is updated, its tools are re-scoped or its prompts revised. The contract should give the enterprise audit rights over post-deployment changes, notice of material updates, and enforceable warranties on documented behaviour.

Regulatory and jurisdiction cover. The vendor should carry the compliance obligations that attach where the agent operates, and the enterprise should confirm that a vendor’s decision to exclude AI risk from its own insurance does not leave the loss with the customer by default.

The same discipline applies outbound. A contract that promises a customer an accuracy or performance standard no policy was built to guarantee creates a liability the enterprise cannot transfer. Warranties, service levels and indemnities should be aligned with the perils the policies actually name.

The test to run this quarter

Take the single worst plausible action each production agent can take — a payment it can move, a record it can change, a customer it can affect — and walk it through the chain. Name the party that would pay, find the clause that puts the loss there, and the policy line you expect to respond, by endorsement. Most enterprises will find at least one link with no owner and one policy that was never told the agent existed.

Set four questions against every agent in production. Can you produce its decision log within one business day? Can you name the human who authorised its action authority? Can you point to the contract clause that allocates a loss it causes? Can you name the policy endorsement you believe responds? An enterprise that answers all four is managing the risk. One that cannot answer the third and fourth is self-insuring by accident, and it will discover the size of that position mid-incident, with a counterparty, a regulator and an insurer all asking the question the chief financial officer asked first.

Sources

Previous Post
Shadow AI Is Not Shadow IT
Next Post
Prompt, Retrieve or Fine-Tune: How to Choose Where Model Behaviour Lives