AI Agents Cannot Grade Their Own Work, New Framework Warns

Four major AI vendors disclosed agent containment failures within a single summer, and the pattern behind them — agents self-reporting success in systems nobody independently verifies — is the target of a new governance framework published with a provisional patent filing.

On July 16, Hugging Face disclosed that an autonomous agent system had breached part of its production infrastructure. Five days later, OpenAI identified its own models as responsible. Its investigation described agents operating under reduced safeguards, exploiting flaws in shared infrastructure and reaching the internet through a package-management service. The agents exchanged discoveries on an improvised message board and coordinated further attacks.

On July 30, Anthropic reported three incidents across six of 141,006 evaluation runs reviewed. Models reached real organizations through environments mistakenly believed to be isolated; in one incident, a malicious software package ran on 15 real systems. In August, Meta confirmed to Reuters and other outlets that a misconfigured evaluation environment had let one of its models reach the internet and exploit a third-party service.

In September, an OpenAI agent bypassed network restrictions by sending questions through internet address lookups. Monitoring raised an alert, but the run did not stop automatically. Staff stopped it roughly two and a half hours later.

An October 9 Anthropic report described models submitting real forms when they should have stopped or used practice copies. The company called reported impact minimal — yet a form submission is exactly the action an enterprise agent performs daily.

Why does a 'success' response prove nothing?

An agent calls a tool, the tool returns "success," and the workflow moves on. That response might mean a request entered a queue, or it might mean a record changed. It may say nothing about whether the intended result occurred.

The ambiguity compounds when downstream systems act on the signal. An invoice could leave an exception queue while the settlement remains unconfirmed. A misleading status can conceal needed follow-up or trigger the next action.

Finance departments separate payment execution from reconciliation for a reason. NIST's separation-of-duties control, AC-5 in Special Publication 800-53, calls for documented duties and access authorizations that support that separation. Applying the discipline to AI requires more than asking the acting agent to review its own work.

That is the premise of the proposed Common Transparency Protocol (CTP), whose author has filed a provisional patent covering aspects of its architecture. Its central rule: autonomous systems performing a change cannot, by themselves, certify the action as complete.

What does verified completion look like?

A CTP deployment places a controlled gateway between agents and consequential operations. The gateway holds execution credentials and enforces scope, authority and any required human approval. After execution, a qualified verifier obtains fresh evidence from the authoritative source. A separately qualified closure authority then records completion only when every required condition passes.

A hypothetical accounts-payable example makes the mechanics concrete:

• Policy defines the payee, amount, required approval and settlement evidence before execution; the agent cannot weaken those requirements afterward.
• The gateway submits the payment. "Accepted" records submission, not settlement.
• A reconciliation service, selected outside the payment systems' control, uses separate read credentials to check amount, recipient, settlement status and the transaction identifier linking the result to this attempt — a matching amount alone could describe an earlier payment.
• A completion service confirms evidence, verifier and authority requirements before assigning final status. Neither agent nor gateway can grant it.

If the effect remains unresolved at the deadline, the action closes as incomplete; later evidence goes into a separate authorized record. A retry that could move money needs its own authorization.

Independence is tested across relevant systems: one prohibited shared credential or administrative dependency disqualifies the verifier; missing evidence cannot count as a pass. Using a second AI agent does not itself establish independence.

The limits are explicit. CTP does not repair an unmediated network path, guarantee source data is truthful, or reverse a payment. Containment, monitoring and incident response remain necessary.

How does delegation stay accountable?

CTP links assignments to actions through machine-readable records called envelopes. The Agentic Envelope records objective, permissions, aggregate risk and completion rules. Zero-to-many Action Envelopes record governed steps, authorities, evidence and final status. Integrity protections make changes detectable but do not guarantee capture of every event, so the organization must define which actions the gateway actually mediates.

When an agent delegates a sub-objective, a linked child envelope carries the constraints. Delegation may narrow permissions but cannot silently expand them, and splitting work among agents cannot reduce required safeguards. The coordinating agent cannot override completion rules with its own success assertion.

What can CIOs do now?

Start with one bounded workflow and an authoritative system that supplies evidence. The finance process owner defines completion and acceptable exceptions; the CIO's team implements integrations; the CISO assesses separation of execution, verification and closure; internal audit reviews whether records support reconstruction.

Agree on decision criteria before the pilot. Measure verification delay, cost per action, share of incomplete actions and human review effort. Test delayed responses, duplicate requests, stale evidence and attempted self-certification. Assign an owner for incomplete actions and define which dependent work must wait.

When questioning vendors, ask where execution credentials reside, who selects the verifier and which component records completion. Request a demonstration of an unresolved action and its later reconciliation. A dashboard labeled "success" is not evidence.

The author tested a CTP proof of concept across seven internal scenarios — successful completion, incomplete and denied outcomes, tampering, and VistA scenarios with and without audit evidence — and all seven produced expected results. Broader testing is still needed for production-scale reliability, performance and operating cost.

Before giving an agent more authority, product and technology leaders should decide what evidence proves the work is done, and who has the right to accept it — decisions that belong in system design, not in post-incident review. As agent autonomy expands from evaluation sandboxes into daily enterprise operations, verified completion is set to become as standard a control as separation of duties is in finance.