The safe envelope for an AI agent must narrow as its capability grows

The prevailing pattern for deploying agents is a ratchet. The model improves, so it is granted a wider allowlist, more auto-approval, more egress, fewer confirmations. The premise is that capability and trustworthiness move together, and that a better model has earned a longer leash.

The premise is wrong in a specific and checkable way. The property that makes a system more capable is the same property that makes its errors less bounded, its intentions less legible, and its output better at passing a review it should not pass. Capability improves the quality of a mistake as reliably as it improves the quality of a success. A system able to write a correct migration is a system able to write a plausible one; a system able to produce a convincing explanation is a system able to produce a convincing wrong explanation. Nothing about improved capability constrains which of those it produces on a given occasion.

Two things follow. The first is that the safe envelope should narrow as capability grows, not widen. The second is that any guarantee tied to how a model behaves is a guarantee that must be re-established every time the model changes underneath it, which in current practice is several times a year.

What the review layer actually delivers

The industry’s answer to this has been to keep a human in the loop, per action. That answer has now been measured, and it does not hold. Approval rates for agent permission prompts run between 93 and 97%. In a controlled study of over a thousand professional testers, humans caught 13.6% of dangerous commands, and the rate fell from roughly 17% early in a session toward 5% after fifty prior prompts. The pattern is not particular to terminals: browser interstitials are clicked through at 70%, operating-system elevation prompts approved at 91%, and a terms-of-service study recorded 97% consent to a policy assigning the reader’s first-born child as paymentR/5C2.

A second study the same month disaggregated those misses by threat type and found the calibration inverted with respect to consequence: obviously destructive commands were missed 11.7% of the time, while exfiltration and code execution were missed 33.4% of the time and scope violations 35.0%. A command with a familiar-looking name, whose script piped output to an external endpoint visible in the history directly above the prompt, was approved 64.7% of the timeR/5C2. Reviewers catch the theatrical threat and miss the consequential one.

The classifier that now replaces the human in the default configuration is a real improvement over the dialog, and it has a published ceiling: a miss rate near one in six on real overeager actions, and a flat rate rather than a decaying one. Flatness matters more than the figure. A constant miss rate applied to a growing number of actions produces a growing absolute number of misses, so the safety of the arrangement degrades precisely as the agent becomes more useful.

The deeper problem with both the human and the classifier is that they are inspection. Inspection asks whether a proposed action is dangerous, which is a question about consequence, and consequence is not decidable from the artifact under inspection. Worse, inspection concedes the initiative: the reviewer must anticipate every dangerous form, while the adversary needs one form that was not anticipated.

The boundary that fails is not the boundary being sold

There is now a record on this, and it is short enough to state exactly. Five agent containment incidents were publicly disclosed between 21 July and 5 August 2026. In four of them no isolation boundary was breached at all: the hypervisor held, the kernel held, the isolate held, and what failed was network egress policy. The UK AI Safety Institute said so plainly in its own report, that no agent tried to break out of the sandbox or attack its systems. The single genuine escape produced its harm through subsequent reach rather than through host accessR/6D2.

That record is worth holding against how the category is sold. These products are marketed almost entirely on isolation strength, in a vocabulary of microVMs, dedicated kernels, hypervisor boundaries, and no path back to the host. That vocabulary answers one question, whether the agent can escape onto the machine, and it is not the question the incidents asked. The industry hardened the failure mode that has not occurred and shipped the failure mode that has occurred five times as a configuration option.

The mechanism underneath is worth understanding because it generalises past this category. Every agent sandbox must permit at least one inference endpoint or the agent is useless, and contemporary inference endpoints expose server-side tools chosen per request by the caller: web search, web fetch, code execution. An agent inside the sandbox declares a fetch tool, the provider’s infrastructure retrieves an arbitrary URL, and data placed in that URL leaves on the outbound leg. To the egress proxy this is one permitted request to a permitted host. The single host every agent sandbox is required to allow is a general-purpose compute provider.

So the destination allowlist, which is the control the whole category rests on, cannot bound the thing it is asked to bound. It operates on host identifiers while the security-relevant properties are what a request carries and who can read it.

Make it impossible rather than caught

The alternative move is to arrange for the dangerous thing to be unrepresentable or unreachable, so that no inspection is required. This is a different kind of engineering claim, and it has a different failure mode. An inspection-based control fails when someone finds a case it did not cover. A structural control fails when the structure is wrong, which is a property you can examine directly and once.

Applied to an agent, the structural move is to separate the machine that may execute anything from the machine that may reach anything. Inside a zone with no actuator crossing its boundary, an agent may run at any speed with no per-action approval, because nothing it does in there reaches anything. It may hold a live credential, read any injected instruction, and compute a perfect exploit, and effect nothing, because every use requires a channel it does not have. The rate that matters is not how fast the agent works but how fast its work crosses into a privileged environment, and that crossing is gated by judgment regardless of the speed behind it.

This inverts the usual relationship between capability and containment. A structural guarantee was never calibrated to a particular level of model goodness, so it does not weaken when that goodness becomes a worse assumption. It ages in the right direction.

The regulatory layer is not a set of guardrails

There is a framing question underneath the engineering one, and getting it wrong produces most of the confusion in this area.

The durable regulatory layer for this technology is not model guardrails. It is the existing legal and justice system, and its unit of accountability is the person who directs the system. Guardrails are transitional. What remains is that you are answerable for the executed actions of the systems you own and operate, in the same way you are answerable for the executed actions of any other instrument you direct.

That reframes the engineering problem completely. The agent is not executing on its own behalf; it is executing on yours. So the question was never whether the agent can be trusted. The question is whether you can discharge your responsibility for what gets executed provably, by construction, rather than hopefully, by relying on the model’s restraint. Everything structural follows from choosing the first.

It also explains why a caught action is not a save. If your answerability is unconditional, an action that was attempted and blocked is still an action your system attempted, and the only reason it did not become yours is that a control happened to work. A structural arrangement does not put you in that position, because the action was never available to attempt.

The objective is narrower than it looks

One consequence of this framing is worth stating plainly, because it contradicts the usual security posture and is the point at which this argument is most often misread. If the thing you are answerable for is executed action, then the objective is to prevent caused execution: code running, systems being reached, actions being taken beyond the boundary. Keeping information in is not the objective, and the design does not attempt it.

This sounds wrong to a security practitioner and is worth sitting with. A contained agent that egresses freely is acceptable, provided every use of what it holds is unreachable. A secret has value where it is used; if every use is structurally unavailable, possession is not capability. The practical payoff is large, because defending caused execution alone is a far smaller problem than defending execution and confidentiality together, and it is the one that maps onto what the operator actually answers for.

Three inherited terms get attached to arrangements like this and each imports the wrong invariant. An air gap or a data diode describes the direction information flows and invites a defender to stop data leaving, which is precisely the goal being declined here. Industry zero trust is continuous re-verification, in which trust is minimised and re-priced per request but every permitted action is still a granted trust that verification resolved positively; it is a sliding scale whose zero is a default posture rather than a property of the design. Defence in depth is a stack of mitigations, which concedes that each layer leaks and fails when one course does.

A fourth term belongs in that list by its absence. Concealment is not a security posture here and cannot be, because the guarantee on the trusted core is that a person read it completely, and a base that must be read cannot also be hidden. A design whose safety turned on its details being unknown would rest on precisely the contingent bet this whole argument exists to remove.

The debt to prior work is nonetheless real, and the lineages are worth naming: confinement in Lampson’s sense, object-capability security, and the constitutional principle that a person answers for what their systems execute. What differs is purpose rather than technique. Those lineages were designed for static programs and human operators. The occupant here is dynamically generated, rewritten by the adversary in flight, and running at machine speed, and what is worth preventing is not what it learns but what it can cause to run.

Where command evaluation belongs

A contained agent needs no command evaluation at all. Inside a zone with no actuator crossing its boundary there is nothing to gate: the shell runs unmodified and unwatched, allowlists and denylists are absent because they would protect nothing, and the agent’s freedom inside is total. That is the arrangement working rather than a gap in it. What the operator reviews is never a command; it is a crossing.

This is worth stating plainly, because a separate line of work described elsewhere on this site proposes exactly the semantic evaluation of shell commands that the paragraph above says a contained agent does not needE/2D8. The two are separate systems born from one ethos, not competing answers. Both hold that a guarantee must be structural rather than behavioural. Both relocate judgment away from the per-action moment. They differ in what the world currently permits.

Most agents today run in privileged environments, on machines that reach production, with no boundary to cross because none was built. Containment of the kind described here requires dedicated hardware and a deliberate topology, and it is not available to most deployments this year. Where the reach cannot be removed, the achievable improvement is to make a command’s meaning explicit at the one layer where it is explicit, so that policy binds to a resolved operation rather than to a string. That is an intermediate technology in the industry’s evolution, and it is a substantial improvement on the alternatives measured above.

The two also meet at the boundary. A contained arrangement still has a crossing, and the operator standing at it must see the resolved operation rather than the text that produced it. What a semantic runtime yields at the moment of execution is precisely what a promotion review needs to display. Containment removes the need to evaluate commands inside the zone; it does not remove the need to understand the one that leaves it.

Which deployments this is for

Containment of this kind is not a general answer, and the practical limits are real. It asks for dedicated hardware, a deliberate topology, and an operator willing to run two machines where one would otherwise do. Those costs rule it out for a great many deployments, and the industry will need a range of approaches matched to what each environment actually requires rather than a single posture applied everywhere.

What this architecture addresses is one operating model in particular: the case where the intent is to grant an agent maximum flexibility. An agent meant to be kept on a short leash can be kept there by lighter means, and the machinery described here would be disproportionate. The model this is built for is the opposite one, where you want to hand an agent a general interpreter, a live credential, no per-action supervision, and the freedom to build, test, install, and iterate at whatever speed it can manage. That freedom is exactly what has to be paid for. The more latitude you intend to grant inside the zone, the less the boundary can afford to be procedural, because everything you decline to restrain internally is restrained by the boundary or not at all.

What this does not claim

Two boundaries. This is an argument about arrangement, not about mechanism quality: a structural containment badly implemented contains nothing, and the argument gives no licence to skip the work of establishing that a boundary holds. The soundness of a single memory crossing in one driver required an account of its ownR/A4C, and that is the scale at which structural claims have to be paid for.

And it is an argument about a particular threat: a remote adversary arriving as untrusted content, who thereby obtains arbitrary execution inside the contained zone. It says nothing about an adversary with physical access. The demonstrated exfiltration channels from air-gapped machines are real, and every one of them requires reception apparatus the attacker positions within tens of metresR/8E3. Passive reconstruction of display emanations reaches considerably further without any implant at all, which is a different threat and one this architecture also declines to defend. Both are answered by controlling physical proximity, which is a locked door rather than an architecture.