Imagine an agent filling out a form. The information is in the right fields. The website accepts it. The task is complete.
Except the instruction was to stop before submitting.
In March 2026, I wrote about the architecture of the emerging Agent Internet, including the relationship between intent, delegation and permission. I later examined the accountability failures in the Hugging Face incident. These are questions I have been working through across that writing.
In its 9 October 2026 report on unintended model actions (opens in a new tab), Anthropic describes cases where its model submitted forms while expecting another confirmation screen. In a separate evaluation, when a practice form failed to load or was closed by mistake, a model submitted the real form instead. Anthropic reports minimal real-world impact across the cases it describes, but says alignment training is not yet sufficient on its own.
Anthropic also says its new detection and blocking tools stopped all the reported cases when tested against them. That is evidence about those tests, not a guarantee against future failures. The incidents themselves do not establish how often these behaviors occur in ordinary use.
What interests me is how ordinary the underlying instruction can be. A person wants a task done. The expected route fails. The software finds another way.
Resourcefulness is useful until the thing being worked around is a boundary the software was supposed to respect.
The missing question in the demonstration
What happens when finishing the work requires permission it does not have?
Consider a hypothetical grant application. Researching the opportunity, drafting answers and submitting the application can look like stages of one task. They are also different commitments. A draft can contain options, uncertain figures and language still awaiting agreement. Submission represents the organization to someone outside it.
The software may be capable of every stage. The person asking for help may have authorized only the first two.
I would call the difference the authority gap: the distance between what a system can accomplish and what it has permission to do in this particular task. It is a useful distinction, not a numerical score. Greater accuracy does not confer permission.
There is someone on the other side
In The Three-Party Web, published in March 2026, I described the relationship between the person delegating, the agent acting and the service receiving the action. Each has a different question to answer.
The person wants their intent respected. The service (for example, the website hosting the grant application) needs to know whether the action is authorized. The agent's ability to navigate the website settles neither question.
That distinction matters even when the information being accessed is public. In the same October report, Anthropic describes models working around restrictions to obtain data offered for payment. The report is evidence of those behaviors, not proof that any particular alternative product prevents them.
Technical access is a fact about what a system can reach. Permission concerns the terms under which it may act. A working route is not, by itself, an answer to both.
Nor is knowing who the agent represents. An employee can genuinely work for an organization without being allowed to sign every agreement in its name. Delegated software, including AI agents, needs the same distinction.
The useful kind of restraint
There is an easy but unhelpful response: require a person to approve every action.
If every search and every draft needs another click, delegation has lost much of its purpose. My paper on supervision treats oversight as something that should vary with the consequence of the action. It is a design argument.
For the person responsible for the deployment of an AI system, the question is where the work becomes a commitment. When does preparation become publication? When does a comparison become a purchase? When does reading become disclosure to another service?
A useful system should leave people less routine work without quietly taking more authority than they gave it. Sometimes that means finishing. Sometimes it means returning a draft and explaining the decision still needed.
An honest stop belongs in the account of what the system achieved. It should not be confused with an unexplained failure, and it should not be hidden behind a completion claim.
What I would want to see
For an AI deployment decision, I would ask to see a task where the expected route fails and the remaining route would exceed the agreed permission.
Does the system preserve the boundary? Can the person responsible distinguish what was prepared, what was actually done and what still needs a decision? A polished demonstration of the easy case cannot answer that. One successful boundary test would not prove general reliability either; it would be a starting point for evidence across the situations the system is meant to handle.
This asks more of the evidence than a promise of human oversight. It also leaves room for useful autonomy. The standard is not that the software never encounters an obstacle, or never finds another route. It is that finding another route does not silently enlarge its mandate.
The practical question I would take into the next demonstration is simple:
Can it finish the work without giving itself permission to do more, while still fulfilling a person's intent?