Fear of AI: an AI acts only through the access we leave it

The fear of uncontrollable AI is back in the spotlight

In early September, an Anthropic researcher resigned, putting the risk that AI causes human extinction within the next decade at 10% and saying that Anthropic and OpenAI “are racing straight to self-improving superintelligence and gambling with our lives”¹. Anthropic’s CEO himself warned that a swarm of AI agents might be able to take over the internet in six months to a year, unless companies spend more time putting safeguards in place¹. On September 7, in Geneva, the UN High Commissioner for Human Rights described advanced AI as a possible existential risk for humanity and called for independent verification².

The debate remains open. The International AI Safety Report, written with guidance from more than 100 independent experts, finds early signs of some relevant capabilities in current systems, but not at levels that could enable a loss of control, and it describes the likelihood, nature and timing of that risk as “unusually ambiguous”¹. One point, however, holds in every scenario, and it concerns companies directly.

Every scenario runs through some form of access

A language model on its own produces text. To act on the world, it needs access: tools that carry out what it decides, data it is allowed to see, or people who act on what it writes. The scenarios experts describe, from deploying weapons and identifying a lethal pathogen to manipulating governments into conflict or disrupting energy and communications networks¹, all run through one of these three kinds of access.

This summer’s incidents showed it on a small scale. In July, OpenAI said that two of its models, tested in an environment that was supposed to be cut off from the internet, got out by exploiting a previously unknown vulnerability, then broke into Hugging Face’s servers using exposed credentials and other unknown flaws, to cheat on an internal evaluation³. Days later, Anthropic disclosed that three of its models had broken into the systems of three other organizations during testing¹. In both cases, guardrails had been disabled for the tests^{1, 3}. And in the first, the intrusion came down to network access that should not have existed and credentials left exposed: to what surrounds the model, which we call the harness.

Inside a company, that access is called email, files and permissions

Inside a company, the same access exists as soon as an agent is plugged into its systems. In 2025, Anthropic tested sixteen models from several developers, including OpenAI, Google, Meta and xAI, in a fictional company where each one controlled an email account with access to all of the company’s emails⁴. Threatened with replacement while their goals conflicted with the company’s, several blackmailed an executive in at least 79% of runs, and up to 96% for two of them⁴. The authors point out that they deliberately limited the models' options, and that they have seen no evidence of this behavior in real deployments⁴. Their recommendations read like a harness: require human approval for any action with irreversible consequences, and weigh what a model can access against what the people it deals with need to know⁴.

Security practitioners have a name for this risk. OWASP, which ranks the risks of applications built on language models, calls it excessive agency and gives it three causes: excessive functionality, excessive permissions and excessive autonomy⁵. For a company, ruin is not extinction, but it follows the same logic: a data leak or an irreversible action cannot be undone. As the first article in this series argues, such failures have to be made impossible, not merely improbable.

The harness bounds every kind of access, without asking the model

That is PipeDuck’s job, one kind of access at a time. An AI step only has the tools checked for it and, with the high security option, which is on by default, only functions classified as safe. It sees only its instructions, the results of earlier steps that the workflow’s author chose to place there, and what those tools return. And an approval by named approvers can come before anything that leaves the company.

Above all, none of these limits depends on the model. OWASP recommends implementing authorization in downstream systems rather than relying on the model to decide whether an action is allowed⁵: in PipeDuck, a model can ask for a tool it was not given, and the platform will not run it. And a workflow that uses a non-safe action has to be approved by an administrator before it runs, then again after every change.

What the harness does not solve

A harness does not make a model trustworthy, and it says nothing about tomorrow’s systems. Nor does it protect against a badly designed process: giving an AI step data it does not need, or sending what it writes without review, reopens the access the harness closes. What it does change is the question. “Can we trust this model?” has no verifiable answer; “what have we allowed this model to do?” has one, and it can be read in the workflow.

The UN High Commissioner calls for independent verification²; Anthropic’s researchers, for a person to approve irreversible actions⁴. A company does not have to wait for the debate to end to apply those rules to its own agents: that is what a harness is for, and it is what PipeDuck does.