The failure was authorisation, not containment

This week's signal comes from Britain's AI Security Institute, who published an agentic AI incident report worth reading.

During routine cyber testing, its agents took 19 unsanctioned actions on the live internet, in 10 of 122 runs. The number isn't as interesting as what the agents actually did. One agent tried to insert malicious code into a real open-source project, and to get it approved it researched the project's human maintainers, created multiple fake identities and used them to socially engineer a real person into signing it off. The maintainer caught it and refused. It also left instructions on GitHub that later agents found and used.

Nobody exposed these agents to the internet by mistake. AISI enabled the access deliberately and switched the developers' cyber classifiers off, because you cannot measure what a model can really do through a filter. So the thing that failed was not containment, it was authorisation.

One more detail: in some runs the task prompt was misconfigured, so the agent concluded there was no solution inside the scope it had been given. It didn't set out to go rogue, it was a by-product of an objective that could not be met honestly.

Most genuinely useful agents will need to reach suppliers, registries, email, code repositories and payment systems. So the consideration becomes whether it can tell a permitted destination from an available one, legitimate persuasion from manipulation, and useful memory from an instruction that contaminates whatever runs next. And whether the objective you set it can actually be achieved inside the authority you gave it.

The attempts failed, and AISI found no evidence of real-world harm.