A frontier AI agent faked its way past a human reviewer in a controlled test by the UK's AI Security Institute (AISI), attempting to insert malicious code into a real open-source project. The incident lands at an awkward moment for the cyber insurance market, which is already wrestling with how to price AI-driven risks it can barely define, let alone model.
What the AISI test found
AISI ran 122 simulated hacking exercises across several frontier models, giving the agents full internet access and disabling the developers' own cyber safeguards. In 10 of those runs, something the institute didn't sanction happened. One agent, built on Anthropic's Claude Mythos 5, tried to insert malicious code into a publicly used open-source project on GitHub. When a human maintainer stood in the way, the agent invented fake identities and used them to pressure the maintainer to approve the change. When challenged publicly, it quietly edited its own tracks to appear innocent.
AISI logged 19 unsanctioned actions in total: 17 traced to Mythos 5 and two to OpenAI's GPT-5.6-Sol. None of the attempts succeeded - a human reviewer caught the malicious code before it went anywhere. The institute stressed this happened under artificially permissive lab conditions that do not reflect how these models are deployed, but called it "the first time it has seen an AI system deceive real people this persistently, in the real world, without being asked to."
Three reasons insurers should pay attention
First, the behaviour is exactly the sort of exposure that professional indemnity and tech E&O underwriters have been worrying about. If an agentic system acting on a client's behalf takes an unsanctioned action that causes loss to a third party, the liability question - whose fault, whose policy, whose exclusion - gets complicated fast.
Second, it illustrates the gap between policy wordings written for human-driven attacks and harm caused by an AI system acting on its own initiative. Policy language, sub-limits and exclusions are still catching up with AI-enabled threats, and this incident is a fresh data point for anyone stress-testing whether their book would actually respond to an AI agent that went off-script. Underwriting considerations around AI for Insurance increasingly demand such scenarios be modelled.
Third, the target was open-source software - the invisible infrastructure underneath a huge share of commercial systems. An AI agent independently trying to plant malware in a dependency thousands of businesses rely on is the kind of aggregation risk that keeps portfolio managers up at night. The agent's use of fake identities and social engineering is a tactic that AI for Cybersecurity Analysts must assess as well.
Social engineering and the legal backdrop
Speaking on a separate industry roundtable, Kareen Boyadjian, VP of Underwriting at Tokio Marine HCC Cyber & Professional Lines Group, said that "as artificial intelligence continues to evolve... we're seeing a huge increase in social engineering scams, voluntary wire transfer fraud, cryptocurrency theft, deep fakes." The line between AI as an attacker's tool and AI as the attacker is getting thinner from both directions.
On the claims side, Kennedys partner Arran Roberts has flagged that some ransomware groups now hand victims AI-generated legal risk assessments to manufacture urgency during the most chaotic hours of an incident. The AISI case adds a different mechanism: an AI agent that can impersonate real people to apply pressure.
Why this matters for insurance professionals
Current policy language, aggregation modelling and incident response planning were not built with agentic AI in mind. AISI's disclosure underscores that frontier models are capable of persistent, creative, unsanctioned behaviour that existing frameworks struggle to capture. For underwriters, this means reviewing exclusions for AI-driven unsanctioned actions. For brokers, it means advising clients on the basics - standard cyber hygiene, verifying outside code, and supply-chain certification like Cyber Essentials - as the gap between vulnerability discovery and exploitation shrinks to months. The question of "what the system understood about the real-world stakes of its actions" is one that insurers, lawyers and regulators will be debating for some time.
Your membership also unlocks:
