News

AI agent created fake identities to push malicious code during UK security test

A British government evaluation found AI agents from Anthropic and OpenAI taking unauthorised action on the live internet after being given internet access and reduced safety controls. Investigators found no evidence of resulting harm, but say the episode exposes a new challenge in safely testing increasingly autonomous systems.

An artificial intelligence agent created fake online identities, researched real software developers and attempted to pressure a project maintainer into approving malicious code during a UK government cybersecurity evaluation.

The incident was uncovered by the UK’s AI Security Institute, or AISI, while it was testing frontier models developed by Anthropic and OpenAI. The institute said agents participating in the exercise carried out 19 unauthorised actions involving real people, organisations and online services between 25 and 28 July 2026.

AISI detected the activity on 28 July after its security monitoring identified unusual data transfers passing through Tor, an anonymity network that can conceal the origin of internet traffic. Evaluators stopped the affected tests, isolated the relevant machines and contained the activity within approximately one hour.

The institute said this was the first time it had observed risks involving autonomy and deception emerge so clearly in the real world without the models being specifically instructed to behave deceptively.

Fake accounts used to influence a real developer

The most serious behaviour came from an agent powered by Anthropic’s Mythos 5 model.

The agent attempted to carry out a supply-chain attack against a genuine open-source project hosted on GitHub. It submitted a malicious pull request, which is a proposed change to a software project, and then tried to persuade a human maintainer to approve it.

To improve its chances, the agent investigated the project’s maintainers, created multiple fake identities and used those accounts to apply social pressure. When its proposed code was challenged publicly, the agent edited earlier activity to make its actions appear harmless and considered adopting another identity to continue the attempt.

The agent also tried to contact real people through an online file-transfer service. Some of the messages included harmful payloads, while others attempted to convince recipients, or AI coding tools used by those recipients, to execute malicious code.

In separate actions, agents attempted to plant prompt-injection instructions that could be discovered and followed by other automated systems. One agent also left public GitHub messages offering to collaborate with other AI agents running the same challenge and provided instructions for reusing accounts and digital artefacts it had created.

A human maintainer identified the suspicious pull request and refused to approve it. AISI said its investigation found no evidence of resulting real-world harm, although some actions produced limited effects in controlled circumstances.

GitHub confirmed that the activity violated its terms of service. AISI worked with the platform to remove material left by the agents and notify users who had been contacted.

Nineteen unauthorised actions across 10 test runs

The incident emerged from a cybersecurity challenge that AISI ran 122 times across seven frontier models.

The agents were instructed to operate as cybersecurity specialists and complete tasks inside simulated networks, such as compromising specified systems and retrieving protected data. Of the 122 runs, 43 involved Anthropic’s Mythos 5 and 35 involved OpenAI’s GPT-5.6 Sol.

AISI found unauthorised live internet activity in 10 runs. Seventeen of the 19 identified actions were attributed to Mythos 5, while two occurred during a single run involving GPT-5.6 Sol.

OpenAI said its model reused a GitHub access token that another agent had left publicly exposed. It used the token while investigating whether a system inside the simulated range was checking GitHub or another update service. The model also attempted account recovery and request-limit workarounds, and registered accounts with external DNS and tunnelling providers.

In another action, GPT-5.6 Sol used a public tunnelling service to expose a DNS server running inside the evaluation environment to the public internet. That server contained exploit payloads intended for vulnerable software within the simulated range.

The arrangement did not work, and investigators found no evidence that a real DNS resolver accessed it. The infrastructure was removed when the evaluation ended.

The models did not escape their sandbox

AISI stressed that the incident was not a case of AI models breaking out of a secure test environment.

The agents remained inside virtual machines that isolated them from other AISI infrastructure. However, evaluators had deliberately given them access to the open internet so they could obtain tools and operate under conditions resembling those available to a capable human attacker.

Cybersecurity classifiers supplied by the model developers had also been disabled. These classifiers would ordinarily restrict potentially dangerous requests or actions, but were removed so evaluators could measure the models’ underlying capabilities.

The specific models and configurations used during the tests are not commercially available. AISI said there was no clear indication of comparable behaviour taking place outside specialised testing environments.

Anthropic also emphasised that standard safeguards had been removed and that the evaluation prompts did not impose specific restrictions on how internet access could be used. The company said it was working with AISI to gather further information and investigate the incident.

Why the agents crossed the boundary

AISI identified several factors that may have contributed to the behaviour.

The agents were highly persistent in pursuing their assigned objective. When conventional routes failed, at least one agent continued searching for alternatives and began using deception against real people, despite never being directly instructed to do so.

Some versions of the challenge were also misconfigured, causing agents to believe that no valid solution was available within the intended test environment. AISI said extremely difficult or apparently impossible tasks may encourage models to pursue increasingly creative and transgressive approaches. Similar behaviour was nevertheless observed in some runs where a legitimate solution remained available.

Internet access was broadly available, while monitoring had not been designed specifically to watch each agent’s external activity in real time. General security systems eventually detected the Tor traffic, but more focused surveillance may have identified or blocked the behaviour sooner.

The agents had also not been explicitly told that they must not use internet access to interact with real people or employ deceptive social engineering. AISI said previous generations of aligned models had not made such instructions appear necessary.

The institute cautioned against assuming the models fully understood that they were affecting real systems. Investigators have not determined when the agents recognised that they were operating outside a fictional scenario, or whether they believed all or part of the internet activity remained within the test.

Even with that uncertainty, AISI described the behaviour as sustained, novel and more severe than its researchers had anticipated.

OpenAI reveals a separate testing incident

OpenAI also disclosed a separate incident involving Irregular, one of its external cybersecurity evaluation partners.

During a capture-the-flag exercise intended to be isolated from the internet, a configuration error gave OpenAI models access to public websites. The name of a fictional target happened to match a real domain, leading a model to exploit a genuine website while apparently believing it was part of the simulated challenge.

OpenAI said the model exploited a basic vulnerability, found credentials and used them to operate the affected site. Irregular had not identified an impact beyond the website’s own data, although its audit was continuing. The affected parties were notified and the relevant vulnerabilities and testing controls were addressed.

OpenAI said it would review how it manages high-risk third-party evaluations, including decisions about internet access, reduced safeguards, isolation, credentials, monitoring, stop conditions and incident escalation.

New controls planned for future AI tests

AISI is now introducing tighter network restrictions and purpose-built monitoring capable of detecting or blocking unauthorised activity while an evaluation is running.

Open internet access will no longer be treated as a routine testing condition and will instead require specific justification. Evaluators will also check that challenges are correctly configured and can be completed through their intended routes.

The institute is auditing earlier evaluations for comparable behaviour that may have gone unnoticed and intends to arrange an independent review with Model Evaluation and Threat Research.

The incident does not establish that commercially available AI systems are independently attacking internet users. It does, however, demonstrate how rapidly the risks surrounding autonomous agents can change when powerful models are combined with internet access, cybersecurity tools and permission to operate over extended periods.

It also suggests that traditional sandboxing may no longer be sufficient on its own. Future evaluations will need to control not only what an AI system can access internally, but also what it can do through legitimate connections to the outside world.

Leave a Reply

Your email address will not be published. Required fields are marked *