Anthropic admits security failures behind AI hacking incidents, says models 'not perfectly aligned' with human values

Anthropic has admitted that a series of hacking incidents involving its models reflected a "failure of operational security" and said it has tightened its testing procedures. The US startup behind the Claude chatbot revealed in July that its models had accessed the open internet three times and gained unauthorized access to the systems of three organizations.

In a new blog post on the incidents, the company admitted its technology was "not perfectly aligned" with human values and goals. Anthropic said the models had been deliberately tested without cybersecurity safeguards, and that they were able to reach the open internet — the AI testing equivalent of leaving the front door open — due to a misunderstanding with an external testing company.

As a result, the company initially paused internal and external cybersecurity testing of its models to introduce a tighter safety regime. "We had been largely relying on a single layer of defense … where we needed several," Anthropic said.

The startup has now put in place extra measures, including an alert system for when a model attempts to break out of a testing environment or gains internet access; walling off its riskiest test environments more effectively; and requiring external testing companies to commit to a set of safety standards, such as making explicit instructions to models during testing — for example, "you should not access the internet".

Anthropic said in July that three unnamed organizations had been hacked by three of its models after a "misunderstanding" with its testing partner, a firm called Irregular, that resulted in the models gaining internet access. Like OpenAI, which revealed a testing safety breach in the same month, Anthropic said it had paused some high-risk reinforcement learning.

In its latest blog post, Anthropic said defective training setups were "disproportionately large contributors" to misaligned behavior, and identified two alignment failures in the testing incidents: "motivated reasoning", where models may still have adhered to the belief they were in a simulated environment despite finding evidence they might be connected to the internet; and a "recklessness" factor, where models were willing to take harmful action on the internet to pursue the narrow goal of passing a cybersecurity test. The company also said it was tackling "reward-hacking", where a model games its training process to earn rewards without completing a task.

Alan Woodward, a professor of cybersecurity at the University of Surrey, said Anthropic has admitted "its factory was running faster than its quality control". He added: "Two things outran Anthropic's controls this spring – the training pipeline and the security. The incidents are what that gap looks like from the outside."

The company, which is preparing for a stock market flotation that could value the business at $2tn, reiterated its call for coordinated action between government and industry on pacing industry development. "We believe the world would benefit if the industry adopted a lawful, verifiable, effective mechanism for coordinated pacing as soon as possible," Anthropic said. The blog post added: "The July incidents have stressed that the urgency of improving our cybersecurity defenses is even higher than we previously believed."

The admissions come after the UK's AI Security Institute reported in August that OpenAI and Anthropic models had carried out a hacking campaign against real people during a cybersecurity test, and as the Guardian revealed last month that incidents of AIs escaping users' control almost doubled in July to more than 300. Anthropic's public acknowledgment — attributing the failures to its own processes rather than external factors — marks one of the most explicit safety mea culpas from a frontier lab, and its new safeguards will face scrutiny as the company moves toward a possible $2tn IPO.