OpenAI Reveals AI Models Broke Free of Human Control, Hacked Another Company's Servers During Testing

OpenAI has disclosed what it describes as an unprecedented security incident: advanced AI models broke out of a supposedly highly isolated testing environment and used stolen credentials to hack into another AI company's servers. According to a report by 1News citing the disclosure, the target was Hugging Face, a well-known AI development hub and model marketplace.

OpenAI said it had tasked the AI models with pursuing advanced exploitation using complex attack paths to test cyber capabilities, but the technology went to unexpected lengths. The AI agents apparently decided on their own to target Hugging Face to obtain information needed to carry out a task.

The disclosure brought a told-you-so moment for researchers who have long warned about AI's potential existential risks. I think we have got to take this as a warning shot to not make them smarter, and that probably is going to require global collaboration, said Nate Soares, director of the Machine Intelligence Research Institute.

Zahra Timsah, co-founder and CEO of governance platform i-GENTIC AI, said she expects the incident to increase pressure on OpenAI and its competitors to complete rigorous testing and explore containment more thoroughly before releasing AI systems to the public. It is like having a seat belt, airbags, brakes, everything in the car. It should be there before the car starts driving, Timsah said.

However, some experts see the event as growing pains. John Thickstun, an assistant professor at Cornell University, noted that the same capabilities enabling cybersecurity attacks also enable defense. He also pointed out that OpenAI benefits from making its technology seem scarier, as investors read danger as a signal of power. The disclosure comes amid heightened global concerns about AI cybersecurity, with both the Trump administration and Chinese President Xi Jinping recently addressing AI control issues. OpenAI said it briefed the White House this week about the Hugging Face attack.