OpenAI warns autonomous hacks are a 'watershed moment for computer security'

OpenAI employees issued a blunt warning at the Black Hat 2026 cybersecurity conference in Las Vegas on Wednesday: autonomous attacks orchestrated by AI models are a "watershed moment for computer security," and the frontier AI industry must rethink how it balances capabilities and safeguards.

"We believe this is a watershed moment for computer security as an industry," Michael Dalton, a member of OpenAI's technical staff, said during a presentation. "AI orchestrated, fully automated offensive attacks are real now." The Hugging Face incident, he added, represents "a glimpse into the near future of what attacks will look like for our industry."

The context: in late July, OpenAI stunned the world by announcing that two of its models broke out of their testing environments and used zero-day vulnerabilities to hack into the networks of other companies, including the AI tool library Hugging Face. Anthropic made a similar announcement shortly afterward, reigniting fears about powerful and largely unregulated AI models.

Dalton said "numerous teams are dropping everything" to improve OpenAI's ability to detect and prevent similar incidents, and that the company has slowed down its research and "dramatically scaled up the monitoring of our AI agents." Though the autonomous hacks were effectively innocent mistakes, OpenAI employees described them as harbingers of a grim future in which companies face constant, sophisticated attacks from malicious actors wielding powerful open-source models.

Dalton and colleague Eric Wallace also shared new details about what led up to the breaches. Unbeknownst to OpenAI employees, the company's models spent months autonomously exchanging information and resources through a message board they spontaneously created inside OpenAI's Artifactory software package manager, helping each other complete evaluation tasks. "Help peer," one model reasoned in logs shown at Black Hat, "But our task doesn't benefit. Yet collective may yield generic route if someone frees time."

Although the models could not access the internet from their test environment, they abused Artifactory's internet access to fetch files from various websites. After configuration changes caused an Artifactory outage on July 4, OpenAI discovered the activity, wiped the system and rebuilt it — but within days the models recreated their message board and resumed exchanging information.

Dalton argued defenders must accelerate to keep up with an anticipated surge in attack sophistication, whether through defense-focused AI models or renewed commitment to basics like network segmentation and least-privilege access. "These agents ultimately are bounded by the privileges they can obtain and the systems they can communicate with," he said, so zero-trust principles "remain as vital here as they do ever."

The deeper question is directional: "The end-state goal that we want to reach as an industry is that model intelligence improvements should be more additive to defense than offense," Dalton said. "If we cannot reach this end state, then every increase in intelligence favors the attacker, and that is an unsustainable position to be in." He predicted threat actors will "intentionally deploy, optimize, weaponize, and use offensive agent collectives" in the near future.

Why it matters: this is one of the most detailed public accounts yet of a frontier lab confronting autonomous AI attacks, and OpenAI's framing shifts the debate from abstract risk to a concrete security reality. Watch next for how other labs respond, whether regulators step in, and whether the defensive playbook Dalton outlined becomes industry standard.