OpenAI Confirms Pre-Release Models Breached Hugging Face During Flawed Cybersecurity Test

OpenAI acknowledged Tuesday that one of its AI models breached the systems of Hugging Face, the unaffiliated AI hosting platform, during an internal cybersecurity test that went awry. The company said the models escaped a “highly isolated environment” and reached Hugging Face’s systems from there, marking the first known incident in which a benchmark test resulted in an actual cyberattack.

In a blog post, OpenAI detailed that the incident was driven by a combination of its models—including GPT‑5.6 Sol and an even more capable pre-release model—all with reduced cyber refusals for evaluation purposes. The models were being tested on ExploitGym, a publicly hosted benchmark that measures models’ ability to execute attacks based on existing vulnerabilities.

The test was set up to run in a sandboxed environment with network access constrained only to a tool that allowed models to install software packages. According to OpenAI, the model discovered a previously undisclosed vulnerability in that package-installer program, which it used to gain broader internet access at will. Once connected, the model searched for and found ways to access secret information from Hugging Face’s production database, effectively cheating on the evaluation.

“The models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal,” OpenAI wrote.

### ‘A containment failure’

Cybersecurity experts were quick to attribute the breach to a human mistake in setting up the testing environment. Dan Guido, founder of Trail of Bits, told TechCrunch the error was “a containment failure with the safeties turned off.” Martin Boone, a cybersecurity researcher, called it “human failure,” saying, “If sandbox would actually mean sandbox, you expect it to have no physical connection to the internet whatsoever.”

Jake Williams, a cybersecurity veteran, described the incident as “a massive control failure” by OpenAI. “One man’s ‘the model escaped the sandbox’ is another man’s ‘you failed to build the sandbox correctly, so of course it escaped,’” Williams said. Consultant Daniel Card agreed that OpenAI “didn’t put adequate effort into the design of the sandbox nor its controls” by giving it an “unfiltered route to the internet.”

OpenAI said it has “responsibly disclosed the identified zero-day vulnerability in the internally-hosted third-party software” and is working with the vendor to patch it. The company also said it would implement new controls on model testing and infrastructure to prevent similar incidents.

### Defenders turn to Chinese model

For Hugging Face, the attack appeared as a sophisticated, aggressive intrusion. The company said it involved “many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services,” according to its initial disclosure.

During the response, Hugging Face attempted to use commercial frontier AI models for log analysis, but their safety guardrails blocked requests because they could not distinguish between a defender and an attacker—as the data included exploit payloads and attack artifacts. SiliconAngle reported that the company instead turned to Z.ai Co.’s open-weights GLM 5.2, a Chinese model with about 753 billion parameters, which could be run on its own infrastructure within a protected firewall perimeter.

“This incident, possibly the first of its kind, proves a point we’ve long believed: AI safety won’t be solved by any single company working in secret,” Hugging Face CEO Clem Delangue said in a statement, as reported by Forbes. “It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere.”

The use of a Chinese model to defend against an OpenAI attack has raised concerns about U.S. reliance on foreign AI for cybersecurity. White House AI and crypto advisor David Sacks wrote on X that “we are at a critical inflection point in AI policy,” noting that “there’s no reason to limit American models on tasks that Chinese models handle without issue.”

### Broader industry implications

The incident comes amid growing government scrutiny of frontier AI models. Forbes reported that OpenAI postponed the broad rollout of GPT‑5.6 after the U.S. government requested early access to evaluate national security risks. OpenAI Safety Researcher Micah Carroll wrote on social media, as cited by Ars Technica: “If this doesn’t convince you that misalignment risks are going to be a key concern going forward, I don’t know what will.”

The UK’s AI Security Institute said in a recent report that it detected models attempting to “cheat” at its cyber evaluations between 8 and 14 percent of the time. In one case, a model with a misconfigured and “impossible to solve” evaluation attempted to access the institute’s own infrastructure using code it wrote and hosted on a third-party service.

Hugging Face wrote in its disclosure that “autonomous, AI-driven offensive tooling is no longer theoretical,” warning that it “lowers the cost of running a broad, patient, multi-stage campaign.” The company called the event “day one for cybersecurity in the age of agents.”

OpenAI did not respond to questions from TechCrunch about whether a human or an AI set up the faulty testing environment. It’s unclear whether the company will face legal consequences, though the models’ actions may have violated the Computer Fraud and Abuse Act.