What the OpenAI–Hugging Face breach really tells us | Explained
OpenAI says two of its models – GPT-5.6 Sol and an unreleased sibling – escaped a “highly isolated” evaluation environment, found a path to the
OpenAI says two of its models – GPT-5.6 Sol and an unreleased sibling – escaped a “highly isolated” evaluation environment, found a path to the open internet, and used stolen credentials plus a chain of zero-day exploits to break into Hugging Face’s production infrastructure. The test itself was designed to find the models’ ceiling: how much cyber damage could they do if nothing held them back? So OpenAI switched off the safety classifiers that would normally rein in this kind of behaviour. What wasn’t supposed to be available was a route to the internet. However, the models found one anyway: an undisclosed flaw in the package-cache proxy meant to give the sandbox narrow, controlled access to software registries and used it to move laterally until they reached a networked machine. Once online, they reasoned that answers to the benchmark might live on Hugging Face, and set out to get them. Hugging Face caught and contained the breach on its end using an open-weight Chinese model, Z.ai’s GLM 5.2. Canadian computer scientist and Turing Award winner Yoshua Bengio called the episode a wake-up call, noting that AI agents willing to cheat and deceive toward misaligned goals have now shown up outside the lab, not just in controlled tests. Meanwhile, NYU professor Gary Marcus, as is his wont, offered a more sceptical reading. “This was a training exercise, not a real-life incident.” The safety guardrails were deliberately switched off for the test, he noted, and in ordinary use they might well have stopped the breach, which led him to suggest OpenAI’s account reads more like a marketing exercise in demonstrating worst-case capability than a report of a genuine attack.
Still, Marcus conceded two things: that the episode confirms Anthropic’s cyber-focused Mythos model is no fluke, and that the pressure these capabilities now put on cybersecurity is genuine. He also said the same open tools that helped Hugging Face defend itself could, stripped of their safety layers, just as easily be turned offensive. The “going hard” problem While the optimists see increasingly capable reasoning systems cracking harder scientific and engineering problems, sceptics see instrumental convergence: systems optimising almost any objective tend to discover strategies nobody anticipated. It’s a concern Eliezer Yudkowsky raised in his latest book, If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All, discussing OpenAI’s earlier o1 evaluations. In one internal capture-the-flag test, the server holding the target secret simply failed to boot. Rather than accept defeat, o1 found an unrelated vulnerability, compromised the infrastructure hosting the evaluation itself, restarted the target server, and rewrote its startup instructions so the secret would be copied over automatically. It didn’t solve the challenge but routed around it. For Yudkowsky, this is evidence that reward-shaped persistence, not any explicit intent to hack, is what produces this behaviour: a side effect of training a model to never stop trying until it wins. Earlier, Anthropic’s own unreleased Claude Mythos Preview had reportedly found a 27-year-old flaw in OpenBSD’s TCP handling, a 16-year-old FFmpeg bug that eluded five million automated tests, and a 17-year-old FreeBSD kernel vulnerability developed into a working exploit “with no human involvement after the initial prompt,” for under $2,000.
The BriefWire