Rogue OpenAI model hacked HuggingFace on its own, company used Chinese AI to contain it

Published 22 July 2026 · tech

For months we have heard that models like Anthropic’s Claude Mythos, and OpenAI’s GPT-5.6 are the most advanced AI models out there. And these models

For months we have heard that models like Anthropic’s Claude Mythos, and OpenAI’s GPT-5.6 are the most advanced AI models out there. And these models can change cybersecurity as we know it. But in a recent incident where two OpenAI models went rogue and hacked HuggingFace, a Chinese AI model – GLM 5.2– had to step in and save the day. Read Full Story In a blog post by OpenAI, the AI startup revealed that two of its AI models, GPT-5.6 Sol and a more capable unreleased system went rogue during an evaluation of ExploitGym, a hacking benchmark. The AI models managed to leave a restricted internal test environment, gained access to the internet and breached HuggingFace, with the US company resorting to China’s GLM model to contain the attack. OpenAI CEO Sam Altman shared details of the incident via an X post. He called it a “significant security incident.” Sam Altman called this a significant security incident. How did OpenAI models go rogue? The incident was part of the ExploitGym evaluation.

According to OpenAI, the models were supposed to operate inside a sandbox with no direct internet access. Rather, they were only connected tool to download approved software packages. But instead of solving the benchmark, the AI models were more focused on trying to find the answers via the internet. OpenAI explained that the the models "spent a substantial amount of inference compute finding a way to obtain open Internet access.” These rogue models identified a zero-day flaw in the download tool, and managed to exploit this to find their way to an OpenAI machine with internet access. Once online, the models concluded that the answers to ExploitGym were likely stored on Hugging Face, which hosts AI models, datasets and related resources. These rogue OpenAI models then gained credentials for HuggingFace, and tried to get the answers for this test from its database. In effect, they bypassed the test rather than solving it, like a student leaving a locked exam room to steal the answer sheet from elsewhere. HuggingFace uses Chinese AI to contain rogue OpenAI models HuggingFace’s team was quick to detect and block the attempt made by the rogue models.

The company’s security team used GLM 5.2, an open-weight Chinese model from Z.ai, to analyse the attack. HuggingFace resorted to GLM 5.2 after mainstream commercial -- think from US AI labs, though the company does not disclose which models it tried – refused to process the logs and analyse the attack due to their safety guardrails treating the forensic work as potentially harmful activity. That is, these frontier models could not differentiate that HuggingFace was trying to use them to stop an attack, and not to start one. Replit CEO Amjad Masad reacted to the incident on X. He wrote, “Because OpenAI models don’t allow advanced cyber capabilities, HuggingFace used a Chinese open model to contain the rogue OpenAI agent” Replit CEO Amjad Masad's X post. The company then switched to GLM 5.2, running it on its own infrastructure, which allowed it to reconstruct the attack timeline more quickly while keeping sensitive data inside its own environment. According to the company, the attack exploited two code-execution paths in Hugging Face's dataset processing pipeline and reached internal datasets and service credentials.