One of China’s Most Powerful AI Models Has Also Escaped Containment
The AI industry is having a rogue agent summer. The latest model to escape onto the open internet during security testing is Kimi K3, a
The AI industry is having a rogue agent summer. The latest model to escape onto the open internet during security testing is Kimi K3, a powerful open-weight offering from the Chinese company Moonshot AI. Frontier Security, a US startup, says that Kimi K3 went outside of its sandbox while testing its defensive cybersecurity skills. As with incidents previously reported by OpenAI and Anthropic, the escape was partly enabled by a misconfiguration in the sandbox designed to contain it. Frontier claims, though, that the incident shows Kimi has fewer cyber safeguards than most other powerful AI models, something that allowed it to go off and use the internet without express permission. “We found a leak in the sandbox,” says Yaron Singer, CEO of Frontier Security. “But we also found that Kimi took advantage of that loophole—suggesting that it doesn't have [the same] internal guardrails.” Unlike other recent incidents of AI agents going off-script, Kimi K3 did not hack anything after accessing the internet—because the answers to the problems it was seeking were easily attainable on GitHub.
Moonshot did not respond to a request for comment by time of publication. The incident is the latest in a string of agent mishaps that suggest increasingly cyber-capable AI models are becoming more challenging to control. Last month, OpenAI disclosed that an unreleased model had broken out onto the internet and then hacked Hugging Face, a company that hosts AI models and data, in order to find answers to problems it was tasked with solving. OpenAI subsequently shared that its AI agents had in fact hacked into four additional services as part of the spree. Shortly after OpenAI reported its incident, Anthropic revealed that several of its models had also gained access to the internet and attacked outside systems. Last week, the AISI also disclosed that in its own testing, versions of OpenAI and Anthropic models that had security safeguards disabled perpetrated multiple hacks across the internet, including a particularly ambitious attempt by Anthropic’s Mythos 5 to plant malicious code in an open-source project on GitHub.
While these AI hacking episodes all vary in both cause and degree, the Kimi K3 is similar to several of them in that a misconfigured sandbox allowed access to a number of websites rather than keeping it contained to a simulated environment. The model was expressly tasked with solving problems that should not have involved going off to find the answers online, and appears to have gone outside of those instructions. The model had to figure out for itself that it had access to certain websites by probing the network settings of the sandbox. While human error appears to have played a major role in each of the breakouts, the consequences have been compounded by the fact that advanced AI models are designed to use reason and take complex actions in order to solve problems. Another key difference between previous incidents and the one discovered by Frontier Security is that it involves a model that is already widely available, with the same safeguards an average user would encounter.
The BriefWire