OpenAI is scared of open-weight models. Should the US be?
The impressive capabilities of Chinese lab Moonshot’s Kimi K3, the biggest open-weight large language model, has kicked off a debate that conflates two things: the
The impressive capabilities of Chinese lab Moonshot’s Kimi K3, the biggest open-weight large language model, has kicked off a debate that conflates two things: the economic possibilities of American AI giants and the future of LLMs as a technology. OpenAI’s head of strategic futures, Dean W. Ball, went so far as to argue that the US government should find a pretext to create regulatory fear, uncertainty, and distrust around the new models, since open-weight models must necessarily deter capital spending by the frontier labs. People freaked out, with tech luminaries like Yann LeCun and Martin Casado arguing that open software can accelerate innovation and coexist with proprietary projects. Ball soon retracted his claims that a regulatory crackdown was the White House’s “best strategy” and that open-weight models necessarily slow down advances in the technology. However, Axios reports that the Trump administration is considering banning K3 and other advanced Chinese models at the behest of American frontier labs. Another report from Politico said that the Department of Commerce would not take that step anytime soon. The benefit for major AI companies is clear: Open-weight models, running on independent infrastructure or inside major enterprises, offers cheaper intelligence than Anthropic or OpenAI’s class-leading models. If users increasingly spend more outside the closed labs, that means smaller return on their massive investments in model training. That view extends far beyond OpenAI. “Strong, frontier-caliber open source models will place a squeeze on the margins and will bring down the prices of the frontier companies,” Braden Hancock, the co-founder of Snorkel AI and a research partner at the Laude Institute, told TechCrunch. “It will not necessarily mean that the amount of AI usage goes down a little bit.
You know, obviously, quite the opposite.” That’s not a problem for people without shares in Anthropic and OpenAI. AI will still proliferate. So what’s the justification for the government to block Americans from purchasing something in our ostensibly free markets? Concerns over Chinese models come in several flavors. One is protecting US data from the Chinese government; the US banned the import of modern Chinese EVs over concerns about their data gathering. But experts tend to think that open-weight models run on US servers are unlikely to leak data back to China, although it’s not impossible that such a thing could be done. Another is that the models may have implicit bias toward the PRC — but it’s not clear what that might mean for, say, coding tasks. A third common worry is that Chinese models lack the guardrails that the US government has mandated (through an opaque process), which aim to prevent leading US LLMs from being used to exploit closed computer systems or create weapons. However, those same guardrails may make US companies more vulnerable: David Sacks, the venture capitalist and Trump adviser, has been sharing cases of US companies turning to Chinese LLMs to close security gaps when US frontier models refuse to do the tasks. But the most significant motivation for restricting the models is that fear that China will be able to outpace the US if the frontier labs slow down. Sam Bresnick, a China-focused research fellow at Georgetown’s Center for Security and Emerging Technologies, says the growing importance of AI to the US military operations gives the US a reason to support continued investment in AI at the frontier labs.
