Anthropic CEO does not know how US used Claude in strikes on girls school in Iran
Earlier this year, the US Pentagon announced that it was going to stop using Claude AI models after a fallout with Anthropic over unrestricted AI
Earlier this year, the US Pentagon announced that it was going to stop using Claude AI models after a fallout with Anthropic over unrestricted AI use. While the US military wanted to use AI for âall lawful purposes,â the AI startup wanted to restrict the use in certain cases. But in February, when the US started strikes in Iran, Claude was still being used by the Pentagon, and is said to have played a role in a strike on a school that killed 175 people. But Anthropicâs CEO, Dario Amodei, says that he has no idea how his companyâs AI tools were used in this strike. Read Full Story âLook, we donât have access to, we donât know exactly how these models were used,â Amodei told Bloomberg.
The Anthropic chief claimed that the strike was âa really terrible thing to happen.â The Pentagon has not claimed responsibility for the incident, but reports indicate that US forces were likely behind the bombing. As per reports, the US military may have launched strikes on the school based on outdated intel that wrongly identified the area as still being a military site. Though it is unclear how the AI tools may have played a role in the incident. School strike shows why AI principles are important, says Dario Amodei One thing that Dario did emphasise upon was that it is likely that a human made the final call regarding the strike, something that was one of the demands from Anthropic over the use of its models by the military.
He explained, âThe principle that we have established, and I think the principle that was obeyed here, is that a human makes the final decision.â According to the Anthropic CEO, the terrible incident only shows why such principles are necessary as militaries start deploying AI tools in combat. He added, âI donât know what role Claude or any other AI had, but if this isnât an illustration why that principle is so important, I donât know what is.â Amodei also said that the use case, if Claude was used in the operation, did not violate the companyâs policies. According to the report, he argued that military leaders make mistakes âeven at the best of timesâ. He also defended the broader use of AI in conflict, saying, âWe donât want a world where China and Russia can build, can analyse all the intelligence with AI, can use AI for attacking Taiwan and Ukraine, and we canât defend them.â Do note that Anthropicâs AI models are said to be phased out by the US Department of Defense soon.
The AI startup was labelled a âsupply chain riskâ after the Pentagon refused two conditions â to not use AI for domestic mass surveillance, and to not develop autonomous weapons. The company has gone to court over this label. OpenAI is said to fully replace Anthropicâs models in the US military. However, recent reports indicate that US agencies like the Security Agency (NSA) are using Claude Mythos to potentially plan global cyberattacks. Ends
