Anthropic calls for global pause on advanced AI development
Anthropic is pushing for a worldwide slowdown in developing the most powerful AI systems, saying the latest models show signs they could escape human control.
The San Francisco company, which builds the Claude family of AI models, released a report Thursday arguing that a global pause would "likely be a good thing." It acknowledged the obvious problem: if only one company stops, competitors will race ahead.
"We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," the company said.
The coordination problem
A real pause would require multiple major AI companies across multiple countries - particularly the US and China - to agree and stop simultaneously under verifiable rules. Anthropic said this won't happen without a global coordination mechanism.
"Without a global coordination mechanism, companies and governments will have to make difficult decisions about safety while under competitive and geopolitical pressures," the company said.
The proposal faces significant resistance. Tech executives and US officials have repeatedly argued that any slowdown in AI development hands China a strategic advantage in what many consider the defining technology competition of this century.
Internal acceleration
Anthropic's report includes internal data showing that AI is already dramatically accelerating AI development itself. This creates a feedback loop that could lead to what researchers call "recursive self-improvement" - where an AI system teaches itself to become smarter without much human intervention.
"We are not there yet, and recursive self-improvement is not inevitable," the report said. But it warned the capability could arrive sooner than most governments and institutions are prepared for.
The company noted that "the human role is narrowing at each step in the AI development process."
What's actually happening
President Donald Trump said he discussed AI safety cooperation with China during a recent Beijing visit. He also signed an executive order this week giving the government 30 days to review the most powerful US AI models before release.
Anthropic compared the coordination challenge to nuclear arms control treaties but said it's harder to manage. AI training is far easier to hide than a missile silo, and the incentive to quietly continue development would be enormous.
The company plans to convene government officials, scientists, advocacy groups, and competing AI firms in coming months to explore how such a system might work.
Anthropic has faced pushback from industry peers and White House officials who say its focus on worst-case scenarios overstates risks and amounts to a competitive strategy disguised as safety concern. The White House has acknowledged the power of Anthropic's Mythos model - which hasn't been released publicly due to cybersecurity capabilities and remains available only to a small number of vetted organizations.
For development professionals, understanding generative AI and LLM capabilities and their acceleration trajectory is becoming essential as these systems move closer to autonomous self-improvement.
Your membership also unlocks:
