Anthropic calls for coordinated AI pause if self-improving systems outpace safety efforts
Anthropic said Thursday that major AI labs should establish a coordinated mechanism to slow or temporarily halt development if advanced systems begin improving themselves faster than society can manage the risks.
The company framed recursive self-improvement-where AI systems build their own successors-as a threshold moment. Once systems reach that capability, "the ways we secure them, monitor them, and shape their behavior all grow much more important," Anthropic said.
At Anthropic's own operation, more than 80 percent of code merged into its codebase as of May was authored by Claude, its AI assistant. That metric illustrates how quickly AI-driven development can accelerate.
Why coordination matters more than solo action
A single company pausing development unilaterally would be straightforward to execute but largely ineffective. Such a move would shift competitive advantage to other labs rather than create space for broader safety work.
A meaningful pause requires agreement among multiple well-resourced labs operating at the technological frontier. It also requires clear rules: what conditions trigger a pause, what lifts it, and who oversees the arrangement.
Poorly coordinated slowdowns carry their own risk. If cautious actors pause while less cautious ones continue, overall safety could decline rather than improve.
What comes next
Anthropic's research arm plans to study and build systems that would support a development slowdown. The company will convene discussions with policymakers, researchers, civil society groups, and competing AI firms in the coming months.
Those conversations will focus on managing AI-specific risks and improving coordination mechanisms across the industry.
The proposal arrives as Anthropic confidentially filed for a US initial public offering on Monday. The company raised funding last month at a $965 billion valuation.
For managers overseeing AI adoption or strategy, understanding these governance questions matters. AI for Executives & Strategy courses cover the policy and coordination challenges that will shape how frontier AI develops.
Your membership also unlocks:
