Anthropic Calls for Coordinated Pause on Advanced AI Development
Anthropic is pushing the world's leading AI companies to coordinate a slowdown or temporary halt on developing the most advanced systems, citing concerns that the technology is improving faster than safety measures can keep pace.
The company said in a blog post Thursday that as AI models accelerate their ability to perform complex tasks-including writing and debugging code autonomously-there's a real risk of losing human control over the systems.
The Self-Improvement Problem
Anthropic's concern centers on "recursive self-improvement," where an AI system becomes capable of designing and developing its own successor. At current development speeds and with sufficient computing resources, this milestone could arrive sooner than expected.
Such a capability would unlock advances in science and healthcare. It would also increase the risk that AI systems could operate beyond human oversight, the company said.
A Different Vision From OpenAI
OpenAI rejected this approach in a report released Wednesday. The company argued that "democratic governments - not private companies acting alone - must ultimately determine the rules, safeguards, and accountability mechanisms."
OpenAI said decisions about AI development pace should not rest with any single lab, company, or interest group.
How a Coordinated Pause Would Work
Anthropic proposed that advanced AI labs verify each other's compliance with a slowdown, preventing any competitor from secretly accelerating work while others hold back. Without such coordination, the company said, the least cautious players could pull ahead, pressuring others to abandon safety measures.
The pause would buy time for what Anthropic calls "alignment research"-work to ensure AI systems match human values and intentions-to keep pace with technical advances.
New Security Threats Emerge
The proposal arrives as researchers at the University of Toronto demonstrated how AI tools can create self-adapting malware that spreads across networks and compromises computing infrastructure.
Lead researcher Nicolas Papernot said the threat extends beyond the largest language models. The team built their proof-of-concept using open-source AI tools available to any developer, showing how accessible these capabilities have become.
Papernot noted that older, overlooked devices-an unused laptop in a basement, for example-can serve as entry points to attack high-value targets like hospitals, power grids, and banking systems. As the cost of mounting cyberattacks drops, everything connected to the internet becomes a potential target.
Papernot called for closer collaboration between companies, government agencies, and academic researchers to develop defenses against AI-powered hacking tools before attackers deploy them at scale.
Learn more about Generative AI and LLM development or explore how AI impacts IT & Development work.
Your membership also unlocks:
