Anthropic CEO Dario Amodei has called on frontier AI companies to slow the pace of model capability improvements, warning that within six to 12 months, AI systems could lead a "swarm" capable of taking over "the entire internet" and causing "hundreds of billions of dollars in damage." Both Elon Musk and OpenAI CEO Sam Altman publicly backed the proposal, with Altman confirming OpenAI will adopt similar third-party oversight measures.
In an essay titled We Must Pace the Frontier, Amodei outlined a three-part plan that includes embedding permanent third-party evaluators inside AI companies with employee-level access. He said Anthropic is unilaterally committing to this step now. "We'll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models' alignment during training," Amodei wrote on X.
Altman and Musk signal agreement
Musk, whose company xAI developed the Grok model, replied directly to Amodei's post: "Dario is right." Altman went further, writing that pacing the frontier has been "a primary topic of discussions we've had at OpenAI in recent weeks" and confirmed OpenAI will implement the embedded evaluator model. "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon."
The public alignment comes as OpenAI faces questions about its timeline for going public. Altman told Fortune magazine that an IPO this year would be "an ill-advised moment" given safety concerns. The New York Times reported in June that the company was weighing whether to delay a potentially trillion-dollar listing until next year. Altman also suggested OpenAI and other leading AI firms may be close to announcing an agreement to slow development and collaborate on safety risks.
The risk landscape driving urgency
Anthropic recently disclosed that bad actors have used its Claude models for weapons development, cyber operations, surveillance, and fraud. Amodei said AI has been "advancing drastically faster" since this summer, driven by AI's growing ability to build the next generation of AI. His timeline for potential internet-scale disruption is measured in months, not years.
Internal safety researchers have voiced stark warnings. Evan Hubinger, a top safety researcher at Anthropic, said there is a more than 10% chance that "AI could kill all humans" in the next decade. Researcher Jacob Coxon resigned from the company, saying Anthropic and OpenAI "are racing straight to self-improving superintelligence and gambling with our lives." Amodei clarified he is not calling for halting model training or technical progress, but for enough time to align and safeguard models before deployment.
The three-part pacing plan
Amodei's framework rests on three coordinated actions. The first - embedded evaluators - gives third-party teams ongoing, employee-like access to verify safety practices, report incidents, and assess alignment during training, not just after models are complete. He noted the approach has precedent in banking, where regulatory supervisors sometimes embed alongside employees.
The second step, democratic coordination, calls for frontier AI companies within democratic countries to establish common safety standards and limits on unchecked AI progress. Amodei acknowledged some forms of coordination would be legally challenging and require government support. The third, global coordination, involves the US and other democratic governments attempting to coordinate with authoritarian regimes while grappling with the difficulties of verifying compliance.
Former UK prime minister Rishi Sunak endorsed the call on X, writing: "Dario is right to call today for slowing the pace of frontier development so that safety can keep up. When the labs themselves are asking for speed limits, governments should listen." Sunak added that even as the US and China compete, "they urgently need a frank conversation about how to avoid catastrophe."
Why this matters for IT and development professionals
For software engineers and IT managers building on or deploying large language models, the embedded evaluator model signals a shift toward regulated development pipelines - akin to compliance frameworks in banking or healthcare. If adopted broadly, third-party access to training processes and alignment assessments could change how teams document model behavior, handle incident reporting, and structure release cycles. Professionals working with AI systems should track these proposals, as they may shape the tooling and governance requirements for production AI deployments. Those looking to build expertise in this shifting landscape can explore AI engineering courses or resources on AI for IT managers to stay ahead of emerging safety and compliance expectations.
Your membership also unlocks:
