Anthropic CEO Dario Amodei's weekend proposal to "pace the frontier" of AI development has landed with a thud among safety advocates and political figures, many of whom see it as a belated attempt to manage risks his own employees helped spotlight only days earlier. The three-point plan, which calls for embedded third-party evaluators, shared safety standards among democracies, and coordination with China, drew sharp pushback from figures inside and outside the Trump administration as well as from leading AI researchers.
The whiplash-inducing sequence began when whistleblowers-including a resigning Anthropic researcher-warned that advanced AI poses an existential threat. Jacob Coxon said "the people building AI earnestly believe that it could kill us all by the end of the decade." Days later, Anthropic disclosed that users had been circumventing controls to use its models "in ways that could support biological weapons development," referencing novel mosquito-borne and bird flu viruses. The revelations prompted lawmakers in London and Washington to call for brakes and bans.
Amodei's response arrived this weekend. His plan asks each US AI company to grant ongoing access to embedded third-party evaluators who would check safety compliance, report incidents, and verify model alignment. Currently, such watchdogs are only brought in at a company's invitation or given limited investigative access after something goes wrong. He also wants common safety standards across democratic nations building frontier models, plus limits on unchecked progress. The third and hardest step would coordinate democratic AI powers with autocracies, notably China, starting with a narrow agreement banning AI for biological weapons production.
Political resistance and accusations of regulatory blackmail
Donald Trump has shown no appetite for framing AI as an extinction risk. He said last week he had no concerns about AI leading to human extinction and doubled down Sunday, saying people were "bringing up things that won't happen." Treasury Secretary Scott Bessent framed the stakes differently: "There is no day after tomorrow if China wins at this. If they were to pull away from us on AI, then nothing else would matter."
David Sacks, co-chair of Trump's council of advisers on science and technology, dismissed Amodei's approach bluntly. "The easiest way not to build superintelligence is for you to agree not to build it," he said. "Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system."
Safety experts call the plan structurally backward
Prof Stuart Russell, a leading AI expert, said Amodei's logic was "completely backwards." The proposal would slow capabilities progress to buy time for safety advances. Russell countered: "We don't just set a slower rate of progress for capabilities and then hope that provides enough time to get the safety right. We set the safety requirements and further progress occurs only when they are met." He drew a comparison to pharmaceuticals: "Imagine if a pharmaceutical company said: 'We're going to release a new cancer drug every year, and we hope that provides enough time for some clinical trials to be completed and for the results to be good.'"
David Krueger, an AI professor and former founding director of the UK's AI Security Institute, was equally dismissive. "Too little, too late," he said. "We need an immediate, indefinite, international moratorium on frontier AI development." The sentiment reflects a broader suspicion among independent experts about a global regulatory framework drafted by the leader of the world's most valuable AI company.
Amodei's ideas did attract early backing from rivals Sam Altman at OpenAI and Elon Musk at SpaceX, signaling that a coordinated slowdown might be more than talk. But the timing-after his own staff had seeded panic and after the company revealed bioweapons-related misuse-left critics questioning whether the plan was a genuine safety move or damage control.
Why this matters for IT and development professionals
For developers and IT teams working with generative AI and LLM tools, the safety debate has direct operational consequences. If embedded third-party evaluators become standard, development pipelines for frontier models will face new compliance checkpoints and incident-reporting requirements that could slow release cycles. The push for common international safety standards may also reshape how APIs and model access are governed across jurisdictions. Professionals building on these platforms should monitor whether voluntary industry pledges harden into enforceable rules-and what that means for access to the most capable systems. For those integrating AI into enterprise workflows, understanding the evolving regulatory landscape is becoming as critical as the technical skills covered in AI for IT & Development training.
Your membership also unlocks:
