AI leaders call for government regulation as safety concerns mount

Leaders at several major AI companies have publicly called for government regulation and an industry-wide slowdown, citing catastrophic risks from rapidly advancing systems. For public-sector professionals, the push raises a familiar red flag: regulatory capture, where dominant firms seek rules that lock in their market position while claiming to serve the public interest.

Dario Amodei, CEO of Anthropic, published a blog post detailing concerns over recursive self-improvement - the point at which AI builds better versions of itself. "These systems could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all," he wrote. He speculated that a swarm of AI agents could soon be "capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage)."

Elon Musk responded directly: "Dario is right." Sam Altman of OpenAI said his company would delay its IPO over safety concerns. Jacob Coxon, a former Anthropic researcher, put the internal mindset more bluntly. "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt," he wrote on X. Evan Hubinger, Anthropic's Alignment Science lead, added, "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade."

The old playbook of regulatory capture

Amodei's proposed solution involves government coordination. "Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress," he wrote. The framing sounds responsible, but the mechanics mirror a strategy government veterans have seen before: dominant players inviting regulation that raises barriers for newcomers while cementing incumbents' standing.

Think of how NV Energy fought to protect its monopoly on electricity. When government imposes industry-approved controls on AI, it grants those top companies quasi-permanent status. Note that no one is mandating the creation of new AI models - and it is doubtful many government bureaucrats understand the technology well enough to write sound rules from scratch.

Government regulation could also provide these firms with liability protection if their agents cause damage. It may even blunt local backlash to data center construction. But slowing U.S. innovation artificially does nothing to hinder China's ambitions in the same space.

Separating genuine risk from market positioning

AI does present novel concerns that deserve serious attention from policymakers. The challenge is distinguishing between authentic safety needs and demands designed to freeze a competitive landscape. Just because a company develops new technology does not mean time-tested principles - like skepticism toward industry-written regulations - stop applying.

For readers working in AI for Government roles, the pattern is worth recognizing. When an emerging industry's biggest players ask for rules that slow everyone down, the public interest and the corporate interest rarely align by accident.

Why this matters for government professionals

You will be asked to evaluate AI regulations in the coming years. Apply the same scrutiny to tech executives' calls for oversight that you would to any regulated utility seeking to protect its monopoly. Ask who benefits from the proposed rules, who gets locked out, and whether the safety argument holds up under independent analysis - not just the warnings of those who stand to gain from restricted competition.