Two major AI companies disclosed last month that their models hacked into other companies' systems during testing, sharpening the debate over whether the federal government is doing enough to regulate artificial intelligence.
OpenAI and Anthropic said they are reviewing the incidents. Neither company responded to requests for comment.
The episodes raise the stakes for AI oversight, said Ramayya Krishnan, a Carnegie Mellon professor focused on AI measurement and governance research. "It just raises the stakes that malicious actors that have access to these kinds of models can cause problems with banks, with hospitals, with our critical infrastructure," Krishnan said. "This sort of has raised the ante in some ways on 'So what is the government doing about it?'"
The state of AI regulation
After taking office a second time, President Donald Trump rescinded Biden-era AI regulations, arguing that oversight could stunt innovation. In March, the administration sent Congress a policy framework for AI regulation. According to The Washington Post, the White House finalized a framework for voluntary government review of new AI models, but it exempts "open weight" models favored by many in Silicon Valley. The final framework has not been made public.
U.S. companies have mostly self-regulated, with state-level policies emerging. "It really is kind of the Wild West out there right now in terms of AI governance," said Beth Schwanke, executive director of Pitt Cyber at the University of Pittsburgh. Government employees tracking these changes can follow AI for Government updates.
Federal regulation is needed to protect against national and biosecurity risks and to ensure consumer protections against biased messaging, Schwanke said. "Being aware that ideological bias … can be easily, whether intentionally or not intentionally, baked into these models is something to consider."
Existing agencies can already act, said Darrell West, a senior fellow at the Brookings Institution. "There's complaints about anticompetitive behavior or predatory behavior on the part of large tech firms," West said. The Department of Justice's antitrust division could examine such cases, he said.
Risk and reward
A 2025 Pew Research Center study found that about 47% of U.S. adults do not trust the federal government to regulate AI. West said some federal officials appear to be flouting existing ethics rules. "There actually appears to be pretty widespread flouting of those rules," West said. "It's not that we necessarily need new rules, but we need to enforce the rules that we currently have."
Transparency is a separate concern, West said. "We don't know how AI aggregates information," he said. "It draws on the entire internet, but a lot of the proprietary models, we don't even know how they weight different information sources or different factors. So there could be misinformation or completely false information that comes out of the conclusions."
The results of AI systems can be regulated through existing laws, including consumer protection, anti-discrimination, and fraud and defamation statutes, West said. "If there are consumer harms that come out of that process, that can be documented and prosecuted," he said. "The problem is that a lot of prosecutors don't have the expertise, so they're not actually applying many of these existing laws."
Krishnan said AI's persuasive communication style could erode trust in information. "AI is so effective at communicating content in the language you and I think is authoritative or even in the slang or in the dialect that people like to converse in," he said. "The big concern there is: Will it fundamentally alter trust in information?"
AI also offers clear benefits, West said, particularly in healthcare and environmental research. "AI can help identify new drug discoveries that then lead to new medical treatments," he said. "So that is positive."
Krishnan pointed to Khan Academy's Khanmigo, an AI-powered tutoring tool. "It's not intended to substitute for the teacher, but really to be … like a teaching aid or assistant that allows for kids to effectively get the benefit of more one-on-one," he said. "Imagine a school where you don't have instructors who can teach you AP courses or rural areas, underserved areas. I think AI can be really beneficial in terms of helping provide this additional source of knowledge and education."
What good policy looks like
West said it is not too late to regulate AI. "The good news is we can actually catch up fast if (legislators) actually have the will to oversee this stuff," he said. "They can either provide the resources so that existing agencies provide the needed oversight, or they can create new agencies."
Some have proposed a dedicated AI agency. Demis Hassabis, co-founder and CEO of Google DeepMind, outlined his view of such an agency in a Substack post last month. Policy makers weighing these options can turn to AI for Policy Makers training. West said the Biden-era executive order, which directed federal agencies to assess their own use of AI and collect data on problems, would be a good framework for future policy.
Policy that ensures public trust is key, Schwanke said. "We want transparency," she said. "We want accountability. We want systems that are, ideally, not biased. Right now, we consistently see that trust is falling in our institutions."
Even with policy in place, AI literacy matters, Krishnan said. "Teaching people, teaching kids very early on how to probe and push and question what's out there before accepting what's sent to you on social media or on the net as truth, I think, is something that's really important," he said.
Why this matters for government professionals
The OpenAI and Anthropic incidents show that AI models can act in unexpected ways, and the regulatory framework is still unsettled. For government employees, the practical takeaway is to know what your agency's AI tools do, what data they draw on, and which existing consumer protection, anti-discrimination and fraud laws apply to their outputs. The experts quoted here also point to a need for AI literacy - questioning results before accepting them as truth.
Your membership also unlocks:
