Senate AI bill stalls over who should test models for catastrophic risk
A bipartisan AI regulatory bill taking shape in the Senate Commerce Committee has hit a wall over a core safety question: whether companies should test their own models for catastrophic risk, or whether federal agencies should do the testing. The disagreement pits ranking member Maria Cantwell, D-Wash., against the bill's sponsors - Amy Klobuchar, D-Minn., John Thune, R-S.D., and committee Chair Ted Cruz, R-Texas - according to multiple people familiar with the negotiations.
The outcome matters for IT and development professionals because the bill's testing framework could determine whether deploying large language models requires federal pre-approval or remains largely voluntary.
Self-testing vs. federal vetting
The draft under development would let AI companies run their own safety tests and submit results to the Commerce secretary for deployment approval. Cantwell wants a different approach: mandatory testing by federal agencies, including national labs and national security bodies.
"Cantwell really wanted to get these large language models tested and to have mandatory vetting, but the Cruz-Klobuchar-Thune version now that she is opposing, it's primarily a voluntary standard type situation," a Democratic committee aide told Nextgov/FCW. "Her position all along has been: we need to address catastrophic risk, but we need to do this in a serious way."
Cantwell's office has offered alternative language to Klobuchar's team, which was described as "very receptive." Cruz's approval remains the deciding factor.
Pressure from the AI safety community
Anthropic and AI safety groups have aligned with Cantwell's position, according to one person familiar with the matter, who said they are "refusing to play ball" with the current draft. The source said Republicans have already included legal duties for companies to manage catastrophic risks and provisions for government verification.
One AI safety group told Nextgov/FCW: "We need pre-deployment testing of frontier models and strong national standards that keep pace with fast-growing capabilities and risks."
The push for stronger guardrails gained momentum after a former OpenAI and Anthropic employee alleged on X that neither company is acting responsibly while developing advanced AI. Cruz called the allegations "highly concerning" and said the forthcoming legislation will address catastrophic risk.
"This is scary stuff, but we're also not going to be able to stick our head in the sand and pretend technology isn't happening," Cruz said Wednesday. "So we've got to put some guardrails on it."
The path forward
Klobuchar said in a statement that AI deployment with little oversight carries "significant risk" and that Congress must act. "That includes requiring developers to work with government experts to verify and test models to make sure AI is safe," she said. "We cannot allow the release of dangerous models, including those that evade the control of their developers."
The bill builds on the Artificial Intelligence Research, Innovation, and Accountability Act of 2023, which Klobuchar and Thune introduced last Congress. That bill stalled, and the new version with Cruz's involvement is likely to be "a fundamentally different bill," according to the Democratic committee aide.
Cantwell took to X on Thursday to reiterate her position: "Meaningful legislation would require the most powerful AI models undergo testing by scientists and experts at our national laboratories to assess whether they could enable sophisticated cyberattacks or aid the development of biological or nuclear weapons."
For developers and IT teams working with Generative AI and LLM tools, the outcome of this fight will shape deployment requirements. If Cantwell's federal testing mandate prevails, organizations may need to route frontier model deployments through government vetting before release. A voluntary standard, by contrast, would keep testing in-house.
Why this matters for IT and development
If the bill moves forward with mandatory federal testing, IT teams deploying large language models in production could face new compliance steps before release - including pre-deployment security reviews by national labs. If the voluntary framework holds, companies retain control over their own safety evaluations but may face stricter legal liability if a model causes harm. Either way, the legislation will define what "catastrophic risk" means in legal terms, and that definition will influence how AI for IT & Development professionals document model testing, track safety metrics, and structure deployment pipelines.
Your membership also unlocks:
