WASHINGTON - U.S. Senate negotiators are debating a bill that would require AI developers to design their products to prevent "catastrophic risks," while giving the federal government authority to block the release of models deemed unsafe. The proposal, still in flux ahead of the November midterms, would also preempt state AI safety laws covering certain model risks.
The measure would create a "duty of care" for AI developers, according to two Senate aides. Companies building the most advanced AI models would need to design products with the goal of preventing severe harms, including the possibility that a bad actor could use an AI system to design nuclear or biological weapons.
Federal courts would arbitrate government blocking decisions
Under the proposal, the U.S. government would gain the power to block the release of certain AI models deemed unsafe, according to a source familiar with the deliberations and a Senate aide. Companies could challenge those decisions in federal court. The exact structure of that authority is still being negotiated.
The bill would apply only to AI models with the most advanced capabilities. U.S. companies in that category include Alphabet's Google, Anthropic, and OpenAI.
State preemption and the negotiating table
Part of the measure would stop states from enforcing their own laws governing specific AI model risks, the two Senate aides said. This preemption provision is a key element for industry groups that have pushed for a single federal standard rather than a patchwork of state rules.
The negotiations are led by Senate Majority Leader John Thune, Senate Commerce Committee Chairman Ted Cruz, and Senator Amy Klobuchar, the top Democrat in the talks. Senator Maria Cantwell, the top Democrat on the Commerce Committee, has also weighed in. The committee oversees the Commerce Department, which houses in-house AI safety researchers.
Klobuchar said in a statement: "I'm continuing to work toward a bipartisan agreement on legislation for government oversight of the greatest risks posed by AI models. That includes requiring developers to work with government experts to verify and test models to make sure AI is safe."
Cantwell posted on X that "meaningful legislation would require the most powerful AI models undergo testing by scientists and experts at our national laboratories" to assess whether they could enable cyberattacks or aid development of biological or nuclear weapons.
A tight congressional calendar
Even if negotiators reach agreement, the calendar is not on their side. The House is scheduled to be in session for only one week before the November 3 midterm elections, while the Senate has three voting weeks remaining.
The talks come amid rising pressure on Congress and the White House to address risks from advanced AI systems. Concerns have grown following reports of AI agents deviating from human instructions and hacking external systems, as well as public warnings from AI researchers who left leading companies over safety concerns. Lawmakers in both parties have called for greater oversight in recent days.
For policy staff tracking federal AI legislation, the duty-of-care framework and state preemption language will shape compliance obligations across agencies. The AI Learning Path for Policy Makers covers the governance concepts underpinning these debates. Those working in public sector roles can also follow developments through AI for Government resources.
Why this matters for government professionals
If this bill moves forward, federal agencies will likely play a role in testing and evaluating advanced AI models before release. Government professionals involved in procurement, cybersecurity, or regulatory compliance should track the duty-of-care language closely, since it may define what "safe" means for AI systems that agencies eventually buy or integrate. The state preemption provision also means your agency's state-level AI obligations could shift depending on the final text.
Your membership also unlocks:
