California lawmakers push guardrails on state agencies' AI decision systems
California lawmakers are moving to restrict how state agencies use artificial intelligence to make decisions about public benefits and licenses. On Monday, the state Senate Committee on Privacy, Digital Technologies, and Consumer Protection approved legislation that would prevent automated decision systems from being the sole reason for rejecting an application.
The bill would ensure that a human reviews any decision to deny someone CalFresh benefits, a teaching license, or a nursing certificate due to an algorithmic error or bias.
The problem the bill addresses
State agencies already use automated systems to process applications faster. The California State University system automatically admits qualified high school graduates. Medi-Cal eligibility determinations run through algorithms that would otherwise overwhelm county offices.
But without oversight, these systems can lock people out of benefits they qualify for. The legislation requires a human decision-maker to review any rejection.
Labor unions oppose the bill despite sharing concerns
Three major labor groups registered in opposition to the bill Monday: AFSCME California, the California Federation of Labor Unions, and SEIU California.
SEIU California said the bill was introduced without enough input from unions and raised unresolved questions about job impacts. The group cited concerns about erosion of professional judgment, algorithmic bias, job monitoring, and due-process issues.
SEIU California also worried the legislation could interfere with contract negotiations between the state and SEIU Local 1000, the largest union representing state workers. Those negotiations begin Wednesday.
State Sen. Christopher Cabaldon, D-West Sacramento, said his bill does not intend to interfere with collective bargaining.
Agencies are already deploying AI systems
The Newsom administration has plans to launch more AI pilots across state agencies. The California Department of Tax and Fee Administration already uses an AI assistant to help call center employees answer taxpayer questions faster.
Cabaldon said lawmakers need to act now because the systems are already in use. "The train is already running," he said. "So if the legislature doesn't step forward with a framework to guide this, it will still be happening but it will be happening without our oversight."
He noted that no other bills this session address issues like algorithmic bias, where AI systems produce unfair outputs based on skewed training data.
Senate Bill 1248 passed its second committee hearing Monday and continues through the legislative process.
Learn more: AI for Government or explore the AI Learning Path for Policy Makers.
Your membership also unlocks:
