A California transparency law meant to catalog high-risk automated decision systems used by state agencies initially reported zero such systems. A follow-up inquiry then uncovered at least six systems making consequential decisions about housing, cash assistance, and medical care - a finding that underscores the law's fundamental design flaws.
Assembly Bill 302, signed three years ago, required the state's Department of Technology to publish a yearly inventory of all high-risk automated decision systems proposed or used by any state agency. The department's first report, released in 2025, found no systems that met the threshold. A public records request for the underlying data yielded a single spreadsheet with a column labeled "Is ADS used" and the word 'no' listed for every agency.
The department interviewed a handful of agencies in a subsequent cycle, and that's when six systems came to light. The process depends entirely on agencies self-reporting, with no verification mechanism and no consequences for failing to disclose a tool. The law also contains no clear definition of what qualifies as "high risk." The Department of Technology's report suggests that agencies themselves decide whether a system is reportable, a loophole that leaves the public with an incomplete picture.
The self-reporting problem
AB 302 describes high-risk systems as those that "assist or replace human discretionary decisions that have a legal or similarly significant effect," covering access to housing, education, employment, credit, health care, and criminal justice. Yet known systems like the Uniformity Assessment System, which reduces In-Home Supportive Services for disabled Californians, and the Risk Segmentation, Stratification, and Tier model used to predict Medi-Cal recipients' risk and service underutilization, were not reported. The omission suggests agencies interpret the statute's language narrowly or simply ignore the requirement.
Similar transparency bills have run into the same wall elsewhere. New York's Public Oversight of Surveillance Technology Act was meant to shed light on the NYPD's use of surveillance tech. The police department exploited legal vagueness to avoid scrutiny, describing its own capabilities in terms that undermined meaningful oversight, according to the Brennan Center for Justice. The NYPD's Inspector General and outside groups documented how the department kept technologies like robotic dogs out of the public eye.
"Community control" bills that require police to disclose their own technology have produced similar frustrations. These strategies allow agencies to frame surveillance in favorable terms, creating what University of Washington law professor Ryan Calo calls an anchoring effect: policymakers "fixate on whatever instantiation of technology" proponents "happen to put in front of them."
Transparency entrenches automated systems
The deeper flaw with regulating AI through transparency is that it starts by conceding that state agencies may adopt automated systems. Then it invests in their continued use by building public bureaucracies around their disclosure. Far from serving an oversight function, this approach normalizes and entrenches automated decision-making as a routine part of government operations. The failure of California's transparency law highlights the challenges of AI for Government oversight, where good-faith disclosure rules can be easily sidestepped.
If California wants to regulate high-risk government uses of technology, the conversation cannot end at transparency. AB 302 was an experiment that failed, and now lawmakers need to consider stronger protections that go beyond voluntary reporting.
Why this matters for Government professionals
For professionals inside government agencies, the episode reveals that transparency mandates without enforcement mechanisms are unlikely to produce accurate inventories. The incentives to underreport or classify systems as low-risk are strong, especially when disclosure carries no penalties. Agency leaders and IT staff should expect that external audits and public records requests will eventually surface what self-reporting misses. Building internal practices that treat AI accountability as a core compliance function - not just a checkbox - will become increasingly important as legislative pressure grows. Understanding the limits of transparency-only approaches is a key part of the AI Learning Path for Policy Makers, which covers governance structures that can actually hold agencies accountable.
Your membership also unlocks:
