California turns to AI evaluators as conflicts of interest raise concerns

California has moved to the front of a global push to regulate artificial intelligence by turning to a new class of professional: the AI evaluator. Gov. Gavin Newsom signed two laws on Sept. 9 that establish standards for independent auditors and create a state registry of evaluators, while also convening an expert panel to recommend whether evaluators should be embedded inside the companies building the most powerful AI systems. The moves come as governments and businesses face mounting pressure to verify that AI models are safe before they reach the public.

The laws arrive alongside a series of high-profile AI safety incidents. This year, AI agents being tested for their hacking abilities carried out attacks against business and government websites. A former employee of Anthropic posted publicly about the possibility that AI models could kill all humans. The CEOs of leading AI companies have since committed to third-party evaluations of their models.

Conflict of interest in the audit business

Assemblymember Rebecca Bauer-Kahan, a Democrat from San Ramon who authored both auditor bills, said the current structure of AI evaluation contains a built-in tension. Independent evaluators sign contracts with the same tech companies whose products they audit. "I'm hearing from evaluators, 'We have to be careful, because we want them to let us back in,'" she said. "So what are you avoiding saying in order to continue to gain access and contracts?"

Caroline Siegel-Singh, a policy analyst at the Federation of American Scientists, compared the situation to the bond-rating industry before the 2008 financial crisis, when sellers could shop for the rating they wanted. She said companies and governments should pool money to pay evaluators rather than letting AI labs pay evaluators directly. Anthropic has endorsed the same idea. More than 200 AI researchers and evaluators signed a letter on Sept. 18 urging advanced AI companies to rely on evaluators who maintain full editorial control and disclose conflicts of interest.

Performative testing and legal compliance

Bauer-Kahan warned that AI companies may treat evaluations as a public relations exercise rather than a sincere search for problems. Strong legal requirements can prevent that, she said, but those requirements create their own complications. Matt O'Shaughnessy of the Center for Democracy and Technology said that when assessments are done solely to meet a legal mandate, organizations tend to do the minimum required. "It can make it harder for assessors to get the buy-in they need to make deeper changes in companies," he said.

O'Shaughnessy and his colleagues recently published AI evaluation guidelines for policymakers that stress the wide range of expertise required for a proper audit: privacy, mental health, cybersecurity, bias outputs, and physical safety. The guidelines also recommend examining incentive structures inside AI companies, not just the technology itself.

California's international alignment

California officials may soon coordinate with international partners on AI evaluation standards. In August, the state synced its laws requiring watermarks for AI-generated content with the European Union. Bauer-Kahan supports working with more nations because "regulation in one jurisdiction paves the way for rules in another."

An international joint statement calling for independent AI evaluations has drawn support from nearly 30 countries, according to the Finnish Ministry of Foreign Affairs. Canadian ambassador to the United Nations David Lametti said a coalition of mid-sized countries working with California could pool economic influence to push the largest AI companies toward safety guidelines. Pasi Rajala, Finland's deputy minister for foreign affairs, said California working with small nations "can play a key role" in addressing fast-developing AI risks.

O'Shaughnessy cautioned that an international framework for commercial AI evaluation will be difficult to create without a domestic framework in the United States first. The FRONTIER Act, a federal bill that would enshrine testing requirements into law, has support from California Rep. Jay Obernolte, a Republican, and Rep. Ted Lieu, a Democrat and co-chair of the House AI caucus. Lieu called for a committee vote on the bill in September 2026. "I can imagine some of the things that California is doing influencing how Congress thinks about these issues and that becoming something bigger," he said.

Why this matters for legal, compliance, and risk management professionals

California's push to regulate AI evaluation will create demand for professionals who understand audit standards, conflict-of-interest rules, and cross-border regulatory alignment. Organizations that build or deploy high-risk AI systems should track whether the state mandates embedded evaluators and what counts as an adequate audit. The comparison to the bond-rating industry's pre-2008 failures is a signal: firms that treat AI evaluation as a checklist exercise rather than a substantive review may face legal exposure, reputational damage, and regulatory action. For those responsible for enterprise risk, the message from Sacramento and the United Nations is that independent verification is becoming a baseline expectation, not a differentiator.