California bill would ban AI therapists as chatbot use surges
California lawmakers are pushing to stop AI from being used as a full-on therapist, as interest in chatbot-based mental healthcare has spiked in recent years. Under Senate Bill 903, companies would be banned from advertising chatbots as therapy, and AI systems could not make therapeutic decisions without review from a licensed professional. The bill would also require patients' permission for health providers to use AI to record therapy sessions or triage care.
"AI has many upsides, and the possibility to improve the lives of Californians across the state, but only if we ensure it is used responsibly," state Senator Steve Padilla of San Diego, the bill's author, said earlier this year. "AI algorithms are not fit to take over the job of human therapists, who have skills and training that AI is incapable of replicating."
"We must act and place guardrails that prevent AI algorithms from being deployed in a way that is potentially harmful to patients," Padilla added.
What the law would change
SB 9 targets three core areas. First, it bans AI products from explicitly offering themselves as therapy. Second, any therapeutic decision made by AI must be reviewed by a licensed professional. Third, patients must opt in before a health provider can use AI to record or triage sessions.
Supporters say the bill protects vulnerable people from unregulated algorithms. The National Union of Healthcare Workers recently filed a complaint with California regulators alleging that Kaiser Permanente used an automated "e-visit" system without clinician oversight to make care recommendations for patients with anxiety or depression. Kaiser said it "does not use AI to diagnose patients, make clinical decisions, or determine medical necessity," and argued the union mischaracterized its process.
The data behind the debate
A 2025 study from RAND, Brown University School of Public Health, and Harvard University found that about one in eight adolescents and young adults now use chatbots for mental health care. Another study from the American Psychological Association found 77 percent of psychologists have spoken with patients who used AI for support or treatment.
Industry critics argue the bill goes too far. "At a time when every county in California faces a shortage of behavioral healthcare workers, SB 903 still puts a clinician bottleneck in front of the intake and screening tools that help patients reach care faster," Robert Boykin, California executive director of the technology industry group TechNet, told state media this week.
OpenAI disclosed that roughly 1.2 million messages per week sent to ChatGPT contain "explicit indicators of potential suicidal planning or intent." The makers of several popular AI applications have faced lawsuits alleging their chatbots played a role in youth suicides.
Health experts say the trend reflects access problems more than any specific technology flaw. "Given long waiting lists and difficulty accessing mental health care and therapy in many countries, it's perhaps not a surprise that increasing numbers are turning to generative AI for support with their emotional wellbeing," Hamilton Morrin, a clinical fellow at King's College London, told Harvard Business Review.
Where the science stands
The research foundation for AI therapy remains thin. "There are few standardized benchmarks for evaluating mental health advice offered by AI chatbots, and there is limited transparency about the datasets that are used to train these large language models," said Jonathan Cantor, a senior policy researcher at RAND.
The American Psychological Association study found no consensus among clinicians about whether AI chatbots are safe for therapy. For healthcare professionals, the core tension is between access and quality. The shortage of therapists drives demand, but regulators are not convinced that AI can handle the nuance that those pathologies require.
For professionals interested in how this sector is evolving, the intersection of AI for Healthcare and the tools already in use, such as those labeled AI for Customer Support, shows how quickly AI applications are moving from simple triage into clinical roles.
Why this matters for healthcare jobs
For nurses, doctors, therapists, and insurers, SB 9 sets a precedent. If it passes, any company using AI in a clinical role in California must keep a licensed professional in the loop, even for intake and triage - tools that currently automate those tasks. The bill could reshape who can delegate decisions to software and increase malpractice exposure for providers that let AI operate without oversight. Healthcare organizations should audit their current AI-driven screening tools now, and ensure any patient-facing chatbot has a documented review path.
If you are experiencing distress, contact the Samaritans in confidence at 116 123 (UK and ROI) or visit their website. In the US, call or text 988 for the Suicide and Crisis Lifeline, available 24/7. For other countries, befrienders.org lists local helplines.
Your membership also unlocks:
