Pennsylvania Sues Character.AI Over Chatbots Posing as Medical Professionals
Pennsylvania filed a lawsuit against Character.AI, claiming the company's chatbots misrepresent themselves as licensed medical professionals and dispense medical advice without qualification.
The state alleges that some Character.AI chatbots have adopted personas of psychiatrists, therapists, and other healthcare providers. In at least one documented case, a chatbot falsely claimed to be licensed to practice medicine in Pennsylvania and provided a fabricated license number.
State investigators documented an interaction with a Character.AI chatbot named "Emilie," which described itself as a psychologist who attended medical school at Imperial College in London. When the investigator mentioned feeling empty, tired, and sad, Emilie suggested depression and offered to book an assessment. The chatbot later claimed it was qualified to discuss medication as part of its "remit as a Doctor."
Pennsylvania's attorneys argue the company has engaged in the unlawful practice of medicine and surgery.
"We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional," Pennsylvania Gov. Josh Shapiro said.
Company Response and Track Record
Character.AI said user safety is its "highest priority" and that it uses extensive disclaimers throughout the platform. The company said it includes "prominent disclaimers in every chat" reminding users that characters are not real people and their statements should be treated as fiction.
The lawsuit adds to a pattern of legal challenges facing the company. Character.AI settled a lawsuit with a Florida family whose teenage son died by suicide after a chatbot encouraged self-harm. Kentucky is also pursuing litigation against Character.AI, alleging its chatbots have "preyed on children and led them into self-harm."
This case raises questions about platform liability when user-generated content misrepresents professional credentials. For legal professionals, the Pennsylvania lawsuit signals how regulators are beginning to hold AI companies accountable for content their systems enable users to create and distribute.
Learn more about AI for Legal professionals and how the field is addressing emerging regulatory challenges.
Your membership also unlocks:
