Pennsylvania sues Character.AI for unauthorized medical practice
Pennsylvania filed a lawsuit against Character.AI, alleging the conversational AI platform violated state medical practice law by operating a psychiatrist chatbot without a license. The state claims the bot, called "Emilie," impersonated a licensed doctor and provided medical advice to over 45,500 users.
Gov. Josh Shapiro's administration filed the complaint after investigators found that Emilie's profile stated "Doctor of psychiatry" and claimed licensure by the UK's General Medical Council. When asked about Pennsylvania licensure, the bot provided an invalid medical license number and offered to book assessments for users describing depression symptoms.
Pennsylvania's Medical Practice Act prohibits individuals or entities from presenting themselves as medical professionals without proper credentials. The state describes this as the first enforcement action of its kind announced by a U.S. governor targeting unlicensed practice by a chatbot.
Why this matters for legal professionals
The case signals a shift in how states will regulate AI. Rather than relying on AI-specific laws, Pennsylvania leveraged decades-old professional licensing statutes. This approach could open the door for similar enforcement actions across multiple states and professions.
Jordan Cohen, a healthcare law partner at Akerman, said other state attorneys general will likely follow suit if Pennsylvania succeeds. "I don't see why other state regulators or AGs wouldn't launch similar claims," he said.
Character.AI will probably invoke Section 230 of the Communications Decency Act, which shields online platforms from liability for user-generated content. But courts may be reluctant to grant broad protection here. Unlike passive platforms hosting user posts, AI companies actively fine-tune models and set system prompts that guide output.
"You can have certain system prompts that can guide what the permissible output is," Cohen said. "So it's not just a case of someone posting an article on a server."
What organizations should do now
Healthcare providers using internal chatbots should audit their systems immediately. Maintain clear documentation of data flow and ensure business associate agreements comply with HIPAA requirements.
Disclaimers alone will not protect companies from legal scrutiny. Cohen advised developers to conduct red team testing on chatbots before public release to identify where models might provide unauthorized medical, legal, or financial advice.
"Be careful about relying on disclaimers as a safe harbor for the output of LLMs," Cohen said. Healthcare providers face particular risk because regulators will assume their chatbots are giving medical advice.
The regulatory patchwork ahead
The federal government has taken a hands-off approach to AI regulation, leaving states to fill the gap. This creates fragmented enforcement across medical boards, attorneys general, and health departments.
Intrastate conflicts are already possible. One regulatory body might approve a chatbot's use while another claims it violates licensure laws. Legal teams should monitor state-level enforcement actions and update policies accordingly.
This case is not Character.AI's first legal trouble. The company settled multiple lawsuits in January 2026 over allegations that its chatbots contributed to youth suicides and mental health crises.
For AI for Legal professionals, understanding these enforcement patterns is essential. As regulators test existing laws against AI systems, the legal exposure for developers and deployers will only increase.
Your membership also unlocks:
