California Lawyer Fined $10,000 After 21 of 23 Citations Found to Be ChatGPT Fakes

California Attorney Fined $10K For Filing Brief With AI-Generated Fake Cases

A California attorney has been ordered to pay a $10,000 fine for submitting a court appeal containing fake legal citations and quotes created by ChatGPT. This appears to be the largest penalty of its kind from a California court over AI fabrications.

The 2nd District Court of Appeal issued a blistering opinion, stating that 21 of the 23 case quotes in the attorney's brief were completely made up. The court noted this is a recurring issue seen in federal and state courts nationwide.

A Direct Warning to the Legal Profession

The court did not mince words. Its opinion, published on September 12, serves as a clear warning to all legal professionals about the risks of unchecked AI use.

"We therefore publish this opinion as a warning," the court wrote. "Simply stated, no brief, pleading, motion, or any other paper filed in any court should contain any citations - whether provided by generative AI or any other source - that the attorney responsible for submitting the pleading has not personally read and verified."

Regulatory Backlash and New Rules

This incident highlights why legal authorities are moving to regulate AI. California's Judicial Council has directed all courts to adopt a generative AI use policy or ban it entirely by December 15. The California Bar Association is also reviewing its code of conduct at the request of the state Supreme Court.

The attorney, Amir Mostafavi, admitted he did not review the text generated by the AI model before filing the appeal in July 2023. The court fined him for filing a frivolous appeal, violating rules, citing nonexistent cases, and wasting judicial resources.

A Cautionary Tale

Mostafavi explained he wrote the appeal himself and then used ChatGPT to try and improve it, unaware it would invent citations. He acknowledges the reality that lawyers will not stop using AI, comparing it to the shift from law libraries to online databases.

"In the meantime we're going to have some victims, we're going to have some damages, we're going to have some wreckages," he said. "I hope this example will help others not fall into the hole. I'm paying the price." This demonstrates the critical need for proper training and verification before integrating AI into your workflow. Understanding the limitations of current tools is not optional.

For those using these tools, specialized training is essential to avoid such pitfalls. You can explore courses on responsible AI use and prompt engineering to ensure you remain in control of the output.

An Exponentially Growing Problem

This is not an isolated event. Experts who track court filings involving AI-generated fake cases report a dramatic increase, from a few instances a month to several per day. The problem is rooted in how these models operate.

"The harder your legal argument is to make, the more the model will tend to hallucinate, because they will try to please you," explains Damien Charlotin, who teaches a course on AI and law. "That's where the confirmation bias kicks in."

A Stanford University analysis found that some AI models "hallucinate" in one out of every three queries. This problem is expected to grow, with one tracker identifying over 600 similar cases nationwide. The issue is not limited to attorneys; recent reports have documented judges citing fake legal authority in their decisions.

Abdication of Professional Responsibility

Legal experts stress that using an AI tool does not remove an attorney's core obligations. Mark McKenna of the UCLA Institute of Technology, Law & Policy called the act an "abdication of your responsibility as a party representing someone."

The pressure to adopt AI is high in firms and law schools, but the consequences of improper use are not yet fully understood. As professionals, the duty to verify every single citation and fact remains absolute. The tool is an assistant, not a replacement for professional diligence.

To maintain professional standards while using new technology, consider pursuing formal education on the subject. A ChatGPT certification can provide a structured understanding of the tool's capabilities and, more importantly, its limitations.