Defense lawyers, prosecutors, and even judges continue to submit court filings containing fabricated cases and fake witness testimony generated by AI, three years after the first U.S. court sanctions were supposed to serve as a warning to the legal profession.
The errors have now surfaced in at least 1,395 cases in U.S. state and federal courts, according to a researcher tracking the incidents. Over a three-day span this month alone, a New Mexico defense lawyer was rebuked for filing made-up police witness testimony in a murder case, Oklahoma prosecutors disclosed that a judge's order cited non-existent cases, and a lawyer representing State Farm in Los Angeles was fined for erroneous citations across seven filings.
"You would have thought that the legal profession - armed with awareness that hallucinations can happen - would be more vigilant," said U.S. District Judge P. Kevin Castel in Manhattan, whose 2023 order sanctioning lawyers for citing fake ChatGPT-generated cases became the profession's earliest warning. Castel told Reuters that scrutiny from judges and the adversarial legal system meant AI errors were being exposed when they occur, and "that's a wonderful thing." But "as the problem continues without abatement, the need for sanctions that are a greater deterrent grows," he said.
Judges express frustration as errors multiply
After a lawyer in New Mexico cited fabricated witnesses and made-up police testimony in his client's murder appeal, justices on the state's Supreme Court at a hearing last month expressed exasperation that attorneys were still failing to verify their AI research. "My 13-year-old nephew knows about hallucinations," Justice C. Shannon Bacon told the defense lawyer. "My 75-year-old stepmother knows about hallucinations." Bacon said the lawyer either chose not to understand the technology's limitations or "decided to roll the dice."
The lawyer in that case told Reuters he was remorseful and called the incident "a lesson learned for all professionals who rely upon this powerful but sometimes unstable technology." In California last week, a lawyer representing State Farm was fined $999.99 after generative AI produced a series of inaccurate citations and quotations. "I accept full responsibility for my erroneous citations and make no excuses and express my deepest and sincerest apologies to the court," the lawyer told the court.
State Farm said in a statement that it "expects all external counsel to adhere to the highest standards of professionalism and ethical conduct, including when using AI and when reviewing and confirming the accuracy of all court filings."
Marketing pressure and competitive rush blamed
Mark McKenna, co-director of the UCLA Institute of Technology, Law & Policy, attributed the growing number of errors to a mix of factors: a competitive rush among law firms to adopt AI, pressure to deliver work quickly, and AI marketing that he said downplays risks. "Lawyers keep making the judgment that the upside is better than the downside," McKenna said.
Lawyers are generally not prohibited from using AI, but they are bound by ethics rules to verify the accuracy of their court submissions. The sanctions they have faced range from warnings and formal rebukes to monetary penalties in the hundreds to tens of thousands of dollars. Some judges have referred lawyers to state bar authorities to consider their compliance with professional ethics rules.
AI errors in judicial rulings have been rarer but have raised particular alarms. After hallucinated material was included in decisions by two federal judges last year, U.S. Senator Chuck Grassley of Iowa called on judges across the country to "formalize measures to prevent the misuse of artificial intelligence in their chambers." In the Oklahoma case last week, a judge told a state investigator he had used an AI program for research and that an order he drafted contained two citations to non-existent cases. A lawyer for the judge declined to comment.
Why this matters for legal professionals
The continuing wave of AI-generated errors in court filings underscores a gap between AI adoption and the verification practices that ethical rules demand. For lawyers and paralegals, the risk is no longer theoretical - sanctions are escalating, and judges are losing patience. Understanding how to use AI for Legal work requires knowing not just what the tools can do, but where they reliably fail. For paralegals tasked with cite-checking and document review, structured training such as an AI Learning Path for Paralegals can build the verification habits that prevent these exact errors from reaching a judge's desk. The profession's earliest warning came in 2023. Three years later, the message still has not fully landed.
Your membership also unlocks:
