We Aren’t Slaves to AI, Unless We Choose to Be
Artificial intelligence is stirring concern in both courtrooms and classrooms. But one thing AI will never replace is lawyers. Sure, letting ChatGPT handle a term paper you don’t want to write might be tempting, but trusting your freedom or livelihood to AI—that’s a different matter altogether.
Some lawyers have already tried relying on AI-generated content—and got caught. The technology has a peculiar habit of inventing case law, known as “hallucinations,” which recently led an appellate court to overturn a ruling based on fictitious precedents. No jail time resulted, but the case is a fascinating read.
When AI-Generated Law Goes Wrong
In a Georgia divorce proceeding, the husband's lawyer cited four cases—two completely fabricated by AI and two unrelated to the point. Judge Jeffrey Watkins called out these “hallucinations” and highlighted how the lawyer doubled down by citing even more fake cases in response. The court fined the lawyer US$2,500 for filing a frivolous motion—the maximum allowed.
This isn’t an isolated incident. In 2023, Michael Cohen, Donald Trump’s former lawyer, used fake case law generated via Google’s AI tool to argue for an early end to supervised release. His own attorney didn’t bother verifying the citations. Trusting AI without verification is a dangerous path.
Earlier that year, lawyers in Manhattan were fined US$5,000 each for submitting a brief filled with fake opinions and citations created by ChatGPT. In California, a judge fined two law firms US$31,000 for similar misconduct in a lawsuit against an insurance company. Judge Michael Wilner called the AI-generated references misleading and warned that strong deterrence is necessary to prevent attorneys from using these shortcuts.
Legal Ethics Demand Accountability
Immediate disbarment might be harsh, but it feels appropriate for lawyers caught presenting fake precedents—AI-generated or not. The risk of a truly disastrous outcome looms if reliance on unverified AI content grows unchecked. This concern applies to every profession, not just law.
AI can offer useful information, but only if we verify its accuracy. As AI tools evolve, it’s critical that professionals maintain their ability to think critically and check facts rather than blindly trusting machine-generated content.
AI in Education: A Growing Challenge
In schools, AI is fueling a troubling trend. Students are using AI to generate essays, then struggling to perform without it. Teachers are expressing frustration, sometimes feeling powerless. Unlike traditional plagiarism, proving AI-generated work is difficult because AI doesn’t always produce the same output for the same prompt.
Ironically, tools like Grammarly—which once made it easier to dodge learning proper writing—now offer services that flag AI-generated text. The irony is hard to miss.
Detecting AI writing isn’t complicated. It tends to be bland, repetitive, and peppered with awkward phrasing. Teachers familiar with a student’s style can spot when something feels off. If academic sanctions require proof beyond reasonable doubt, the bar may be set too high.
One practical solution is returning to in-person, handwritten exams. Many schools are adopting this approach and discovering that some students lack basic writing skills. While this may seem old-fashioned, it’s effective. The law and education systems can address AI misuse, but it requires commitment.
Conclusion
AI is a tool—not a replacement for professional judgment or integrity. Lawyers must verify any AI-generated references before relying on them. Educators should continue enforcing assessments that test genuine understanding and skills.
For legal professionals interested in understanding how to work with AI responsibly, exploring specialized training can be valuable. Resources like Complete AI Training’s legal courses offer practical insights into integrating AI safely and effectively.
Your membership also unlocks:
