When Using Artificial Intelligence Goes Wrong: Judge Slams Lawyers for Legal Bungle
A recent case in the Gauteng High Court, Johannesburg, exposed the risks of relying on artificial intelligence (AI) for legal research. Acting Judge DJ Smit discovered that two legal citations submitted by Northbound Processing in an urgent application did not exist. When questioned, counsel admitted these were AI "hallucinations" generated by a legal research tool.
Northbound Processing sought to compel the South African Diamond and Precious Metals Regulator to issue a refining licence. While drafting his judgment, Judge Smit noticed that two cases cited in the heads of argument were fictitious. The judge asked the legal team for clarification.
The counsel responsible explained the pressure of the urgent application and said they used an online subscription tool called “Legal Genius,” which claims to be exclusively trained on South African legal judgments and legislation. Despite the non-existent citations appearing in the written arguments, the senior advocate confirmed he did not rely on these cases during oral submissions.
The junior lawyer accepted full responsibility for the errors and stressed there was no intention to mislead the court. The senior advocate apologised on behalf of the team, explaining he relied on a competent legal team and only performed a basic check on the citations before filing. He acknowledged that he did not have time for a thorough verification.
Judge’s Response and Implications
Judge Smit referenced a recent English King’s Bench Division ruling warning about the dangers of AI in legal research. The English judge highlighted that AI tools can produce entirely inaccurate information and even cite sources that do not exist. This raises serious concerns for the administration of justice and public trust in the legal system.
While Judge Smit accepted the apologies, he emphasised that negligence in this context could have severe consequences. The conduct of the lawyers involved has been referred to the Legal Practice Council for further investigation.
Lessons for Legal Practitioners
- Verify all AI-generated legal citations. AI tools can produce false or fabricated case law, which can undermine your arguments and credibility.
- Don't rely solely on AI for urgent or critical filings. Time pressure should not compromise the accuracy of legal research.
- Ensure senior counsel thoroughly reviews all references. Even a quick “sense-check” may not be sufficient to catch errors.
- Use AI as a supplementary tool, not a replacement for traditional research methods. Always cross-check AI outputs against trusted legal databases.
Legal professionals using AI tools for research should stay informed about the limitations and risks of such technology. For those interested in improving their skills with AI in legal practice, relevant courses and certifications can provide practical guidance and best practices. Explore options for AI courses for legal professionals to learn how to integrate AI responsibly.
Your membership also unlocks:
