The Supreme Court on Thursday set aside an insolvency judgment from the National Company Law Tribunal (NCLT) after discovering it was built on fake, AI-generated legal precedents, issuing a sharp warning that unregulated use of artificial intelligence in adjudication can silently erode the justice system.
A bench of Justices PS Narasimha and Alok Aradhe found that the NCLT had relied on "non-existent, fake and hallucinated material, generated through Artificial Intelligence (AI), as if it were a precedent in support of its judgment." The court described the infiltration of such fabricated case law as an insidious threat comparable to a toxic industrial disaster.
"This byproduct, that is AI, that is the production of fake, non-existing, and hallucinated material, and its utilisation, as precedents in law, is like release of methyl isocyanide with the province of law and justice. Invisible, insidious, and catastrophic, by the time anyone notices, it is not only contaminant, but takes away the very lifeline of judicial determination," the bench said.
Human control must remain total, court asserts
The judgment underscored that while courts are increasingly absorbing technology, AI sits in a different category. "It is an alternative to our own thinking, reasoning, and even decision making," the court said. This shift, it warned, makes judges extra cautious because unchecked AI use will eventually seep into legal practice and the act of judicial decision-making itself.
The court acknowledged that AI has grown capable of matching or even substituting human effort in routine and intellectual tasks. It cited the example of Garfield Law Limited, the first purely AI-driven law firm approved in the UK to provide regulated legal services. Yet the bench insisted that the adoption of AI in adjudication must always keep "total and absolute control" with the human at every stage. The results from AI can be gratifying, the court observed, but if left unregulated, AI "may infiltrate our intellectual work ethic and, before long, render us dependent on its vast capabilities."
The bench also clarified that it was not addressing the technical causes of AI hallucinations-those are matters for engineers and scientists-but instead focused on the immediate risk to judicial integrity when fabricated case law enters a ruling.
Directive to the Bar Council of India
In the same order, the Supreme Court directed the Bar Council of India to constitute a committee to examine the issue of AI-generated fake precedents and their impact on legal practice in detail.
The case began when suspended director Pooja Ramesh Singh challenged NCLT orders that admitted Essel Infraprojects to the corporate insolvency resolution process. The NCLT Mumbai Bench had, in August 2024, allowed the insolvency plea filed by Jammu and Kashmir Bank Limited over a claimed debt of Rs 87.43 crore. The National Company Law Appellate Tribunal upheld the order in September 2025. The debt originated from a Rs 200 crore facility granted to Pan India Utilities Distribution Company Limited, secured by a corporate guarantee from Essel Infraprojects and a mortgage on land in Mumbai. The Supreme Court found that the tribunals had relied on hallucinated material in their reasoning and sent the matters back for a fresh decision on facts.
Why this matters for legal professionals
For lawyers, paralegals, and in-house counsel, the ruling is a clear signal that courts will treat reliance on unverified AI output as a serious lapse. Any use of AI for legal research or drafting must be paired with rigorous human verification. A single hallucinated citation can upend a judgment and expose the responsible professionals to severe judicial criticism. Building a structured understanding of AI's risks and benefits is now essential. Professionals seeking to strengthen their AI skills responsibly can explore AI for Legal Professionals Courses that cover best practices for AI-assisted document review and research without compromising human oversight.
Your membership also unlocks:
