The Supreme Court has set aside orders from the NCLT and NCLAT after discovering the tribunals relied on fake, AI-generated judgments, and directed the Bar Council of India to form a committee examining how advocates submit hallucinated legal precedents to courts. The intervention establishes that accountability for AI errors in legal work stays squarely with the lawyer or judge, regardless of the tool used.
The top court's July 2026 order comes as courts and law firms increasingly use AI for research, drafting, and administrative tasks. While efficiency gains are real, the incident highlights the consequences when AI output goes unverified. "A blind reliance on AI-generated output for legal work is problematic at various levels," said Advocate Arya Tripathy, partner at Cyril Amarchand Mangaldas. "Not only does it impact the client's rights, but it also dilutes the trust and privilege reposed and showcases poor professional ethics."
The message from the bench and legal experts alike: AI should remain an assistive tool, never a substitute for human judgment. Justice Wasim Sadiq Nargal of the Jammu and Kashmir and Ladakh High Court separately warned judicial officers in June against relying on unverified AI legal research, after incorrect citations appeared in a trial court order. That same month, the Allahabad High Court asked Uttar Pradesh police to implement speech-to-text AI for witness statements, while the Gujarat High Court introduced a policy in April that allows controlled AI use for translation, grammar checks, and cause list management-requiring human oversight at every stage.
Where the lines are drawn
Overreliance on AI is already producing a steady stream of fake citations and flawed legal reasoning. Advocate Akshat Pande, managing partner at Alpha Partners, noted that non-lawyers using AI to bypass professional legal advice may cause more harm than lawyers misusing the technology. Mandatory disclosure rules could also create a stigma, he said, where courts view AI-assisted drafts with suspicion. "It will be better if courts also use technology to catch incorrect data, drafting or fake case quotations by AI and reject the drafts on that basis."
The Supreme Court's draft Regulations for Use of Artificial Intelligence in Courts, 2026, already mandate disclosure of generative AI content and verification through a designated mechanism. Advocate Mathuvanthy Mathavan of Poovayya & Co said that the focus must shift to "proper regulation and supervision," including clear disclosure requirements and disciplinary consequences for filing fabricated material. Training, too, is emerging as a priority. As Tripathy put it, "To be unable to spot the hallucinations and bias inherent in these instances simply may indicate the need for lawyers to be trained as 'AI-enabled lawyers'."
Regulatory step or overreach?
The BCI's new mandate puts it at the center of defining what verification looks like in practice. Advocate Anshul Verma, partner at SKV Law Offices, sees this as a clear opening to create binding guidance with real disciplinary teeth. Others remain skeptical. Pande argued the BCI "generally have a hard time understanding most modern facets of the legal profession; with AI, God knows what they will end up doing," and suggested the regulator might do best to stay away.
Complete negation of AI is not on the table. The judgment "embraces AI as a useful tool while firmly insisting on human control," Mathavan said. For legal professionals trying to understand how to use these tools ethically, structured training programs can bridge the gap between assistance and overreliance. AI for Legal Professionals Courses focus on precisely that boundary-how to integrate AI into legal workflows without surrendering professional responsibility.
Why this matters for legal professionals
The Supreme Court has signaled that AI-generated content will be treated no differently from any other material submitted to a court: the advocate or judge who presents it owns the consequences. Verification is not optional, and the risk of disciplinary action for submitting fake precedents is now explicit. For practicing lawyers, the immediate takeaway is that every AI-assisted draft, citation, and summary must be independently checked against primary sources before it goes anywhere near a judicial forum. Courts are adopting AI at their own pace, but the expectation of human accountability remains absolute.
Your membership also unlocks:
