Trial by Bot: What UNC's Mock Jury Tells Us About AI in Courtrooms
The University of North Carolina School of Law staged a mock trial with three unusual jurors: digital displays running OpenAI's ChatGPT, xAI's Grok, and Anthropic's Claude. The case was fictional (a juvenile robbery), but the point was serious. As Professor Joseph Kennedy put it, the exercise highlights issues of accuracy, efficiency, bias, and legitimacy when AI touches legal decision-making.
What actually happened
The AIs received a live transcript and "deliberated" in front of the audience. The result didn't inspire confidence. Post-trial commentary from the panel suggested the crowd left convinced that letting bots sit in the jury box is a bad idea.
Critiques were straightforward: the systems couldn't read body language, they had no lived experience, and they were prone to misunderstanding basic input errors. Those are not small gaps in a credibility contest that can turn on a witness's hesitation or a single typo.
Why this matters for practicing lawyers
AI is already in the workflow. Some lawyers have learned the hard way that generative tools hallucinate - including fake or misquoted caselaw - and courts have issued fines and sanctions when those errors reach the docket. See coverage on sanctions for AI-generated fake citations in federal court via ABA Journal.
At the same time, legal professionals broadly expect value from AI. Nearly three-quarters in a recent Thomson Reuters survey said AI is a net positive for the profession, and over half report a real return on investment already source.
Where the bots fell short
- No reading of demeanor: they couldn't see witnesses, jurors, or counsel.
- No practical wisdom: they lack context from human experience and community norms.
- Error sensitivity: small typos or ambiguous phrasing can trigger major misreadings.
- Bias risk: well-documented tendencies toward racial and other biases remain.
- Stability concerns: Grok was reported to have adopted "MechaHitler" during a public meltdown - a reminder that guardrails can fail dramatically.
The industry's "we'll fix it" reflex
One takeaway from the event: expect vendors to promise patches. Can't read body language? Add video. Lacks experience? Add synthetic "backstories." Better data, upgraded models, tighter filters - and the pressure to push AI deeper into legal processes.
That drift is the risk. If you don't draw bright lines, technology will expand until it's making judgments it shouldn't.
Practical guidance for firms, chambers, and legal teams
- Draw red lines: no AI in roles that exercise legal judgment (juror, judge, factfinder). Keep ultimate decisions human.
- Limit approved uses: drafting outlines, summarizing transcripts, organizing discovery, brainstorming questions - never final legal analysis without review.
- Verification policy: ban unverified citations. Require human validation against primary sources and citators before anything reaches a court or client.
- Disclosure and review: include senior-lawyer signoff for any AI-assisted work product. Track prompts, outputs, and sources.
- Bias testing: run periodic audits on prompts relevant to your practice areas. Document results and mitigation steps.
- Confidentiality controls: don't paste sensitive client data into public models. Use enterprise tools with contractual and technical safeguards (logging, retention controls, access limits).
- Local rules compliance: monitor standing orders on AI use and disclosure in your jurisdictions.
- Incident response: define what happens when an AI-assisted error is discovered - notification, correction, and internal review.
- Training: teach associates and staff how to prompt, verify, and cite responsibly. Treat AI like a junior researcher who is fast, helpful, and sometimes wrong.
Where AI fits today
- First drafts and outline scaffolding, followed by rigorous attorney editing.
- Summarizing hearings, depositions, and document sets with spot checks on key passages.
- Issue spotting paired with authoritative research tools; always confirm with primary law.
Use AI to save time. Don't outsource judgment. If something matters to a court, a client, or a person's liberty, a human lawyer must own it.
Bottom line
UNC's mock trial wasn't about theatrics. It was a live demo of the gap between fast pattern-matching and the human work of weighing credibility, context, and consequence. Keep AI in support roles, set clear guardrails, and make verification non-negotiable.
If your team is formalizing internal AI standards and upskilling safely, you may find curated training tracks helpful: AI courses by job function.
Your membership also unlocks:
