Stanford Law School researchers found that law professors preferred answers drafted by artificial intelligence over those written by their peers 75 percent of the time. The study suggests AI platforms can handle complex legal reasoning, offering a reliable, on-demand tutoring resource for students as educational institutions adapt to new technologies.
How the study was conducted
Professors from 14 U.S. law schools created 40 questions typical of first-year contracts students asking during faculty office hours. Faculty members wrote their own answers to these prompts. Researchers then had two AI platforms, Google's Gemini 2.5 Pro and NotebookLM, generate responses to the same questions.
The professors blindly evaluated the short answers head-to-head. The AI platforms matched the performance of the highest-rated professor in the study.
Quality and safety of AI responses
The results exceeded the research team's expectations. "We were frankly surprised by the magnitude of the results," lead researcher and Stanford law professor Julian Nyarko said. "These weren't just simple questions with obvious answers."
Safety metrics also favored the automated responses. Evaluators flagged less than 4 percent of the AI-generated answers as harmful to student learning, compared to 12 percent of the professor-written answers.
Integration into legal education
Law schools are actively debating how to incorporate these tools into their curricula. While some institutions mandate AI instruction for first-year students, others, like the University of California Berkeley School of Law, recently adopted policies restricting student use of AI in academic work. Previous research has already demonstrated that AI can pass the bar exam, earn top law school grades, and grade exams effectively.
The new findings point to practical AI for Education applications, allowing students to get reliable, on-demand answers instead of relying solely on sporadic emails or peer support. This shift could reduce faculty administrative burden while maintaining academic rigor.
Why this matters for educators
Educational professionals can view these findings as a signal to rethink student support structures. AI tutors can provide immediate, accurate feedback that complements traditional classroom instruction. "We find that, when evaluated by legal educators, AI tutors can offer high-quality, on-demand support that complements classroom instruction, and may broaden access to expert guidance," study co-author and Stanford researcher Alejandro Salinas said. Institutions may need to evaluate similar tutoring models across disciplines to manage faculty workload and standardize student support.
Your membership also unlocks:
