Artificial intelligence detection tools widely used by universities remain unreliable and can wrongly accuse students of misconduct, according to an analysis published in Nature. The findings raise concerns about the fairness of screening student work with these systems, especially as adoption grows in admissions and academic integrity cases.
The case of Lauren Jager, a chemistry student at Idaho State University, illustrates the problem. While applying for PhD programs, she learned that some universities planned to use AI detectors on personal statements. Jager had written her essays without AI assistance, but several online tools classified them as almost entirely AI-generated. Worried about rejection, she rewrote her statement in a less polished style until the detectors gave it a lower AI score. She was later accepted into a PhD program at the University of Utah.
Detection tools struggle with consistency
Researchers say the growing popularity of generative AI has complicated academic integrity assessments. Unlike traditional plagiarism software, which compares submissions against existing sources, AI-generated text often produces original wording that conventional methods miss. Universities have responded by adopting specialized AI detection tools such as GPTZero, Copyleaks, Turnitin, Grammarly and QuillBot. However, the reliability of these systems is a critical concern for those working in AI for Education.
A 2025 study reported that GPTZero correctly detected many AI-generated papers but falsely flagged around 16% of human-written essays as AI-generated. Another study found several leading detectors performed better on older AI models than on newer systems, while delivering inconsistent results on genuine human writing. Even historic texts like the U.S. Declaration of Independence were repeatedly identified as AI-generated by some detectors.
Bias against non-native English speakers
The technology may also disadvantage non-native English speakers. A Stanford University study found that more than half of English language essays written by Chinese students before the release of ChatGPT were incorrectly labeled as AI-generated. Essays written by U.S. students were classified much more accurately. Researchers attributed this disparity to differences in vocabulary and sentence complexity. The fact that detectors often misclassify human writing has direct implications for developers of AI for Writers software, as users may be unjustly penalized.
Legal and institutional pushback
Legal and institutional challenges have already emerged. A New York judge overturned disciplinary action against a student accused of using AI after the allegation relied on an AI detection app. In the United Kingdom, the higher education ombudsman warned universities about relying on AI detectors following student complaints, including one who argued the software was biased against their writing style. Several universities declined to adopt Turnitin's AI detection tool because of transparency concerns.
Shifting focus to process-based assessment
Rather than focusing solely on detecting AI, many researchers believe universities should redesign assessments to evaluate students' learning process instead of only the final written product. Some institutions and technology providers are shifting toward tools that record drafting and editing histories, allowing instructors to see how an assignment was developed.
Why this matters for education professionals
Educators and administrators should not rely on AI detectors as the sole arbiter of academic integrity. The evidence shows these tools carry significant false positive rates and bias risks that can damage students' academic standing. A more effective approach integrates process-based assessment and transparent policies that account for the limitations of detection technology. Faculty and instructional designers should advocate for assignment designs that capture student thinking over time, reducing the pressure to police writing through unreliable software.
Your membership also unlocks:
