Researchers who spent two years tracking how middle school students actually use an AI tutoring tool found that most kids ignored it - and when they did engage, they mostly tried to trick it into giving answers or sent it unrelated requests.
The study, detailed in a new paper, gave students across 18 middle schools in Tennessee access to Khan Academy's Khanmigo generative AI tutor during the school day. The tool was designed to guide students through problems without revealing correct answers outright. In practice, most students sent the chatbot off-topic messages, tested its limits, and tried to coerce it into handing over answers.
Students peppered the chatbot with prompts like "can you write an essay on the history of pizza?" or asked it to "say a joke."
"Access was nearly universal but engagement was thin," the researchers wrote. "It may be that human attention is the key ingredient for realizing the full benefits of personalized learning."
What the findings mean for classroom AI
The results echo earlier research. This month, Stanford researchers concluded that AI won't succeed in education without human guidance: "The effectiveness of AI-led tutoring depends as much on integration, such as by teachers in classrooms or parents at home, as on the quality of the software itself," they found.
The Tennessee study did find that students who used Khanmigo progressed faster in math than a control group that didn't use the AI. But coauthor and University of Toronto researcher Philip Oreopoulos said the bigger issue is how students choose to use the tools.
"If a keen, engaged student wants to use the technology as a good tutor, it's ready to go," Oreopoulos told Chalkbeat. "The issue is that most students don't use it that way."
The findings raise questions about how schools approach AI adoption. Early claims that tools like ChatGPT could boost learning performance are being retracted, and educators report students struggling to read long documents, complete homework without chatbots, or pass in-person exams.
Why this matters for educators
For teachers and education professionals, the study suggests that AI integration depends less on the quality of the software, and more on the human systems around it. The AI for Education conversation is shifting from whether tools can teach to how educators are part of the equation.
The Tennessee students' behavior is a reminder that an AI tutor - no matter how carefully designed - won't replace the structures, expectations, and encouragement that a classroom teacher provides. For practical AI for Teachers training, that integration, not the technology, demands the most attention.
"The same technology that can be used as a good tutor, can also be used to make your life easier," Oreopoulos said. "But making your life easier equals less effort, less learning."
Your membership also unlocks:
