Instruction Partners maps 20 AI learning tools, finds mixed results
A new large-scale effort to map and evaluate AI-powered learning tools finds some areas where the technology has promise for propelling student achievement-but also warns that other products are "isolating," "boring," and could leave children's learning "significantly damaged." The analysis, from education consulting nonprofit Instruction Partners, includes 16 deep dives into individual AI products, each presenting user feedback, instructional capability reviews, and available research on effectiveness.
The project represents one of the first attempts to weigh the various types of AI services on offer and their implications for teaching and learning. It arrives as educators and experts have called for more systematic assessments of the sprawling, and ever-growing, market of AI-enabled learning tools.
"Not all AI is created equal," said Emily Freitag, co-founder and CEO of Instruction Partners. "We need to have nuanced decisionmaking about where it can offer real value and where it creates real risk."
Three categories of tools, three different risk profiles
The Instruction Partners team examined three kinds of AI tools: general-purpose chatbots like ChatGPT, Claude, and Gemini; education-focused tools with many applications, like MagicSchool or Google Classroom; and instructional tools with a discrete purpose, like Amira Learning, which only offers reading practice. Each product profile explains what students do, what the AI does, what the teacher does, and what role school or district leaders can play.
The profiles include "wins" and "limitations" drawn from interviews with educator users, Instruction Partners' own analyses, and a list of available studies evaluating effectiveness. Each also includes a section "from the developer," with statements from or video interviews with product designers.
The hope, Freitag said, is that the analysis "cuts through the marketing speak and helps [leaders] make sense of the product."
General-purpose chatbots carry the biggest risks
Across the board, teachers reported that some tools seemed more of a hindrance than a help to learning. "We definitely saw the biggest risks from the general-purpose chatbots," Freitag said. "It's also the way AI is showing up in search."
Classroom observations revealed a pattern: kids were asked questions by an AI tool, went to Google, asked Google the same question, then put the answer back into the AI tool. Teachers commonly reported students using chatbots to "get out of work that requires effortful thinking," the analysis reads. "Learning anything requires effort, and products that intentionally or unintentionally reduce students' effortful thinking will significantly compromise learning."
For educators seeking structured guidance on integrating these technologies thoughtfully, the AI Learning Path for Teachers provides a framework for evaluating when and how AI fits into classroom practice.
Purpose-built instructional tools show more promise
The review team was more bullish on tools designed for a specific instructional use-those with a "clear point of view about the parts of the lesson that teachers are best positioned to deliver and which parts AI can support," the report reads. In reading and English/language arts, this looked like tools that could provide actionable feedback on student writing faster than an individual teacher could on their own, or diagnose which phonics skills students had mastered and which they still struggled with.
In math, the team praised tools that could identify students' misconceptions, or collect and analyze many students' explanations of their mathematical reasoning. Still, purpose-built AI tools had shortcomings. The team cited concerns about tools that rewrote text at a lower reading level, which could potentially decrease rigor.
"We did not see anything that is ready to do the pedagogical job independently," Freitag said. "We also heard from students, especially, that they value the relationships with their teachers, and they want to learn from their teachers."
At the same time, leaders shouldn't avoid AI entirely, she said. Educators should pick the approach best suited to the job at hand. "We do not think we should be precious about instructional methods when they are not producing the outcomes we want for kids."
Evaluation systems lag behind adoption
Organizations like Instruction Partners and EdReports are wrestling with an ongoing tension: because AI tools are still relatively new, there's little consensus on how to evaluate their instructional quality. But companies are marketing these products now, and districts are adopting them-or have already done so. Usable evaluation systems are a present need.
On its website, Instruction Partners stakes a middle ground, underscoring that its product analyses aren't ratings. They're not evaluating tools against a predetermined list of criteria. Most potential "instructional wins" also come with caveats, meaning district leaders must weigh the pros and cons.
For example, on tools' ability to provide feedback on student writing, Instruction Partners outlines upsides: it can boost the amount of feedback students get overall, a shift that could improve their writing. But the profiles also list limitations. AI-generated feedback can be inconsistent, and students said some comments felt impersonal. Some research has shown that ChatGPT and Meta's Llama change the kind and quality of writing feedback they give when they know a student's demographic background.
"Eventually, the field needs raters and regulators," Freitag said. "Right now, the field needs sense-makers."
Critics say privacy gaps leave the picture incomplete
Instruction Partners' project is intentionally limited in scope, focusing solely on tools' instructional capabilities-not other concerns such as data privacy. Some observers say that choice leaves out important information.
"It misses the mark on accurately informing the people that these reviews are for, because it's only giving one glimpse into it. It's not giving the entire picture," said Bridie Lee, one of the leaders of the group Schools Beyond Screens L.A., part of a national coalition that advocates for stricter guardrails around technology in the classroom. The organization should incorporate other groups' analyses of privacy and safety risks into its own reviews, Lee contends, especially if those analyses have found products unsafe for minors.
In response, Freitag said: "We totally agree-both as educators and as parents ourselves-that data privacy information is incredibly important, and organizations with expertise on these areas should absolutely be doing this kind of review." She added that existing data privacy reviews from organizations like ISTE and Common Sense Media tend to be lighter on instructional information, "this is the specific gap and need that we are targeting."
This coming school year, the organization plans to analyze at least an additional 20 tools and partner with Stanford University's AI in Education Lab to conduct additional reviews of evidence-of-impact studies. For educators interested in broader context on AI's role in schools, the AI for Education resource hub offers training materials and practical guidance.
Why this matters for educators
District and school leaders are making purchasing decisions about AI tools right now, often without independent evaluations of instructional quality. Instruction Partners' profiles give you a starting point for comparing products based on what actually happens in classrooms-what students do, what teachers do, and whether the tool reduces or preserves effortful thinking. Before adopting any AI product, ask whether it has a clear instructional purpose, whether it supports or supplants teacher-student relationships, and what evidence exists that it improves learning outcomes. The data is thin, but it's better than marketing materials.
Your membership also unlocks:
