Weak evidence: Much of the research on AI in schools doesn't hold up

How well does AI work in the classroom? It is harder to answer than the headlines suggest. Several reviews now indicate that the foundation is thin and that some of the research has serious flaws.

Errors in the syntheses

Patrick O'Neill, who teaches at the community college Ivy Tech in Indiana, has carried out a detailed audit of 14 of 39 existing meta-analyses on AI in education. A meta-analysis pools results from many studies to give a single overall answer. Justin Reich (MIT), Beth Tipton (Northwestern) and Susanna Loeb (Stanford) contributed to the work.

According to KQED MindShift, the audit found impossible statistics, fabricated or nonexistent references and exaggerated conclusions. Some meta-analyses compare students using AI tools with students receiving no instruction at all. Others cite research on generative AI that predates the technology's public launch.

The 14 meta-analyses have more than 3,000 citations and gain roughly 200 new ones each month, The Hechinger Report writes. The errors are thus spreading further. A meta-analysis in Nature from May 2025 was retracted just under a year later.

According to O'Neill, two thirds of the calls for papers on AI in education specifically invited research on benefits. That can create publication bias, meaning that positive results take up more space than others. Justin Reich says this is not reliable evidence and that all of the meta-analyses are flawed.

Few solid studies

Loeb's Stanford team collected roughly 900 studies on AI in education. Only 20 were of high enough quality. A further 32 were identified, but that is too few for a reliable meta-analysis. O'Neill's own meta-analysis of 44 well-designed experiments found effects that averaged around zero.

In March 2026, Stanford SCALE published a review that, according to EdTech Innovation Hub, found that AI tools improved performance in math, writing and programming while the tools were in use. The results were mixed when students were tested without AI. Most causal studies focus on math, and there is too little on reading, social-emotional development and equity. According to the review, there are no high-quality causal studies of students' use of AI in US K-12 classrooms. Early, limited evidence suggests that AI tools for teachers can cut preparation time.

Whom should you trust?

Ben Williamson of the University of Edinburgh warns that weak academic research could give tech companies such as Google and OpenAI the authority to act as primary sources of evidence for education policy. Their evidence may be influenced by commercial interests.

Beth Tipton puts it this way: "Anyone can pretty much claim they have evidence for anything, and share it with anyone."

For readers, this means that claims that AI boosts student learning should be taken with caution until there are more good studies.