A study from Stanford University and Bocconi University presented at the International Conference on Machine Learning in Seoul on 8 July warns that using large language models for scientific peer review can be gamed through "paper laundering" - cosmetic rewrites that boost AI scores without improving substance. The findings, published on arXiv, show that AI reviewers also reduce the diversity of expert opinions, an essential part of rigorous evaluation.
AI reviewers mimic human feedback but stifle disagreement
Researchers found that LLM-generated reviews were similar to those written by humans and often repeated common phrases across different papers. This suggests the assessments are generic rather than tailored to each submission. The study notes that disagreement among human experts is a critical feature of peer review. "Disagreement, though, among the perspectives of diverse human experts is an important feature of peer review, which is why the work of senior committee members in aggregating those views and collectively making final acceptance decisions is so important," the paper states.
Paper laundering: how authors can game the system
The team discovered that AI reviewers favour a particular writing style. An author can use another LLM to rewrite their paper to match that style, artificially raising the review score. "They can be gamed to improve scores through fully automated paper rewriting. We call this paper laundering: cosmetic paper rewrites to increase AI review scores without improving the scientific substance," the researchers write. The study adds to a growing body of work on AI for Science & Research, where automated tools face increasing scrutiny.
The risk of an intellectual monoculture
If paper laundering spreads, scientific writing could converge toward whatever style the AI reviewer rewards. "If paper laundering becomes widespread, scientific writing will converge toward whatever style the AI reviewer rewards, risking an intellectual monoculture and discouraging diverse ways of presenting ideas," the study says. The authors argue that replacing human judgment with AI that cannot meet basic requirements undermines the high-stakes process of deciding what research gets published and funded.
Why this matters for Science and Research
For professionals who depend on rigorous Research, the study underscores that automating peer review without validated safeguards threatens the integrity of scientific publishing. Speeding up the review process cannot come at the expense of diverse expert judgment. "An effective solution requires validated tools, not a simple replacement of human judgment with systems that fail to meet basic requirements," the paper concludes. Institutions and reviewers should demand evidence that AI tools preserve disagreement and resist gaming before they are adopted in editorial workflows.
Your membership also unlocks:
