A retracted paper in an Elsevier journal began with "Certainly, here is a possible introduction for your topic"-one of ChatGPT's standard openers. It passed peer review without anyone noticing. Generative AI has made scientific paper fabrication trivially cheap and fast, worsening a publish-or-perish problem that has plagued academia for decades. A study of over 15 million PubMed biomedical abstracts found that at least 13.5% of papers published in 2024 showed signs of AI assistance, with rates climbing as high as 40% in certain disciplines. Meanwhile, the number of papers produced by paper mills-organized operations that fabricate and sell fraudulent research-is doubling every 1.5 years.
"AI is an accelerant, not the kindling," said Ivan Oransky, cofounder of Retraction Watch, which tracks retractions across scientific literature. "The kindling is publish or perish."
Publishing is crucial for many scientists' careers, especially in academia. It determines jobs, funding, tenure, and immigration status for international researchers. The pressure to publish has always tempted some to take shortcuts. With generative AI, a researcher can now create a convincing manuscript in a few hours, complete with fake data and citations. Many journals lack the resources or motivation to catch this.
The spectrum of AI fraud
The problems range from the absurd to the dangerously convincing. At one end are manuscripts with signature AI phrases such as "As an AI language model" left in the text because no one read the paper carefully enough. At the other end are papers that use AI to generate text, fabricate datasets, and create realistic-looking figures that evade standard detection. "Generative AI can make digital images completely from scratch. It's really, really hard to tell the difference," said David Resnik, a bioethicist at the National Institute of Environmental Health Sciences.
These fraudulent papers follow scientific conventions, cite references, and present data that conform to expected statistical distributions. Once published, they spread. Other scientists build on results that don't exist, wasting time, funding, and resources. "Other researchers cite these papers, and AI systems train on them," said Guillaume Cabanac, a research integrity expert and computer scientist at the University of Toulouse. Each repetition of flawed information compounds the error, ultimately threatening public knowledge and undermining trust in science.
There are legitimate uses of AI in scientific writing. It can smooth prose, improve grammar, and help non-native English speakers present work more clearly. "It can even draft sections of a paper with appropriate human oversight," Resnik said. "But at the end of the day, the human being must be responsible, and the contribution of AI should be appropriately disclosed."
Detection systems are losing the race
Cabanac built the Problematic Paper Screener in 2020 after spotting roughly 40 papers with definite signs of AI-generated text that had cleared peer review. The tool now trawls 160 million papers indexed by the Dimensions bibliographic database. It flags what Cabanac calls tortured phrases-grammatically correct but scientifically nonsensical expressions like "formic corrosive" instead of "formic acid" and "weighty metals" instead of "heavy metals." For AI-generated text, he looks for smoking guns: traces left when users copy ChatGPT output without reading it. "Regenerate response" is one he sees often. "No researcher, no human would say that," Cabanac said. "It is hideous."
When the screener flags a paper, Cabanac posts a comment on PubPeer, a postpublication peer review platform, and contacts the publisher directly. More than 3,000 papers have been retracted as a result of his work. Several major publishers have implemented tortured-phrase screening at submission, catching problematic manuscripts before they reach peer review.
But detection remains a rearguard action, and its creators are honest about its limitations. The tools only catch crude fraud. "I suspect there are cases where people generate datasets that are more difficult to find," Cabanac said. Resnik's 2025 paper described the potential for AI to create highly realistic fake data as "a ticking time bomb" in scientific publishing. There is no reliable method for distinguishing AI-fabricated datasets from real experimental data, and as models improve, there may never be one. "I think it's an arms race," Resnik said. "As tools improve, models get better at evading them, and users also get better at evading them. I think it's ultimately probably a losing battle."
Oransky sees AI use as nearly inevitable, not just for publishing papers but for grant applications. Applying for grants is so burdensome that "you should just accept that AI is going to be how people do most of it," he said. "Even if we were to get a detector that works, it won't solve all our problems. This notion that if we just tackle the AI problem we'll get back to something great-it was never so great. Editors have been complaining about not being able to find peer reviewers for decades."
The root cause: what science rewards
Research integrity experts agree that the underlying driver of AI fraud is the incentive structure of modern science. Researchers are evaluated primarily on publication volume and journal prestige. Tenure, funding, promotions, salary, and institutional rankings all depend, to varying degrees, on how many papers a researcher produces. This system predates AI by decades. "We don't need generative AI to make a dog's breakfast out of scientific literature," Oransky said.
The San Francisco Declaration on Research Assessment (DORA), drafted in 2012, calls on universities, funders, and publishers to stop using publication volume and journal prestige as proxies for research quality. More than 25,000 institutions and researchers worldwide have signed it. Canada's federal funding agencies have introduced narrative curriculum vitae for grant applications. The US National Institutes of Health declared AI-generated grant applications ineligible and capped grant submissions at six per year per researcher.
Cabanac pointed to a simpler approach: "If I had a magic wand, we'd ask people to list only their top five articles, evaluating quality over quantity. In France, hiring and promotion committees are already asked to focus on a short selection of works." But Resnik is less optimistic about the pace of change. "We've been talking about not measuring quantity over quality for decades. It hasn't happened."
Detection and disclosure buy time. "But only worrying about catching the bad guys isn't going to get you anywhere," Oransky said. "Should we end inequality, or should we punish people who end up having to steal?" He argues both are necessary. "You're never going to let it all run rampant while you fix the upstream."
Resnik is working on a proposal for data attestation statements-a signed declaration submitted with every paper or grant application that the data are real and original records exist. The idea is that real science can always be traced back to its source. "When you have AI, you don't have an original source. It's just made up," he said. He also raises the possibility of digitally certifying experimental images the way fine art is certified, logging exactly which instrument produced an image and on which day. If someone can prove a result came from a real microscope on a real Tuesday, fabrication becomes much harder.
The barrier to fraud has never been lower. When fabricating a paper required months of effort and specialized skill, the cost was high enough to deter most people. When it requires an afternoon and a chatbot, the cost-benefit calculus changes entirely. "Funders and funding agencies need to take responsibility, and institutions need to develop better AI policies," Resnik said. "We need to slow down. Publish less and let peer reviewers do their job."
Why this matters for science and research professionals
The AI fraud problem doesn't just waste funding-it creates an uneven playing field. Researchers who fabricate long publication records with AI compete directly with honest scientists for jobs, grants, and recognition. Every fraudulent paper that enters the literature becomes part of the training data for future AI models and the citation record for future studies, compounding the damage over time. For individual researchers, the most practical defense is to scrutinize the provenance of papers they cite, participate actively in postpublication peer review on platforms like Retraction Watch and PubPeer, and advocate within their institutions for evaluation criteria that reward quality over quantity. The system won't change overnight, but the scientists who push for it will help determine whether the literature remains trustworthy.
Your membership also unlocks:
