Study finds 146,900 fake citations in scientific papers generated by AI
Researchers from Cornell and UCLA identified nearly 147,000 fabricated citations across four major scientific databases, a sharp increase tied to the widespread adoption of large language models like ChatGPT and Gemini.
The problem stems from how these models work. Large language models generate plausible-sounding text that can be entirely false-a phenomenon called hallucination. When researchers use chatbots to draft citations without verification, the models often produce references to papers that don't exist.
The research team analyzed 111 million references from 2.5 million scientific papers. They searched for citations with titles that matched no actual publication. While some mismatches were spelling errors, many were clear hallucinations.
The problem is widespread, not isolated
The fake citations appear across many papers rather than concentrated in a few, suggesting numerous researchers are relying on AI-generated references without fully checking them.
The team also examined citation patterns before 2023, when chatbots were less common. They found a sharp rise in non-existent references after large language models became ubiquitous.
The four affected databases-arXiv, bioRxiv, SSRN, and PubMed Central-host preprints that researchers upload before formal publication. These repositories give papers immediate visibility to the global scientific community.
Trust in research is at stake
Usha Haley, a management professor at Wichita State University, called the proliferation of fake citations a serious warning. "Fake or AI-generated citations undermine trust in the scholarly record that provides the foundation on which peer review and cumulative knowledge rest," Haley said.
The concern extends beyond individual papers. When false citations spread through the scientific record, they can misdirect research and make it harder to identify legitimate findings.
arXiv announced this week that it will ban authors who submit work containing hallucinated citations or unverified AI content. Steinn Sigurdsson, arXiv's scientific director, said the issue is diluting the scientific corpus. "A lot of the AI stuff is either actively wrong or it's meaningless. It's just noise," he said. "It makes it harder to find what's really happening, and it can misdirect people."
What researchers should do
The findings underscore a basic principle: verify every citation before submitting a paper. If you use AI tools to draft references, check each one against the original source.
Understanding how large language models work and their limitations can help you use them responsibly in your research. The same applies to broader questions about AI's role in scientific research.
The stakes are high. Research forms the foundation for everything from internet protocols to battery technology. When false citations enter the record, they weaken that foundation.
Your membership also unlocks:
