AI is boosting scientific output - and burying the signal
Since late 2022, labs have been shipping more manuscripts with help from large language models. Editors are seeing the flip side: a flood of clean prose with thin scientific value. A new Cornell study confirms both trends - productivity is up, but signal detection is getting harder.
"There's a big shift in our current ecosystem that warrants a very serious look," said Yian Yin, assistant professor of information science at Cornell Ann S. Bowers College of Computing and Information Science. The paper, "Scientific Production in the Era of Large Language Models," was published Dec. 18 in Science.
What the researchers did
The team analyzed more than 2 million preprints posted from January 2018 to June 2024 across three major servers: arXiv (physical and computer sciences), bioRxiv (life sciences), and SSRN (social sciences). They trained a detector by comparing pre-2023, presumably human-written text with AI-style writing to flag papers likely drafted with LLMs.
With those flags, they tracked who adopted AI, how output changed, and whether flagged papers eventually cleared journal acceptance.
Key findings
- Productivity jumps: +~33% on arXiv; +50%+ on bioRxiv and SSRN for authors likely using LLMs.
- Language equity: Researchers from Asian institutions posted 43.0% to 89.3% more after adopting AI, depending on the server. Expect a shift in where high-volume science is produced.
- Better literature surfacing: With Bing Chat, users pulled in newer publications and relevant books more often than with traditional search. "People using LLMs are connecting to more diverse knowledge," said first author Keigo Kusumegi.
- Quality confusion: For human-written work, higher writing complexity correlated with journal acceptance. For AI-written work, high complexity did not - many well-written papers were judged to have low scientific value.
Why this matters for editors, funders, and PIs
- Volume is losing credibility as a performance proxy. Track acceptance, novelty, data/code quality, and replicability instead of raw counts.
- Review pipelines face heavier triage. Clear checklists and structured abstracts help. AI detectors can assist, but aren't ground truth.
- Equity gains are real. Reduced language friction will change who publishes the most and where collaborations form. Plan for it.
Practical moves for research teams
- Write an LLM policy for your group: where AI is allowed (editing, outline, literature surfacing), and where it isn't (data, results, claims). Keep humans accountable for interpretation.
- Disclose AI use. Note models, versions, and the scope of assistance. Keep prompt logs for internal QA.
- Use AI for search - verify every citation. Prefer primary sources. Be explicit about how references were found.
- Decouple writing polish from scientific merit in internal reviews. Weight methodology, data availability, and validation higher than style.
- Adopt pre-submission checklists: contribution clarity, claim-evidence alignment, ablation/controls, limitations, and reproducibility assets (code, data, configs).
Implications for policy and publishing
- Standardize AI-use disclosures across journals and funders.
- Rebalance incentives away from pure output metrics. Reward openness, replication, and field impact.
- Invest in reviewer tools and training to spot overconfident prose that outpaces substance.
- Support causal studies on LLM use in research workflows, as the Cornell team proposes.
What's next
The current results are observational. The team plans controlled experiments that randomly assign LLM access to measure causal effects on quality and output. A Cornell symposium on generative AI's role in research is slated for March 3-5, 2026, in Ithaca, to discuss how scientists and policymakers should respond.
As Yin put it: the question isn't "Did you use AI?" It's "How did you use it - and did it help?"
Resources
Skill up your team
If you're formalizing lab-wide LLM practices or training, see curated options by role here: Complete AI Training - Courses by Job
Study contributors
Authors include Yian Yin; Keigo Kusumegi; Xinyu Yang; Paul Ginsparg (founder of arXiv); and Mathijs de Vaan and Toby Stuart of UC Berkeley. The work was supported by the National Science Foundation.
Your membership also unlocks:
