AI news ·
QED Score provides reliable proxy for preprint quality, study finds
Researchers built an AI tool that scored 57,000 scientific preprints to evaluate unpublished work. Experts agreed with its quality ratings 75 percent of the time.
Researchers at Tel Aviv University developed an AI tool called QED to evaluate the quality of scientific preprints, scoring more than 57,000 life science manuscripts to address the severe bottlenecks in traditional peer review. The system offers a standardized metric to assess unpublished work, providing scientists and institutions with a faster alternative to journal rankings for evaluating research validity.
Scoring unpublished research
Traditional peer review often takes up to 18 months, forcing scientists to spend unpaid evenings and weekends evaluating manuscripts. Oded Rechavi, a molecular biologist at Tel Aviv University, and his team built QED to help researchers gauge the quality of preprints before formal publication. The team recently expanded the tool to generate a QED Score, which measures the originality and validity of findings in blinded evaluations.
The AI processes manuscripts without author names, institutions, or countries to remove location and gender biases. Researchers used this pipeline to score over 57,000 bioRxiv preprints submitted over a one-year period. "Everyone would agree that the way science publishing works at the moment is far from ideal," Rechavi said.
Testing the metric against experts
To validate the QED Score, the team compared the AI ratings for nearly 5,000 preprints against the impact ranks of the journals that eventually published them. The scores showed a positive correlation with journal rankings, though some high-scoring papers landed in lower-tier journals and vice versa.
The researchers then sent these contradictory pairs to domain experts, instructing them to ignore the final publication venue and judge the raw manuscript quality. Experts agreed with the higher QED Score 75 percent of the time, favoring the AI assessment over the journal prestige.
Pedro Beltrao, a biologist at ETH Zürich not involved in the project, acknowledged the need for such tools but warned against using a single metric for career evaluations. "As a scientist, nobody wants to be evaluated by a single metric," Beltrao said.
Integrating AI into the review process
The development of tools like QED reflects a broader trend in adopting AI for Science & Research to handle the overwhelming volume of daily publications. While some scientists worry about outsourcing critical thinking, experts argue the technology simply accelerates the initial screening process.
Sandhya Koushika, a neuroscientist at the Tata Institute of Fundamental Research, pointed out that researchers already receive AI-generated peer reviews from journals. She argued that AI cannot conduct or troubleshoot biological experiments, meaning human oversight remains essential for practical methodology.
Rechavi emphasized that QED is not designed to replace human peer review. He views it as a complementary system to filter papers and highlight gaps in methodology before experts invest their time.
Why this matters for science and research professionals
Scientists and research managers can use the QED Score to quickly assess the validity of preprints without waiting for formal journal acceptance. While the metric should not dictate hiring or funding decisions, it provides a practical, bias-reduced filter to manage the growing volume of unpublished literature.