AI Models Learn to Detect Misleading Scientific News with New Dataset

AI Models Learn to Detect Misleading Scientific Reporting

May 29, 2025

Artificial intelligence can sometimes generate inaccurate information, especially large language models (LLMs) like Llama and ChatGPT, which may "hallucinate" facts. But what if AI itself could help identify misleading scientific news and guide people toward more trustworthy information?

Researchers at Stevens Institute of Technology have developed an AI system that uses both open-source and free commercial LLMs to spot potentially misleading narratives in scientific news reports. This approach aims to flag inaccurate claims automatically and give readers clearer insights into the facts behind scientific discoveries.

Building a Unique Dataset

The team started by compiling a dataset of 2,400 news reports about scientific breakthroughs, focusing on COVID-19. These included:

• Human-written reports from reputable science journals and low-quality outlets known for fake news
• AI-generated reports, half reliable and half containing inaccuracies
• Original research abstracts from the CORD-19 dataset, paired with each news report for accuracy verification

This dataset is the first of its kind to systematically help train AI models to detect inaccuracies in science reporting publicly available online.

How the AI Detects Inaccuracies

The researchers designed three AI architectures to analyze news reports. One notable method involves a three-step process:

• Summarizing the news report and extracting key points
• Comparing each claim at the sentence level against evidence in the original peer-reviewed research
• Deciding if the news report accurately reflects the scientific findings

They also introduced "dimensions of validity," targeting common errors like oversimplification or confusing correlation with causation. Instructing the AI to consider these dimensions improved its accuracy significantly.

Results and Next Steps

The AI models correctly distinguished reliable from unreliable news reports about 75% of the time. Interestingly, they performed better at spotting errors in human-written content than in AI-generated reports. The reasons are still unclear, though humans also find it challenging to detect technical errors in AI-generated texts.

The research team plans to refine these models further, potentially creating AI tools specialized in specific scientific fields. This could help the AI "think" more like human experts and improve detection accuracy.

Practical Implications

In the future, such AI systems could be integrated into browser extensions that warn users about inaccurate scientific news as they browse. They might also help rank publishers based on the accuracy of their science coverage or assist in developing LLMs that generate more accurate scientific explanations.

As AI becomes a bigger part of how we access information, tools like this can support better decision-making and help reduce the spread of misinformation.

For professionals in research and human resources, keeping an eye on these developments can be valuable. Understanding how AI evaluates scientific content can inform training, communication strategies, and critical evaluation of information sources.

Learn more about AI and its applications in science and research at Complete AI Training.

Further Reading

For a detailed look at the dataset and AI pipelines, see the research paper: CoSMis: A Hybrid Human-LLM COVID Related Scientific Misinformation Dataset and LLM pipelines for Detecting Scientific Misinformation in the Wild.