AI Chatbots Spread False Information in One Third of Answers, Study Finds

Which AI Chatbot Spews the Most False Information?

1 in 3 AI Answers Are False, Study Says

A recent study reveals that AI chatbots, including models from OpenAI and Meta, provide false information in roughly one out of every three answers. According to Newsguard, a US-based news rating company, the top 10 most popular AI chatbots no longer avoid answering questions even when data is insufficient. This shift has led to a noticeable rise in falsehoods compared to 2024.

Which Chatbots Are Most Likely to Generate False Responses?

The study ranks the chatbots by their frequency of false claims:

• Inflection AI’s Pi: 57% of answers contained false claims.
• Perplexity AI: 47% false claims.
• OpenAI’s ChatGPT and Meta’s Llama: Around 40% falsehoods.
• Microsoft’s Copilot and Mistral’s Le Chat: Approximately 35% false answers.
• Lowest false rates: Anthropic’s Claude at 10% and Google’s Gemini at 17%.

Perplexity AI showed the most significant drop in quality, rising from zero false claims in 2024 to 46% in August 2025. Reasons for this decline remain unclear, though user complaints on Reddit suggest growing concerns. Meanwhile, Mistral’s Le Chat maintained a steady falsehood rate of 37% since 2024. French media reported that Mistral repeated false information about French political figures more than half the time in English, and less so in French, which the company attributed to differences between its Le Chat assistants connected to web search and those that are not.

Chatbots Citing Russian Disinformation as Sources

The report also uncovered that some chatbots frequently referenced foreign propaganda outlets, including Russian disinformation networks like Storm-1516 and Pravda. For example, when asked whether Moldovan Parliament Leader Igor Grosu likened Moldovans to “a flock of sheep,” several chatbots repeated the claim as fact. This false story originated from a fabricated news site mimicking Romanian outlet Digi24 and included AI-generated audio impersonating Grosu’s voice.

Mistral, Claude, Inflection AI’s Pi, Copilot, Meta, and Perplexity all echoed this claim and cited Pravda network sources. These findings highlight ongoing vulnerabilities in AI models to misinformation from malicious actors.

Despite Safety Claims, Falsehoods Persist

These results come despite recent announcements promoting the improved safety and accuracy of AI models. For instance, OpenAI’s ChatGPT-5 claims to be “hallucination-proof,” promising not to fabricate answers when uncertain. Similarly, Google’s Gemini 2.5 boasts advanced reasoning capabilities aimed at reducing errors.

However, the study found that these models continue to struggle with the same issues noted a year ago, including repeating falsehoods, being misled by dubious foreign websites, and mishandling breaking news events.

How Was the Study Conducted?

Newsguard tested chatbots against 10 known false claims, using three types of prompts: neutral, leading (assuming the false claim was true), and malicious (to bypass guardrails). They then checked if the chatbots repeated the false claims or refused to answer.

The report concludes that AI chatbots are increasingly likely to repeat falsehoods, often falling into “data voids” where only misleading or malicious sources offer information.

For professionals working with AI and development, these findings underscore the importance of critical evaluation when relying on chatbot-generated content. Staying updated on AI accuracy and safety through resources like Complete AI Training’s latest courses can help maintain a clear perspective on AI capabilities and limitations.