New AI Models Make More Mistakes, Creating Risk for Marketers
New AI tools, expected to be smarter, are actually making more factual errors than older versions. Tests reveal error rates as high as 79% in advanced systems from companies like OpenAI. This presents challenges for marketers who depend on AI for content creation and customer support.
Rising Error Rates in Advanced AI Systems
Recent evaluations show a clear trend: newer AI models are less accurate than their predecessors. OpenAI’s latest system, o3, had a 33% error rate when answering questions about people—double the errors of the previous generation. Even worse, the o4-mini model missed facts 48% of the time on similar tests.
When it comes to general questions, the error rates are even higher:
- OpenAI’s o3 made mistakes 51% of the time
- The o4-mini model was wrong 79% of the time
Google and DeepSeek’s AI systems show similar issues. Amr Awadallah, CEO of Vectara and former Google executive, commented, “Despite our best efforts, they will always hallucinate. That will never go away.”
Real-World Consequences for Businesses
These errors have real impacts. Last month, Cursor, a programming tool, faced backlash after its AI support bot incorrectly told users they couldn’t use the software on multiple devices. This false information led to canceled accounts and public complaints. Cursor’s CEO, Michael Truell, clarified: “We have no such policy. You’re of course free to use Cursor on multiple machines.”
Why Reliability Is Declining
Why are newer AI models less reliable? The answer lies in training methods. Companies like OpenAI have already used most available internet text for training their models. Now, they rely on reinforcement learning, which teaches AI through trial and error. While this improves math and coding skills, it appears to reduce factual accuracy.
Researcher Laura Perez-Beltrachini explained that these models tend to focus heavily on one task and start forgetting others. Additionally, newer models "think" step-by-step before answering, and each step introduces more chances for mistakes.
For marketers, this means AI-generated content with errors can damage search rankings and brand trust. Pratik Verma, CEO of Okahu, warns: “You spend a lot of time trying to figure out which responses are factual and which aren’t. Not dealing with these errors properly basically eliminates the value of AI systems.”
Protecting Your Marketing Operations
Here are practical steps to protect your brand when using AI:
- Have humans review all AI-generated, customer-facing content
- Establish fact-checking processes for AI materials
- Use AI mainly for structure and ideation, not for factual accuracy
- Consider AI tools that provide source citations, known as retrieval-augmented generation
- Set clear protocols for handling questionable AI information
The Road Ahead
Researchers and companies like OpenAI are actively working to reduce these high error rates. Meanwhile, marketing teams must implement safeguards while benefiting from AI efficiency. Businesses with strong verification processes will maintain a better balance between speed and accuracy.
Striking this balance will remain a key challenge as AI tools continue to develop.
For marketers looking to deepen their AI skills and learn best practices, explore practical courses on Complete AI Training.