Major Insurers Deploy AI for Claims Decisions, Raising Oversight Concerns
Large health insurance companies are increasingly using artificial intelligence to approve or deny medical claims with minimal human review, according to Jude Odu, a healthcare technology expert with 25 years of industry experience. The shift has drawn scrutiny over whether the systems perpetuate existing healthcare disparities rather than address them.
Odu worked in the appeals and denials department at United Healthcare early in his career. He witnessed the previous process firsthand: nurses and medical directors would review cases for about 30 seconds before a medical director rendered a decision. Now, insurers have automated that judgment almost entirely to AI systems.
AI Amplifies Existing Discrimination
The core problem is straightforward. AI systems learn from historical data. When that data reflects past discrimination in healthcare, the algorithms reproduce and magnify those patterns.
"The AI is only as good as the data it was trained on, and only as good as the people that wrote the computer program," Odu said. "In cases of health disparities, what AI essentially does is, it takes the existing frameworks of discrimination, and it just magnifies them."
Real-world examples demonstrate the risk. United Health Group acquired NaVi Health in 2020 for $2.5 billion, gaining access to an AI algorithm now facing lawsuits. The company alleges that nine out of 10 predictions from the system get reversed when patients appeal - suggesting the AI frequently makes incorrect decisions.
A separate AI scheduling system produced 33% longer wait times for Black patients. The model relied on ZIP code, employment status, insurance type, and past no-show rates - variables that correlate with race.
The Business Incentive Problem
The U.S. health insurance system operates differently from other developed nations. Outside the United States, people do not face bankruptcy from medical bills. Here, healthcare is structured as a profit-driven business.
Denials benefit insurers financially. The less companies pay out in claims, the more profit shareholders receive. Adding AI to that equation creates pressure to automate decisions that deny coverage.
How AI Could Work Better
Odu argues AI has legitimate uses in health insurance if implemented with different priorities. The goal should shift from efficiency to patient health outcomes.
One approach: deploy AI to audit 100% of claims and flag patterns suggesting discrimination. An AI system could detect when certain procedures face higher denial rates for specific ZIP codes or demographics.
Another option involves training AI on inclusive datasets to identify high-risk patients for chronic conditions, then using the system to proactively close gaps in care for vulnerable populations.
"The efficiency gains are going to come no matter what," Odu said. "But what is efficient for a health insurance company may not be efficient for you and I as patients."
Government Plans May Follow
The Centers for Medicare and Medicaid Services are already deploying AI at scale. Federal programs like Medicare and Medicaid will likely adopt similar systems in the coming years.
Odu expressed concern less about whether AI will expand into government health plans and more about how institutions channel that power. "There's no holding this dam back," he said. "But I'm not necessarily worried about the dam breaking as much as I'm worried about how we channel the water."
Guardrails around AI deployment in healthcare will determine whether the technology improves outcomes or creates unintended consequences.
Learn more about AI for Healthcare and AI for Insurance applications in medical practice and claims processing.
Your membership also unlocks:
