Major insurers use AI to deny claims. Experts warn of hidden bias.
Large health insurance companies now rely on artificial intelligence to make medical coverage decisions, raising concerns about whether the technology reproduces and amplifies existing disparities in healthcare.
Jude Odu, a health technology expert who spent 25 years in the industry including time at United Healthcare, said insurers have shifted from human review to near-total automation. "It used to be that before an insurer denied a claim, a human being had to look at [the claim], look at all the clinical context," Odu said. "But then what has happened over the past few years is that they are now farming that out almost 100% to AI."
How the system worked before
When Odu worked in United Healthcare's appeals and denials department, medical directors would review cases in person. "Nurses would come in, the medical director would come in, they would look at the case and talk about it for maybe 30 seconds, and the medical director would simply put the gavel down and say, 'Denied,'" he said.
That process was flawed. But it included human judgment and the possibility of discussion.
The profit motive behind denials
Health insurance in the U.S. operates differently than in other developed nations. "People, for example, don't go bankrupt because of a hospital bill" in most other countries, Odu said. "That is so totally different from here in the U.S. where healthcare is big business."
For shareholder-owned companies, denials are profitable. The fewer claims paid out, the higher the profit margin.
AI amplifies existing discrimination
When AI enters this system, it doesn't fix bias - it scales it. "The AI is only as good as the data it was trained on, and only as good as the people that wrote the computer program," Odu said. "In cases of health disparities, what AI essentially does is, it takes the existing frameworks of discrimination, and it just magnifies them because that's what it's trained to do."
Real-world examples demonstrate the problem:
- United Health Group acquired NaVi Health in 2020 for $2.5 billion for its AI algorithm. The company now faces a lawsuit alleging that 9 out of 10 predictions from the system get reversed when patients appeal - suggesting the AI is not functioning as intended.
- An AI system designed to schedule patient appointments produced 33% longer wait times for Black patients. The model used ZIP code, employment status, insurance type, and past no-show rates - all variables that correlate with race.
Where AI could work in insurance
Odu does not argue against using AI in health insurance. Instead, he emphasizes implementation matters. "This all boils down to the right type of implementation of AI systems. That means that you have to consciously look out for the possibility that an AI system will amplify the preexisting disparities."
AI could audit 100% of claims to detect discriminatory patterns - such as certain procedures being denied at higher rates for specific ZIP codes or demographics. Models trained on inclusive data sets could identify patients at high risk for chronic conditions and trigger proactive care interventions to close gaps.
The difference lies in purpose. "The 'North Star' for using AI in healthcare spaces should not be efficiency, but rather patient health outcomes," Odu said.
Government adoption on the horizon
The Centers for Medicare and Medicaid Services are already deploying AI at scale. The Trump administration has discussed using AI for government-backed health plans.
"There's no holding this dam back. It's going to break," Odu said. "But I'm not necessarily worried about the dam breaking as much as I'm worried about how we channel the water."
Without guardrails, he added, AI deployment in health insurance could produce "very unintended consequences."
For healthcare professionals managing claims and coverage decisions, understanding how AI systems work - and their limitations - is increasingly essential. Learn more about AI for Healthcare and AI for Insurance.
Your membership also unlocks:
