AI Bias in Healthcare: A Growing Clinical Risk
Artificial intelligence is moving into routine clinical decisions, but a documented problem is moving with it: AI systems trained on biased data produce biased results, and those results can deepen existing health inequalities.
A 2019 study in Science found that commercial algorithms guiding healthcare decisions exhibited significant racial bias. When the system used healthcare costs as a proxy for health need, Black patients received the same risk scores as White patients who were actually less sick. Fewer Black patients were identified for additional care as a result.
More recent research shows the problem persists. Nine large language models evaluated for clinical bias recommended mental health assessments six to seven times more often for cases labeled as LGBTQ+ compared to clinical indication. Advanced imaging was recommended more frequently for cases labeled as high income than for low or middle-income cases.
The concern extends beyond reproduction of existing bias. AI systems can amplify bias through what researchers call a feedback loop: biased training data produces biased outputs, which then influence future decisions and training data. A 2024 study in Nature Human Behaviour found that clinicians who repeatedly used biased AI systems themselves became more biased over time-a pattern not observed in clinician-to-clinician interactions.
What Regulations Exist
The EU AI Act specifically requires bias mitigation for high-risk AI systems used in healthcare. The UK has no dedicated AI legislation but addresses the issue through existing frameworks.
UK regulators currently rely on the General Data Protection Regulation (GDPR), which requires data controllers to prevent discriminatory effects through technical or organizational measures. The Data (Use and Access) Act 2025 and the Equality Act 2010 also apply. The government has published a Data and AI Ethics Framework and signed the Bletchley Declaration, an international agreement on AI risk mitigation.
The Medicines and Healthcare Products Regulatory Agency (MHRA) recently consulted on AI regulation in healthcare and acknowledged bias in training data as a critical issue. Through its National Commission on AI, the regulator is convening clinicians, patient groups, and developers to establish standards emphasizing fairness and transparency. Recommendations are due this summer.
Steps to Reduce Bias
Bias enters AI systems at multiple points: in the data itself, in the algorithm design, and in how clinicians apply the outputs. Mitigation requires action across all three.
For developers and organizations deploying AI, practical steps include:
- Ensuring training data are representative and diverse
- Conducting formal bias audits
- Building diverse development teams
- Making model outputs transparent so humans can understand them
- Establishing data governance and risk management systems
- Monitoring for bias after deployment and reporting incidents
Clinicians have separate responsibilities. The General Medical Council requires fair treatment of patients and prohibits discrimination, but specific guidance on AI bias in clinical practice remains sparse.
Research in PLOS Digital Health outlines practical steps clinicians can take:
Human oversight. Clinicians should be prepared to review, challenge, and override AI recommendations using their own clinical judgment. This requires confidence in questioning the system-often harder than discussing a decision with colleagues. The clinician remains responsible for the final decision.
Training. Understanding how an AI system works and its limitations helps clinicians assess whether recommendations apply to specific patients or groups. The GMC expects clinicians to seek education on AI tools they use in practice.
Patient involvement. Patients should know when AI is involved in their care and understand its uncertainties and limitations. Individual patient needs, circumstances, and preferences must remain central to decision-making.
Governance engagement. Clinicians should participate in local governance processes for AI systems and report incidents or patterns-particularly if problems affect specific patient groups. This aligns with existing obligations to report adverse events involving medical devices.
The Clinician's Role
AI promises efficiency gains in healthcare, but clinicians cannot assume the systems are fair. Understanding the limitations of any digital tool before using it, identifying data bias where it exists, and ensuring bias does not dictate patient care quality are professional obligations.
The MHRA's regulatory framework should provide assurance for bias mitigation in software classified as AI medical devices. That framework, however, addresses only part of the problem. Individual clinicians and healthcare organizations deploying AI must actively work to identify and challenge bias in their own systems.
For more information on AI for Healthcare and understanding AI Data Analysis in clinical contexts, additional resources are available.
Your membership also unlocks:
