Healthcare AI needs safety standards before clinical use, experts say
Artificial intelligence is improving healthcare workflows through automated documentation, clinical scribing, and administrative support. AI systems also help clinicians manage medical knowledge by synthesizing research and patient data into actionable insights.
But integration into clinical settings requires caution. Dr Stephen Barnett, co-founder of MedLuma, warns that current AI tools carry significant risks that must be addressed before widespread deployment.
The reliability problem
AI "hallucinations"-where systems generate confident but incorrect medical information-pose a direct threat to patient safety. Some AI models also struggle to accurately identify emergency situations, raising questions about their reliability in critical care.
These failures underscore why validation cannot be optional. Healthcare systems need proof that AI tools work as intended before clinicians depend on them.
Building medical-grade AI
Dr Barnett advocates for AI specifically engineered and tested for healthcare use, rather than general-purpose models adapted for medical settings. Medical-grade AI requires different standards than consumer applications.
Human oversight must remain central. Clinicians need to verify that outputs trace back to credible sources and can be checked against established knowledge. This traceability is non-negotiable in clinical environments.
Education and standards
Stronger safety standards alone are insufficient. Healthcare professionals need better training to assess and deploy AI tools responsibly. Without this education, even well-designed systems can be misused.
Industry-wide standards are essential for distinguishing safe AI tools from untested ones. These standards create a baseline that healthcare organizations can rely on when evaluating new systems.
Learn more about AI for Healthcare and how to assess these tools effectively.
Your membership also unlocks:
