LAS VEGAS - At HIMSS26 on June 26, Medicomp CMO Jay Anders said provider skepticism around clinical AI remains high, fueled by hallucinations and unreliable outputs that make clinicians hesitant to use the tools in patient care. He called for stricter governance and technical guardrails before trust can rebuild.
"They're concerned, among other things, about hallucinations and inconsistent outputs," Anders said, adding that more governance and stronger guardrails are needed to gain provider buy-in. His comments came during a session focused on AI quality and safety in clinical workflows.
What's eroding provider confidence
Clinicians face a sharp dilemma: AI can draft summaries and suggest diagnoses, but when the output shifts unpredictably between queries, it creates risk. Anders pointed to examples where small rephrasing of a clinical question produced materially different recommendations, undermining confidence in any single response.
The core obstacles, as he described them, include:
- Hallucinated clinical facts that contradict the patient's record
- Inconsistent outputs across similar prompts
- A lack of organization-wide protocols for reviewing AI-generated content before it reaches a clinician
Governance as the missing layer
Anders told attendees that technical fixes alone won't solve adoption. Health systems need documented escalation paths when an AI's output is questionable and review processes that match the risk level of the task. Without that structure, he said, physicians treat AI as a curiosity rather than a dependable tool.
Anders's call for tighter oversight comes as organizations increasingly look to pair governance frameworks with practical AI for Healthcare training that teaches clinicians to spot hallucinations and evaluate model outputs. He argued that IT teams cannot be the sole gatekeepers; clinical users must understand what the model can and cannot do.
Why this matters for healthcare professionals
For physicians, nurses, and clinical informaticists, the gap between AI's promise and its everyday behavior is a safety issue. Governance rules and training initiatives directly affect whether a recommended treatment plan is trustworthy. Until those guardrails are standard, provider buy-in will remain fragile - and the tools will sit unused when they could assist with documentation, order review, or decision support.
Your membership also unlocks:
