AI Scribes in Hospitals Face Real-World Pushback on Quality
Paul Boyer, a psychotherapist at Kaiser Permanente in Oakland, California, spends part of his workday correcting notes written by artificial intelligence. The hospital deployed Abridge, an AI scribe that summarizes patient visits automatically, to save clinicians time on paperwork. For Boyer, the software creates the opposite problem.
The AI "is not good at picking up on clinical nuance, at picking up on the emotional tone" that matters in mental health treatment, Boyer said. For patients experiencing mania, how something is said carries more weight than what is said - a distinction the software misses.
Hospitals across the country are rolling out similar note-taking tools. A year after installation, doctors using these products most frequently saved more than 30 minutes of work daily, according to a study of five hospitals published in April in the Journal of the American Medical Association. Many clinicians report positive experiences with the software.
Yet Boyer's experience points to a persistent gap between promise and performance. While he corrects the AI's errors, safety researchers worry that clinicians might not catch all mistakes - meaning future doctors could rely on inaccurate information.
The Regulatory Gap
No federal safeguard currently vets AI scribe software before deployment, said Raj Ratwani, a researcher specializing in how people interact with technology at MedStar Health. The concern extends beyond scribes to the broader wave of AI tools entering healthcare.
The Office of the National Coordinator for Health IT - which regulates electronic health records - has proposed rules that would weaken existing requirements. The proposed changes would eliminate mandates for user testing with actual doctors and nurses. They would also remove requirements that companies disclose how their AI systems make decisions.
The Obama and Biden administrations had required developers to test products on clinicians and be transparent about AI use. The Trump administration argues those rules burden innovation and competition. Removing them, HHS officials say, will expand choices for hospitals and other healthcare providers.
Abridge, the scribe company, said it "broadly supports" the proposed rules as necessary to keep pace with AI development. The company said it monitors clinician edits, ratings, and feedback after deployment.
Safety Concerns in the Details
Poor design in electronic health records can create serious errors. A confusing medication list might lead a doctor to select the wrong drug when placing an order. Removing requirements for user testing makes such mistakes more likely, Ratwani said.
The American Hospital Association and American College of Physicians have pushed back against eliminating transparency requirements. Both groups said removing AI transparency tools would undermine clinician trust and patient safety.
Even some electronic health record developers are uncertain about the changes. Leigh Burchell, vice president for policy at Altera Digital Health, said her industry group held "a lot of different perspectives" on the issue - unusual for a trade group typically aligned on regulatory matters.
Evidence of Effectiveness Remains Thin
A recent study comparing 11 AI scribes for potential use in the Veterans Health Administration found the software performed worse than humans across five simulated scenarios. "Although ambient AI scribes can generate complete notes, the overall quality remains broadly below that of human-authored documentation," the authors wrote, with missing information being particularly concerning for follow-up care.
Boyer said he can mostly ignore his AI scribe for now. But he worries management will schedule more patients based on expected time savings, forcing him to spend hours both seeing patients and fixing the software's errors.
"When I am correcting that note, I feel like this is too much work," Boyer said. "This is definitely making this worse, and this is taking up time that I need to not be spending on correcting an AI tool."
Kaiser Permanente said it does not require clinicians to use the AI scribe.
Your membership also unlocks:
