A study published in the International Journal of Medical Informatics identifies five socio-technical risks of AI speech-to-text tools in clinical documentation, from inconsistent consent practices to unclear accountability for errors. The research, led by Nelly Elsayed, associate professor at the University of Cincinnati, concludes that human review remains essential to prevent mistakes in medical records.
Elsayed's paper examined existing research, ethical guidelines, and government regulations to show how AI adoption is outpacing oversight. "These AI tools, the quality is improving, but there are other aspects we have to care about to make them more efficient and transparent," she said.
Five risks in clinical speech-to-text
- Inconsistent disclosure and consent practices
- Decreased performance for accented and disordered speech
- Extraneous noises at clinical facilities lowering accuracy
- Lack of human review over AI-generated text leading to unchecked mistakes
- Unclear accountability for errors: Is the software or the clinician responsible?
Elsayed said that having a human review data before it is finalized can reduce a fair amount of these concerns. "We need to have a human in the loop to check whether the text is exactly what has been spoken," Elsayed explained. "And that test needs to be done for the entire text, not just for the first couple statements."
Real-world conditions break AI performance
AI speech-to-text systems are often trained in ideal settings, without the noise of a real doctor's office - machines beeping, staff conversations, and background bustle. Without training large-language models on specific scenarios, accents, and speech disorders, they cannot be reliable in clinical practice. The paper notes that these tools perform worse with accented or disordered speech, which can lead to critical errors in patient records.
Training clinicians on the software before adoption is another way to curb errors. "The organization developing the system needs to give guidelines for the doctor, what they can use, what they cannot use and what to look out for," Elsayed said. Healthcare professionals can deepen their understanding of these tools through resources like Speech-To-Text AI Courses, which cover the capabilities and risks of voice recognition in clinical settings.
Why this matters for healthcare professionals
The study underscores that AI scribes can reduce charting time, but without rigorous human oversight, they introduce errors that could compromise patient safety. Clinicians and administrators should demand transparency from vendors, insist on human review of all AI-generated text, and ensure that systems are tested on diverse speech patterns and noisy environments before deployment. The responsibility for accuracy remains with the care team, not the software.
Your membership also unlocks:
