AI Scribes Under TGA Scrutiny: Are They Medical Devices?
The Therapeutic Goods Administration (TGA) has raised concerns about AI scribes used in healthcare, questioning whether these tools qualify as medical devices and if they meet regulatory standards. The TGA’s recent report, Clarifying and strengthening the regulation of Medical Device Software including Artificial Intelligence, highlights issues around accountability, transparency, and responsibility among developers of AI scribes.
What Are AI Scribes and Why the Concern?
AI scribes are digital tools designed to summarise clinical notes. However, the TGA points out that many of these systems go beyond note-taking by suggesting diagnoses or treatment options, which could place them under the definition of a medical device. If so, they must be registered and approved before entering the Australian healthcare market.
The report specifically notes: “Users report digital scribes frequently propose diagnosis or treatment options for patients beyond the stated diagnosis or treatment a clinician has identified during consultations.” This raises red flags about potential risks to patient safety if these tools are not properly regulated.
International Moves and Regulatory Context
Similar to the UK's National Health Service (NHS), which has classified AI scribes as medical devices, the TGA appears to be moving in the same direction. The TGA already regulates software and AI systems that meet the medical device definition under the Therapeutic Goods Act 1989.
In 2021, the TGA updated its classification system to better reflect risks associated with medical software, while carving out low-risk products or those regulated by other agencies. The 2024 review aimed to check if current laws still address the growing use of AI and medical software effectively.
Key Findings on Developer Practices
The TGA’s report reveals a reluctance from some developers to engage fully with regulatory processes. It states: “Developers have expressed views that healthcare providers should take responsibility for validating and verifying the outputs of deployed systems they choose to use, while simultaneously limiting access to information about the datasets used to train their product or to test the model used to operate it.”
This approach conflicts with the need for transparency and accountability, especially in healthcare settings where patient safety is paramount. The report also highlights the misconception among some developers that software products, particularly those with a human in the loop or providing information only, pose minimal risk.
Potential Risks Identified
Stakeholders, including clinicians and patients, have flagged concerns about the lack of transparency and regulatory oversight. The absence of clear information about AI involvement in clinical workflows could lead to harm if users are unaware that AI influences decision-making.
The report warns that these factors combined may create conditions where patient safety is compromised.
Focus on Digital Mental Health Tools
The TGA also pointed out that current exclusions for digital mental health tools may no longer be appropriate. It called for an urgent review in partnership with the Australian Commission on Safety and Quality in Health Care to ensure these tools are properly regulated and safe for consumers.
What This Means for Healthcare Professionals
Healthcare providers should be aware that AI scribes and digital mental health tools might soon face stricter regulatory requirements. It’s critical to stay informed about these developments and understand the implications for clinical practice and patient safety.
For those interested in learning more about AI in healthcare and related training, resources like Complete AI Training courses offer up-to-date information and skill development opportunities.
Your membership also unlocks:
