Balancing Regulation and Risk of AI and Machine Learning Software in Medical Devices
August 13, 2025
Advantages of AI/ML Software
AI-based software is increasingly integrated into medical devices, with over 1,015 AI-enabled devices authorized by the FDA in the US as of March 2025. One notable example is the Sepsis ImmunoScore, approved in April 2024. This AI software analyzes demographic data, vital signs, and blood culture results from electronic medical records to estimate a patient's risk of developing sepsis within 24 hours.
A November 2024 study demonstrated the ImmunoScore’s high diagnostic accuracy in predicting sepsis and its ability to forecast critical secondary outcomes such as in-hospital mortality, ICU admission, and the need for mechanical ventilation. This showcases AI’s potential to support healthcare professionals in making informed decisions. However, the trustworthiness of such AI systems requires clear regulatory oversight and standardization.
Artificial Intelligence and Machine Learning Software Types
Recognizing the differences among AI software types clarifies why some require tighter regulation. According to the AAMI/BSI Initiative on AI, AI in medical devices can be rule-based or data-driven. Rule-based AI applies fixed logic for predictable decisions, while machine learning (ML) systems learn from data inputs and adjust outputs.
ML systems break down further into locked models, which require external approval to change, and continuous learning models that automatically update their algorithms. Continuous learning models present the greatest regulatory challenges due to their dynamic nature and less predictable outputs.
Concerns With AI/ML in Medical Devices
Continuous learning models raise concerns because their outputs can change after initial validation. This unpredictability challenges the principle that medical devices must produce reliable and consistent results. To address this, inputs for training algorithms should be predefined and relevant, and performance must be continuously monitored against expected outcomes.
The FDA recommends that developers describe anticipated algorithm changes before marketing devices to minimize the risk of unexpected results during clinical use.
Regulation of AI/ML-Based Medical Devices
AI/ML software intended to diagnose, treat, or prevent disease is classified as Software as a Medical Device (SaMD). The FDA categorizes medical devices by risk into Class I (low risk), Class II (moderate risk), and Class III (high risk). Higher-risk devices require more rigorous premarket approval (PMA) processes.
The adaptive nature of AI/ML SaMD complicates regulation because algorithms can evolve after receiving initial approval. In 2019, the FDA requested feedback on managing modifications to AI/ML SaMD and later introduced the concept of Predetermined Change Control Plans (PCCPs). PCCPs allow developers to outline expected algorithm changes upfront, potentially avoiding the need for a new PMA with every update.
The FDA continues to collaborate with organizations like AAMI and BSI to establish standards and guidelines for managing AI-driven medical device risks.
Recent FDA guidance includes final recommendations on marketing submissions for PCCPs (December 2024) and draft guidance on total product lifecycle management for AI-enabled devices (January 2025), with public comments requested by April 2025.
Conclusion
AI and machine learning hold promise for enhancing medical devices, but their benefits depend on sound regulation. Healthcare professionals must advocate for regulatory frameworks that ensure these technologies remain safe and reliable while allowing innovation to progress.
Staying informed about developments in AI regulation is essential for maintaining patient safety and integrating these tools effectively into clinical practice.
For those interested in expanding knowledge on AI applications in healthcare and medical device software, exploring specialized courses in AI technology can be valuable. Visit Complete AI Training for relevant resources.
Your membership also unlocks:
