Healthcare organizations spent the first phase of the AI boom asking whether the technology worked. The next phase asks a harder question: Who is responsible when it does not? As AI moves deeper into clinical decisions, patient communications, claims administration and health-data exchange, the regulatory environment remains fragmented, pushing AI-native self-governance from a compliance afterthought into both legal protection and commercial infrastructure.
Existing Law Still Applies to Healthcare AI
Waiting for lawmakers to produce a single AI rulebook is not an option. Laws governing privacy, discrimination, consumer protection, contracts and professional duties already apply to AI-enabled activity, even when they do not mention the technology directly.
"There are existing bodies of law that, while not passed or promulgated for the reason of AI, are still applicable to AI solutions," Alaap Shah, member of the firm at Epstein Becker Green, told Competition Policy International. That means a patient-facing model that produces different recommendations across demographic groups may create discrimination exposure. A clinical tool that influences a physician's decision can become evidence in a malpractice case. AI enters healthcare's existing liability structure - it does not replace it.
The Food and Drug Administration (FDA) continues to regulate certain software as a medical device, but the boundary remains "still a little murky," Shah said. A developer may call a product decision support. A state medical board may see a system behaving like an unlicensed clinical operator. "There are still going to be situations where a company may push right up against that line and FDA may come back and say, 'Actually, this is something we'd like to regulate.'"
AI Governance Must Become a Defensible Record
"Self-governance matters because defensibility matters. We have already seen that risk is manifesting with respect to the use of AI technology in the healthcare sector," Shah said. The problem for hospitals is that they often do not control the models they deploy. Vendors may own the system logs, training process and performance data, but the hospital is still expected to explain what went wrong when a tool fails. Contracts become the first line of defense.
Hospitals need more than standard security promises. They need logging requirements, preservation obligations, audit rights and clear rules governing access to the evidence a model produces. "To the extent that any events could be logged in the AI processing, that is something that needs to be happening so we can understand how that AI operated and why the input led to the output," Shah said. A clinician cannot defend a decision influenced by a system that cannot be reconstructed. The black box is now a contractual problem. These requirements are central to AI for Healthcare, where liability does not stop at the vendor's firewall.
De-identification and Bias Raise the Stakes
AI also challenges one of healthcare's most familiar privacy safeguards: de-identification. Removing direct identifiers can reduce risk, but AI systems can combine datasets, infer attributes and reconnect information that appeared anonymous in isolation. "It's a real possibility that AI algorithms could re-identify individuals if sufficient data gets put in, even if de-identified in the first instance," Shah said.
Bias compounds the exposure. AI can influence treatment, prior authorization, insurance coverage and patient communications. When those systems produce different outcomes for protected groups, organizations may face litigation, regulatory scrutiny or public backlash regardless of federal enforcement priorities. Internal governance - inventorying systems, classifying risk, assigning accountable owners, training employees and documenting controls - becomes the record an organization points to when something goes wrong. The goal is not to prove failure was impossible. It is to prove the organization was not careless.
Why this matters for healthcare, IT and development professionals
The shift means that accountability cannot be outsourced to a vendor, a compliance department or a future law. Healthcare delivery organizations must build a defensible record across every AI system they touch, from procurement through deployment. IT and development teams carry the technical burden: they must ensure models log what they do, preserve that evidence under the hospital's control and enable audits that will hold up in court or before a regulator. Implementing these controls requires a focus on AI for IT & Development, where system-level logging and audit trails are built into the software lifecycle - not bolted on after an adverse event. For clinicians and administrators, the lesson is the same: if you cannot reconstruct how a system reached a decision, you may struggle to defend it.
Your membership also unlocks:
