Healthcare organizations deploying agentic AI must conduct thorough data inventories and understand exactly what patient data they can use under HIPAA, according to attorney Jordan Cohen of Akerman LLP. Missteps that expose protected health information (PHI) outside permissible uses can constitute a reportable breach, he warned.
"If you fall outside of a permissible use, then technically, if protected health information is involved, that can be considered a reportable breach," Cohen said in an interview with Information Security Media Group.
Many of the steps HIPAA-regulated entities should take aren't specific to AI, he added. A detailed data flow inventory is essential. "Diagramming and accounting for how you're ingesting data, processing it, storing it and how it's leaving your systems, how vendors are touching it and what they're doing to that data is going to be critical," Cohen said.
Without a clear picture of what data exists, where it resides, and who accesses it, securing systems and protecting patient privacy becomes extremely difficult. These practices have been discussed for years, but the rise of agentic AI makes them urgent.
Where Agentic AI Meets Patient Data
Cohen noted that agentic AI tools are already handling clinical decision support and administrative workflows, often using electronic health record data and other PHI. The speed and autonomy of these systems increase the risk that data moves beyond approved purposes without proper oversight.
As the adoption of AI for Healthcare grows, understanding exactly how vendors process data becomes non-negotiable. A breach triggered by an AI tool's unauthorized data use carries the same legal weight as any other HIPAA violation.
Beyond HIPAA: Legal and Technical Safeguards
Cohen's practice also addresses fraud and abuse laws such as the Anti-Kickback Statute and the Stark Law, which can intersect with AI-driven referral or billing tools. Technical safeguards, incident response plans, and ongoing monitoring are all part of the compliance picture, he said.
Transparency and patient consent remain critical. Organizations must be able to explain how AI systems use health data, not just to regulators but to the individuals whose information is at stake. AI for Legal considerations in healthcare extend to state breach notification laws and evolving federal guidance.
Why this matters for healthcare professionals
For healthcare leaders and IT teams, Cohen's message is clear: before deploying agentic AI, map every data flow. Know what PHI enters the system, how it is processed, where it goes, and what vendors do with it. A gap in that inventory can turn an AI project into a reportable breach. The same data governance fundamentals that have always applied to HIPAA compliance are now the frontline defense in AI adoption.
Your membership also unlocks:
