Generative AI is changing healthcare cybersecurity: what to fix now
Health systems are rolling out generative AI for documentation, analytics and administrative lift. That opens new doors for attackers - and gives them faster tools. AI systems will be targeted, and criminals will use AI to probe, phish and move faster than your team can click "remediate."
Taylor Lehmann, director in Google Cloud's Office of the CISO, underscores two hard truths: it's tough to tell when AI is wrong, and even harder to tell if it was nudged to be wrong by an attacker. The answer isn't guesswork. It's provenance, identity and speed.
Why AI increases the "is this wrong - and why?" problem
Hallucinations haven't been fully solved. Now add the possibility that a model's outputs or actions were bent by prompt injection, poisoned data or malicious tooling. Detecting that chain is difficult without deep visibility.
Lehmann points to practices like model cards, cryptographic binary signing and data lineage as must-haves. In short: know which model you're running, where the code came from, who trained it, and what data touched it - from inception to retirement.
Build provenance and transparency into every AI workflow
- Model provenance: Require signed artifacts for models and serving code. Reject anything unsigned or mismatched.
- Data lineage: Track training, fine-tuning and inference data sources with immutable logs.
- Radical transparency: Log every prompt, response, tool call and decision path. Keep audit trails that separate model behavior from user behavior.
- Environment integrity: Pin dependencies and verify them at build and run time. Treat model updates like high-risk code changes.
Identity is the new control plane
Most hospitals think "user ID and password." That's not enough. You need identities for humans, machines and AI agents - and you must tell them apart.
- Separate identities: Distinct identities for the model, the user and any agent/tools the model can call.
- Least privilege by default: Scope access per identity and per action. Block implicit trust between agents and systems.
- Context-aware controls: Tie policies to identity, data sensitivity and task risk (e.g., stricter controls for EHR access or order-entry tools).
Red team your AI - before attackers do
Create an AI red team that tries to make models misbehave. Push them to produce unsafe content, perform unintended actions and bypass guardrails. Test both the model and the surrounding tools and agents.
- Test surfaces: Prompt injection, data leakage, over-permissive tool use, jailbreaks, function-call abuse.
- Evaluate fitness: Is the model over- or under-fit for the use case? What breaks under distribution shift?
- Close the loop: Feed findings into model updates, guardrails and policy changes - and retest.
Governance needs clinical + engineering + risk
High-stakes settings demand governance with real authority. You need people who understand how AI is built and how care is delivered - and who can weigh regulatory and patient safety impact.
- Standing AI risk committee: Security, clinical, legal, compliance and operations at the same table.
- Clear approval paths: Gate new AI features with risk reviews and deployment standards.
- Operational metrics: Track incidents, unsafe outputs, near-misses and time-to-remediate.
Speed wins: design for seconds, not hours
Weaponized AI moves faster than humans. You won't get a one-hour SLA to contain a ransomware dropper or kill a rogue agent session. You'll get seconds - maybe milliseconds.
- Rebuild on demand: Can you redeploy a clean environment quickly? If not, fix that first.
- Patch velocity: Measure how fast you can roll out a control or patch across critical systems.
- Automated containment: Pre-wire playbooks to kill sessions, revoke tokens, rotate keys and block egress with one action.
- Telemetry to action: Go from detection to enforcement automatically, with human review after containment, not before.
A 90-day plan for healthcare security leaders
- Inventory: List every AI system, model, dataset, agent and tool integration touching PHI or clinical workflows.
- Provenance baseline: Require signed artifacts for models and serving stacks. Document training and fine-tune data sources.
- Identity refactor: Separate identities for users, models and agents. Enforce least privilege and short-lived credentials.
- Logging and audits: Turn on full prompt/action logging with tamper-evident storage. Establish review intervals.
- AI red team pilot: Run an initial exercise; fix the top five issues; retest in two weeks.
- Speed drills: Time your patch, rollback and credential-rotation procedures. Drive hours to minutes, minutes to seconds.
- Governance cadence: Stand up a cross-functional review that greenlights changes and tracks incident metrics.
Helpful references
Upskill your team
If your clinicians, engineers and security analysts need a shared foundation on AI safety and operations, explore curated learning paths by job function.
The takeaway: treat AI as a first-class system with provenance, identity and speed built in. Get visibility, test it like an attacker, and prepare your team to act in seconds. That's how you keep care running when AI is in the loop.
Your membership also unlocks:
