Doctors Are Building Their Own Clinical Tools With AI - But Security Risks Are Growing
Physicians are increasingly using AI coding assistants to build custom patient care applications without extensive programming backgrounds, a shift that could accelerate innovation in health systems but introduces serious security vulnerabilities that IT leaders must address now.
During a webinar Thursday, doctors demonstrated building clinical workflow tools using Claude Code, an AI assistant that can read code, edit files, and integrate with development tools. The message from clinicians was direct: health systems should let physicians build the software they need rather than waiting for traditional vendors.
"If the EHR is a problem, maybe just create your own," said Dr. MichaΕ Nedoszytko, an interventional cardiologist who won third place in Anthropic's recent hackathon.
The Security Problem
Security experts warn that AI-generated code often contains vulnerabilities that novice developers won't catch. Dave Kennedy, CEO of security firm TrustedSec and former NSA analyst, told Forbes this introduces "serious defects" and is "very alarming."
The concern has intensified since Anthropic announced Claude Mythos, a new AI model apparently capable of detecting system vulnerabilities. The company is providing early access to 40 organizations including Amazon, Apple, Google, Microsoft, and JPMorgan Chase to test defenses against AI-driven attacks.
On November 14, 2025, Anthropic disclosed the first AI-orchestrated espionage campaign. Chinese state-sponsored groups had used Claude Code to autonomously run full attack chains across roughly 30 global targets, conducting reconnaissance through data exfiltration.
HIPAA and Compliance Gaps
Doctors acknowledged privacy concerns under HIPAA and other regulations when using Claude Code. Dr. Nedoszytko noted that while his hackathon tool was built with HIPAA pathways from the start, production-ready code still requires professional engineering oversight.
"This always needs to be run through your team," he said of any tool handling live patient data.
Anthropic is developing regulatory plug-ins beyond its current HIPAA audit skill. The company also released Claude Code Security last month to scan codebases and suggest patches, though access requires joining a waitlist.
A 90-Day Deadline for IT Leaders
The Cloud Security Alliance released a whitepaper April 12 titled "The 'AI Vulnerability Storm': Building a 'Mythos-ready' Security Program." The paper urges every organization to complete a 90-day preparedness plan immediately.
"AI-driven vulnerability discovery and exploit development have accelerated dramatically," the CSA said. "The time between disclosure and exploitation is shrinking, and security teams are being asked to respond faster than current operating models allow."
The whitepaper's contributors included Jen Easterly, former director of the U.S. Cybersecurity and Infrastructure Security Agency; Chris Inglis, former National Cyber Director; and Heather Adkins, CISO at Google.
What IT Leaders Should Do Now
The CSA recommends organizations turn AI agents inward on their own code and dependencies. Priority actions include:
- Ask AI agents to conduct security reviews of any code immediately
- Build toward full audits within CI/CD pipelines
- Add security review capabilities directly into developers' coding agents
- Ensure all code-human or AI-generated-passes LLM-driven security review before merge
Asset management is also critical. Understanding all systems and dependencies helps identify both known and unknown vulnerabilities in an AI-driven threat environment.
Radi El Haj, CEO of payments company RS2, said the shift extends beyond any single model. "As AI continues to accelerate both insight and threat, the institutions that succeed will be those that treat cybersecurity not as a function, but as a core component of resilience and trust," he said.
Your membership also unlocks:
