Health systems weigh build or buy decisions while managing AI portfolio risks

BOSTON - Health systems are grappling with how to track and govern the growing number of artificial intelligence tools embedded in everything from hospital beds to electronic health records, executives said Thursday at the AI in Healthcare Forum. The discussion surfaced urgent challenges around compliance, cost, and the risks of shadow AI as organizations weigh whether to build or buy these technologies.

AI's hidden footprint in standard upgrades

"A standard upgrade now has AI in it," said Rebecca Mishuris, CHIO and vice president of Mass General Brigham. "The technology doesn't exist to do the monitoring we need to do. All of a sudden the data is out there." That data exposure can give vendors access to information they shouldn't have, panelists warned. Even hospital beds now include AI capabilities, said Deepti Pandita, CMIO and vice president of clinical informatics at University of California Irvine Health. "We can manage inventory on our own," Pandita said. "What we need is compliance."

Adam Landman, chief digital information officer and emergency physician for Brown University Health, stressed that health systems must adopt a new approach to life-cycle management. "We're going to need AI to detect vulnerabilities and then patch them," he said. "We need a whole new approach to life-cycle management." The growing complexity of AI lifecycle management is a central theme in guidance on AI for Healthcare.

The lifecycle problem and shadow AI

Kevin Day, CTO of Rhapsody, explained that vendor assessments refresh annually, but shadow AI still emerges when people with privileged access build tools that later circulate without proper credentials. "Some people get privileged access to a system. They will build something great and then pass it along to someone else who suddenly has access to information they shouldn't have," Day said. Proper credentialing must be in place, he added. Landman said his system now holds governance meetings specifically to address this issue.

Build versus buy: Dating, marriage, and divorce

Landman compared vendor relationships to dating: the vetting process should be thorough before any long-term commitment. Brown University Health starts by examining existing platforms to see if functionality can be built in. If not, they look to outside vendors with validated solutions. "Has it been done elsewhere, has it been validated?" Landman said. Only when that fails do they build it themselves. "The biggest nightmare is to sign the contract and (find) it's not working," Mishuris said. "That divorce requires a lot of time, effort and money."

Building in-house comes with its own complexity. "It takes a lot to do it ourselves," Mishuris said. "We are not a software company. I would much rather buy something than build our own." The panel agreed that a trusted network of colleagues can help advise on these decisions. "That network is incredibly important," Mishuris added. "No one thing makes or breaks a decision." The panel's discussion reflects the kind of strategic decision-making covered in resources on AI for Executives & Strategy.

Cost and co-learning: Why vendor partnerships need guardrails

Cost can spiral quickly. "The cost connection can get out of control quickly. You can be looking at a very large bill you did not plan for," Mishuris said. Pandita emphasized that vendors must understand the specific problem the health system aims to solve. If the data does not represent the patient populations the hospital serves, the product falls short. She called for "co-learning" between vendors and providers. "AI will not fix broken workflows," Pandita said, "and AI will not fix broken AI." Day agreed that partnerships need to be a two-way street. "AI can be a superpower," he said, "and also exacerbate weaknesses."

Why this matters for healthcare professionals

For chief medical information officers, clinical informatics leaders, and IT decision-makers in healthcare, the panel's insights underscore a shift: AI is no longer a discrete purchase but a continuous governance challenge. Failing to monitor AI's spread can open data security holes and balloon budgets. Building internal expertise in AI lifecycle management and vendor evaluation is now as critical as clinical quality measures. The consensus among experts points toward stronger collaboration with vendors, rigorous credentialing, and reliance on peer networks - practical steps to avoid the costly divorces and shadow IT pitfalls that can derail an AI strategy.