Healthcare experts say sustainable AI requires smaller edge models and automated governance

Hospitals and health systems are adopting AI for clinical decision support and operational automation, but experts at the HIMSS AI in Healthcare Forum in Boston warned that true AI sustainability will require smaller models deployed at the edge and automated governance to ensure quality outputs. They also cautioned that a growing digital divide could soon separate care quality by system size and resources.

AI in the EHR and diagnostic support

Stanford Healthcare has integrated a large language model directly into its electronic health record workflows, giving clinicians the ability to query patient charts in real time. Dr. Michael Pfeffer, the health system's chief information and digital officer, recalled an email from a cardiology fellow who said the tool, called ChatEHR, "changed my life."

Dr. Eric Alper, chief quality officer at UMass Memorial Health, said he sees AI's potential in advanced diagnostic support and closing care loops. "I'm really excited about the way that AI will improve the quality of care that we're providing in the hospital, outside of the hospital, population health, and make sure that patients are getting the screening that they need - with agents looking at charts and closing loops," Alper said. He believes that will help get closer to Six Sigma for diagnostic care.

Both leaders agreed that AI is here to stay, but the industry is still in the hype cycle. "There are so many opportunities, so many things that are currently coming our way," Pfeffer said. Each day brings a new model or platform. The key, he said, is to focus on outcomes and how to measure them. "We're not quite there yet."

The digital divide and governance challenges

Alper noted that UMass Memorial, a "underbedded" system with full emergency rooms, is now asking not whether to use AI, but where and how. "It's about trying to figure out, What is your strategy for scribes? What's your strategy for the in-basket? What's your AI strategy for agents? Is it build versus buy?" he said. The discussion underscored the need for a coherent AI for Healthcare strategy that balances build-versus-buy decisions.

"How are they going to be able to access these kinds of tools, which will transform healthcare?" Alper said. "We're creating a digital divide for the haves and have-nots in the community." Pfeffer agreed, describing an unfair division in patient care: an ambulance that turns left to a well-funded hospital with advanced AI tools offers different care than one that turns right to a facility without them.

Both leaders said they rely on EHR vendors to help deliver some AI tools. Alper added that his organization has implemented a governance process for AI that evaluates risk up front, but it lacks the resources to monitor models in real time after they go live.

Rethinking human-in-the-loop

Pfeffer warned that clinicians cannot realistically audit every piece of AI-generated text. "You see this summary once - it looks good. You see it twice - it looks good, and then you trust it forever," he said. "And we cannot put AI out there that clinicians have to check." Checking does not save time, making the concept of human-in-the-loop "a question mark."

Instead of having humans double-check every output, the future state will rely on secondary AI agents to automatically monitor and flag inaccuracies, which are then elevated to humans. Alper envisions AI providing clinicians with active diagnostics to help eliminate misdiagnoses and delayed diagnoses. But he acknowledged that some AI investments won't generate direct financial returns-they are simply the right thing to do for patients.

Smaller models and edge computing for sustainability

The cost of running large AI models is a growing concern. Pfeffer said that identifying every patient who needs a colonoscopy would overwhelm the available gastroenterologists, so utility and cost must be balanced. "Overall, it's got to be positive because if it's negative, it's not sustainable," Alper said.

Pfeffer pointed to edge computing and smaller, customizable models as a path forward. "We're going to have to move from throwing everything at these large language models to much more thoughtful, customizable, smaller and at-the-edge use cases for this to make it more sustainable," he said. He also predicted that autonomous coding will eventually become standard, removing it as a payment differentiator.

Why this matters for healthcare professionals

Healthcare leaders must plan for AI governance that includes automated monitoring, not just upfront risk assessments. The digital divide means smaller organizations need vendor partnerships and shared infrastructure to access advanced tools. And as AI costs rise, the shift to smaller, edge-based models will determine whether clinical AI can scale without breaking budgets. For clinicians, the message is clear: AI outputs will increasingly be trusted, but only if systems are built to catch errors automatically.