Most Health System AI Projects Fail to Deliver Results. Here's What's Different at Those That Succeed
Up to 80% of health system AI projects never move beyond pilot stage. Roughly 95% fail to generate a return on investment. Hospital leaders report repeated failures and struggle to evaluate vendors before committing resources.
A working session organized by the Digital Medicine Society and Qualified Health brought together leaders from 12 health systems and six AI developers to examine why. The discussion revealed five operational practices that separate systems capturing real value from those stuck in endless pilots.
Anchor AI decisions to enterprise strategy
Most health systems make AI decisions outside their core strategic framework. Investments follow vendor activity, peer adoption, or internal demand rather than a clear line to system-level objectives.
In academic medical centers, competing priorities-care delivery, research, education-each carry different incentives and timelines. Across all systems, financial and clinical goals are not consistently aligned. The result: AI portfolios expand without coherent rationale.
Systems delivering measurable results define a small number of enterprise priorities and use them to guide selection, implementation, and evaluation. Common anchors include improving access, stabilizing margins, and addressing workforce constraints. These priorities determine which use cases move forward, how success is measured, and whether a solution scales or stops.
Treat prioritization as an ongoing discipline
Requests for AI solutions come from every direction-executive teams, frontline clinicians, operational leaders. Taken together, they exceed most systems' capacity to act.
Prioritization requires trade-offs that many organizations avoid. Saying yes to one use case means delaying or declining others, even when each appears valuable alone. Without discipline, portfolios accumulate pilots without a clear path to scale.
Leading systems focus on use cases with a defined implementation path and clear links to enterprise objectives. Early success clusters in revenue cycle management, ambient documentation, referral management, and targeted clinical workflows like cardiology. Some systems are also reducing backlogs and expanding capacity to improve access.
Funding source matters. Centrally funded initiatives align better with enterprise priorities. Department-driven efforts move faster but may not scale. Without coordination, they work at cross purposes.
Move governance from principles to processes
Governance is often discussed in abstract terms-principles, oversight structures, risk frameworks-without translation into operational decisions. This introduces delay without improving decision quality.
Effective governance consists of three functions: selecting solutions, implementing them effectively, and ensuring sustained performance without introducing unacceptable risk. When these are not clearly defined, accountability diffuses and decisions stall.
Systems making progress treat governance as an operational capability, not an oversight function. They establish structured pathways for evaluation, defined environments for piloting, and clear processes for monitoring performance over time.
A critical gap: defining where vendor responsibility ends and health system responsibility begins. As AI becomes embedded in products, vendors increasingly provide monitoring and validation. Health systems retain accountability for performance in their environment, but that accountability is often unclear.
Accelerate decision cycles to match AI's pace
The volume and speed of AI innovation exceed most health systems' capacity to evaluate and adopt solutions. Many still rely on monthly committee cycles where a missed meeting delays decisions.
Meanwhile, expectations for performance are rising. AI is being introduced into environments requiring reliability across teams and workflows, with ongoing monitoring and adjustment. These demands strain operating models not designed for continuous iteration.
Leading systems use smaller, empowered groups to evaluate and approve use cases. They shorten testing cycles and create tighter feedback loops between deployment and monitoring. Vendor relationships are shifting too-long-term agreements with fixed escalation structures no longer fit the pace of development. Systems are seeking shorter terms, greater flexibility, and clearer performance accountability.
Measure value across multiple domains
All participating health systems said quantifying AI value proved harder than expected. While implementations deliver clear benefits, translating those benefits into metrics that support decision-making has been complex.
CFOs prioritize measurable return on investment. Chief medical officers focus on workflow integration and quality. Operational leaders track throughput and capacity. These perspectives are not misaligned-they are simply different.
Systems making progress define value across a small number of domains and track performance across all of them. Value clusters today around financial health, provider experience, and access to care.
This requires moving beyond point-in-time evaluation. AI systems must be monitored over time with clear accountability for real-world performance. Without this, initial gains degrade and confidence erodes.
Payment and regulation set the boundaries
Better execution within health systems can capture substantial value. But realizing AI's full potential requires alignment between payment models, regulatory expectations, and the outcomes these technologies are designed to deliver.
Health systems respond predictably to financial incentives. Under fee-for-service models, investments that expand access or improve efficiency can compete with existing revenue streams. Under value-based models, those same investments align with financial performance.
This shapes which use cases advance. Applications improving revenue cycle performance or documentation are easier to justify because returns are immediate. Applications improving clinical outcomes or reducing burden require different financial logic, even when their long-term impact is significant.
Regulatory requirements for safety and accountability are necessary. But how they are operationalized will determine whether they enable or delay deployment. Health systems face increasing responsibility for monitoring and oversight, often without clear standards or sufficient infrastructure.
The execution gap
Leading health systems are already capturing value from AI. What distinguishes them is how they make decisions about what to prioritize, how to implement, and how to sustain performance.
These approaches remain inconsistent across organizations. Variability in strategy, governance, and measurement limits the ability to scale what works. Much published literature on health AI remains high-level, focused on generalized frameworks or narrow evaluations of single solutions. It does not capture how AI is actually deployed today, where multiple use cases interact within the same system and performance depends on portfolio-level decisions, not on individual tools.
The work required now is operational: establishing capabilities to prioritize competing demands, implement across settings, and maintain accountability for performance over time. Health systems that master these disciplines will see results. Those that don't will continue cycling through pilots.
Learn more about AI for Healthcare and AI for Executives & Strategy to support your organization's approach to AI implementation.
Your membership also unlocks:
