Beyond the AI pilot: Four questions leaders should ask before scaling

Scaling AI in healthcare takes more than a promising pilot. The harder challenge is turning that promise into technology that works reliably across the enterprise and creates meaningful clinical, operational and financial value. 

A proof of concept can demonstrate that an AI model works. Scaling it across a health system requires something bigger: understanding the workflow around it, ensuring data can be trusted, establishing clear governance and accountability, preparing people for change and defining what success actually looks like. 

These are not simply technology decisions. They are operating decisions. 

That transition from possibility to impact is something I’ll be discussing with other healthcare technology leaders at HLTH 2026. Our panel will explore lessons organizations are learning as they move AI beyond proof of concept and toward enterprise-wide clinical and operational impact.

One question I keep coming back to is simple: What problem are we actually trying to solve? It sounds obvious, but it can be easy to lose sight of when the conversation begins with a technology rather than the work itself. 

I recently saw a similar philosophy articulated by Muhammad Siddiqui, vice president and CIO at Reid Health, in an American Medical Association article. His team asks four questions in front of major technology initiatives:  

• Does it solve a real problem affecting patient access and safety?
• Is the workflow ready and the data reliable?
• Are the right governance, security and operational ownership in place?
• How will the organization measure the value of the technology? 

I think those questions provide a useful framework not only for selecting technology, but also for determining whether an AI initiative is truly ready to scale.

An AI pilot can be technically successful and still fail to create meaningful value. That is why I don’t think the starting point should be, “Where can we use AI?” It should be, “Where is there a significant problem that technology can help us solve?” 

Look for work that creates unnecessary burden, rework, delays or friction. Then ask what would materially change if that burden were reduced. In healthcare information systems, for example, AI can be applied across clinical documentation, coding and revenue cycle workflows. But automation itself is not the outcome. The important question is whether it improves the underlying work. 

One potential example comes from AI-enabled clinical documentation improvement, where a customer experienced clinical documentation integrity (CDI) efficiency equivalent to 4.6 additional CDI RNs, a 10-fold increase in potential opportunity from expanded all-payer review, a 9% increase in query response rate, and a 20% improvement in patient safety indicator (PSI) reporting accuracy.  

We should not scale AI simply because a pilot proves the technology can perform a task. We should scale it because there is evidence that it improves an outcome that matters. 

AI does not operate independently from the environment around it. If a process is fragmented, adding automation can simply accelerate fragmentation. If the underlying data is incomplete or unreliable, scaling AI can amplify those weaknesses. 

Before focusing on the model, leaders need to understand the workflow around it:

• Where does the information originate?
• Who acts on it?
• What happens when the AI is uncertain?
• Where does human judgment remain essential?
• How does AI change the work of the people using it? 

Trust also depends on more than accuracy. I think of trust across several dimensions. AI needs to perform reliably, fit naturally into the workflow and complement human expertise rather than get in its way. If AI adds another burden or another disconnected step, even technically sophisticated technology will struggle to earn adoption. 

That reflects a recurring theme in our work: clinical trust depends on accuracy, explainability and transparency, while workflow trust requires AI to simplify work rather than add complexity. The objective is not to insert AI into an existing process. It is to determine how people, technology and workflows can work better together. 

As AI moves from an isolated pilot into enterprise operations, governance becomes increasingly important. But the question is bigger than who owns the technology. Who owns the outcome? 

Health system leaders should establish clear answers to questions such as:

• Who is accountable for monitoring performance?
• When is human review required?
• How are exceptions handled?
• Can decisions be traced and audited?
• What happens when performance changes?
• What evidence is required before automation expands? 

These decisions become especially important when AI touches clinical, financial or operational processes. The emerging model may also change how human expertise is applied. Rather than reviewing every case, employees may increasingly focus on exceptions, risk and governance. Existing strategy work anticipates this transition toward exception-based workflows spanning areas such as documentation, coding, quality, denials, prior authorization and utilization management. 

Organizations therefore need to be intentional about where AI can act autonomously, where people must review exceptions and how performance will be monitored. Governance cannot be something added after successful experimentation. It has to be part of the scaling strategy from the beginning. 

Finally, leaders need to answer a deceptively difficult question: Did the AI actually make the work better? Technical performance matters, but it is only part of the picture. Depending on the use case, measures might include: 

• time returned to clinicians or staff
• reductions in manual work and rework
• workflow adoption
• quality and accuracy
• clinician or employee experience
• financial and operational impact
• denial reduction or revenue realization 

Solventum's existing AI adoption framework similarly calls for measuring outcomes such as reduced burnout, time savings, productivity improvements and return on investment rather than simply evaluating the technology itself. For example, a Midwest health system automated outpatient coding while maintaining accuracy.  

The right measures will vary by use case. What matters is defining success before scaling rather than searching for evidence of success afterward. This represents an important evolution in how healthcare organizations evaluate AI. The question is becoming less about whether a model performs well in isolation and more about whether the technology contributes to better performance across the workflow and organization. 

The healthcare AI conversation is evolving. The question is no longer simply whether AI can perform increasingly sophisticated tasks. Leaders now have to determine where AI deserves a place in the enterprise and what needs to be true for it to succeed there. That requires moving beyond experimentation and asking tougher questions about the problem being solved, workflow and data readiness, governance and accountability. 

The goal should not be to accumulate AI projects. It should be to turn the right AI initiatives into sustainable ways of working that make healthcare better for the people delivering and receiving care.

Hari Bala is chief technology officer, health information systems, at Solventum.