Proving the Value of AI in Healthcare: Stop Counting Activity and Start Measuring Impact

Healthcare organizations are investing heavily in AI. The real question is no longer how much AI we are using, it is what is measurably better because we are using it.

Artificial intelligence is moving at warp speed. Across healthcare, organizations are launching pilots, purchasing AI-enabled technology, experimenting with copilots and generative AI, and beginning to explore more sophisticated agentic workflows.

But as we move from experimentation to implementation, leadership needs to ask a harder question: Is AI actually creating value?

The number of AI tools an organization owns is not a measure of transformation. Neither is the number of employees using AI or the number of pilots launched. Those are measures of activity, while what ultimately matters is impact. For leaders, measuring AI value in healthcare means identifying what is measurably better because the technology is being used.

Stop Counting Activities. Start Measuring Impact.

One of the most important shifts healthcare leaders can make is moving from measuring AI adoption to measuring what AI actually changes.

Instead of asking how many employees are using AI, leaders should ask whether AI has improved their ability to do their jobs. Instead of counting implemented workflows, they should determine whether those workflows are faster, more accurate, more consistent, or less costly. And instead of reporting only the number of hours saved, they should ask what the organization accomplished with the capacity it created.

That is a very different measurement conversation.

AI Activity → Operational Performance → Business Outcome

Activity tells us AI is being used, operational performance tells us whether it is improving the work, and business outcomes tell us whether those improvements are creating organizational value. Healthcare organizations need to measure all three—but ultimately, business outcomes are where value is realized.

Start With the Business Outcome

Before implementing AI, leadership should be able to answer a fundamental question: What are we trying to improve?

Depending upon the organization and workflow, the desired outcome might be revenue growth, greater cost efficiency and productivity, improved patient or client retention, faster time to market, increased innovation, reduced compliance exposure, or greater organizational capacity.

Once we identify the outcome, we can work backward and determine which operational improvements should lead us there.

This is important because business outcomes often take time to materialize. We need earlier indicators that tell us whether we are moving in the right direction.

Operational Drivers: The Early Signals of AI Value

This is where operational drivers become particularly important. Think of them as leading indicators that help us determine whether AI is changing the organization’s underlying performance before the ultimate financial or strategic outcome is fully visible. Four are particularly relevant to healthcare.

1. Decision Quality

Is AI helping people make better decisions? Answering that question requires looking at accuracy, error rates, human overrides, approval rates, consistency, completeness of information, and the quality of recommendations.

For example, an AI system may help a compliance professional analyze a regulatory change significantly faster, but speed means very little if the professional regularly has to correct the AI's conclusions.

Efficiency without decision quality is not transformation.

2. Process Velocity

How much faster is the work moving? This can be particularly valuable in healthcare, where professionals spend enormous amounts of time gathering information before they can actually use their expertise. Consider regulatory intelligence. Today, a compliance professional may need to search CMS, OIG, state Medicaid agencies, Medicare Administrative Contractors, payer websites, newsletters, and other sources before determining whether anything significant has changed.

An AI-enabled regulatory intelligence workflow could potentially identify, retrieve, organize, summarize, and prioritize those changes before the professional begins the substantive review.

The organization could then measure the time required to identify a regulatory change, move from publication to internal review, and proceed from identification to client notification. It could also track professional hours required and the volume of sources monitored. Now we are measuring whether the workflow actually improved.

3. Knowledge Leverage

This may become one of the most valuable—and underappreciated—measures of AI transformation.

Healthcare organizations possess tremendous institutional knowledge in policies, audit findings, corrective action plans, regulatory interpretations, training materials, historical decisions, and subject-matter expertise. The problem is that much of this knowledge is fragmented across systems, folders, emails, documents, and individual employees.

AI creates an opportunity to make that knowledge more accessible and reusable.

We should therefore measure whether people are finding answers faster, duplicate research is declining, and previous analyses are being reused appropriately. We should also examine whether employees can access knowledge that previously depended on knowing whom to ask and whether expertise is being preserved when experienced employees leave. That is not simply productivity; it is knowledge leverage. For organizations built around professional expertise, that can be extraordinarily valuable.

4. Employee Empowerment

We should also measure what AI does for people—not simply whether employees are logging into the technology. Are they able to spend more time performing work that requires their expertise? Is AI reducing repetitive administrative work, and are employees confident using the technology appropriately? Are they satisfied with the workflow, able to accomplish more without increasing workload, and spending less time finding information and more time thinking about what it means?

That distinction matters.

The goal should not be to turn people into faster task processors. The opportunity is to give professionals more capacity for judgment, analysis, problem-solving, communication, and strategic work.

Connect Operational Drivers to Business Outcomes

This is where AI measurement becomes much more powerful. We can begin connecting operational improvements to organizational results. For example:

Regulatory research time ↓ → Professional capacity ↑ → More clients supported → Revenue opportunity ↑

Audit preparation time ↓ → Faster audit completion → Lower cost per audit → Improved profitability

Decision quality ↑ → Fewer errors and corrections → Reduced compliance exposure → Lower organizational risk

Knowledge accessibility ↑ → Less duplicated research → Greater workforce productivity → Reduced operating cost

Now leadership can begin seeing the chain between the technology and the business outcome.

ROI Still Matters, but It Isn't Enough

Healthcare executives should absolutely understand the financial return on AI investments. AI carries costs for technology, licensing, integration, implementation, cybersecurity, governance, training, monitoring, workflow redesign, and vendor management. Organizations should understand whether those investments produce financial value. But traditional ROI does not tell the entire story in healthcare.

An AI system could produce significant labor savings while simultaneously increasing coding errors. It could accelerate claim processing while increasing compliance exposure or improve productivity while producing unreliable recommendations. That is why I believe healthcare needs a broader equation: AI Value = Business Impact + Quality + Risk + Human Capacity All four matter.

Quality Must Be Part of the Equation

Suppose an AI workflow reduces regulatory research time by 60%. That sounds impressive, but if human reviewers discover that 15% of the summaries contain material omissions, the efficiency gain suddenly looks very different. Healthcare organizations should measure accuracy, completeness, consistency, source reliability, rework, human correction, missed information, and overall output quality.

AI should not simply allow us to produce more. It should help us produce better.

Risk Is a Performance Metric

This is particularly important in healthcare.

We should be measuring whether AI is reducing risk or creating new risk. That includes tracking unsupported conclusions, hallucinations, outdated information, missing source attribution, human overrides, escalations, privacy incidents, unauthorized AI use, control failures, incorrect automated actions, and vendor-related issues.

One measure I believe will become particularly valuable is the human override rate. How frequently does a qualified professional disagree with the AI—and, more importantly, why? Those disagreements can tell us where the system needs improvement.

Measure the Governance, Too

If we are going to build governance into AI-enabled workflows, we should also measure whether those controls actually work. For example, leaders should ask how often the system stopped an output because a source could not be verified, how many low-confidence conclusions were escalated, and how frequently required human approval was missing. They should also determine how often a governance control identified an issue before an action occurred and, perhaps most importantly, whether the system ever proceeded when it should have stopped.

A governance control that exists but does not work provides false assurance. Healthcare compliance professionals already understand this: controls must be tested, and AI controls should be no different.

Measure Human-AI Collaboration

As AI becomes more agentic, another important measure will emerge: How effectively are humans and AI working together? This is central to what I describe as Human-in-Governance™.

We should understand when AI appropriately escalates, when it fails to escalate, and when humans override its recommendations. We also need to identify which decisions consistently require expert judgment, where AI performs reliably, and where its authority should be expanded or reduced.

The goal is not maximum AI autonomy. The goal is appropriate AI autonomy within clearly defined human governance.

What Happens to the Time We Save?

This is one of my favorite questions because it gets to the heart of AI transformation.

Suppose AI saves a compliance professional ten hours each week. The value is not simply 10 hours × hourly labor cost; the more important question is what happens to those ten hours.

Perhaps that professional now performs additional compliance monitoring, conducts deeper risk assessments, provides more education, supports additional clients, investigates emerging issues, develops policies, or spends more time on complex work requiring professional judgment.

That is not simply time savings. That is capacity creation.

For healthcare organizations facing workforce shortages, increasing regulatory complexity, and rising administrative demands, capacity creation may ultimately become one of AI's greatest sources of value.

Build an AI Value Scorecard

Rather than relying on one ROI number, healthcare organizations should consider a balanced AI value scorecard. Business outcomes should include revenue, cost, productivity, capacity, turnaround time, and financial impact. Operational drivers should capture decision quality, process velocity, knowledge leverage, and employee empowerment.

The scorecard should also measure quality through accuracy, completeness, consistency, rework, and reliability; risk and governance through overrides, escalations, incidents, unsupported outputs, and control effectiveness; and workforce impact through AI literacy, appropriate adoption, employee confidence, and time redirected toward higher-value work. Together, these measures provide a much clearer picture of whether AI is actually transforming the organization.

What Boards Should Be Asking

Boards and executive teams don't need hundreds of technical AI metrics.

They need answers to a handful of important questions. What problem are we solving, what has measurably improved, and what value have we created? How do we know the AI is reliable, what new risks have we introduced, and are our governance controls working? How frequently are humans overriding the system, and are we creating meaningful workforce capacity?

One of the most important questions is also one of the simplest: Would we know if the AI stopped performing as intended?

That is both a measurement question and a governance question.

The ProCode Perspective

As AI adoption accelerates, healthcare organizations need to move beyond measuring whether AI is present. We need to measure whether it is making the organization better.

That means connecting strategy to operations, operations to people, and ultimately people and technology to measurable business outcomes.

AI may allow us to work faster, but speed alone isn't value. It should help us make better decisions, leverage knowledge more effectively, reduce unnecessary work, increase professional capacity, improve quality, manage risk, and ultimately produce better organizational outcomes.

Healthcare leaders should stop asking, “How much are we using AI?” The better question is, “What is measurably better because we are using it?” Because at the end of the day, AI activity is not transformation. Impact is.