AI ROI in Healthcare: Track Value, Not Activity

Why healthcare leaders need to measure what AI actually delivers—not simply how much work it performs

Healthcare organizations are investing heavily in artificial intelligence with expectations of greater productivity, faster workflows, reduced administrative burden, and lower costs. AI is already being used across coding, auditing, revenue cycle, documentation, compliance monitoring, communication, and other areas of healthcare. As adoption grows, however, healthcare leaders need to become more disciplined about determining whether those investments are actually delivering the value they were intended to create.

Measuring AI ROI in healthcare requires looking beyond activity and productivity to determine what actually changed because the technology was implemented. An organization can report thousands of claims reviewed, documents generated, hours saved, or employees using an AI tool and still have little evidence that the organization is better off. Those numbers demonstrate activity. They do not necessarily demonstrate value.

That distinction matters because AI can make an organization faster without making it better. It can increase output without improving quality, reduce time without reducing cost, or automate a process while introducing new compliance risk. The real measurement challenge is not determining whether AI is doing something. It is determining whether it is producing the outcome the organization expected when it approved the investment.

Productivity Is Not the Same as Value

One principle I return to repeatedly in my AI leadership work is simple: track value, not activity.

Healthcare compliance and revenue cycle professionals have dealt with this distinction for years. Completing 100 audits does not necessarily mean compliance improved. Processing claims faster does not mean the claims were processed correctly. Sending more education does not demonstrate that provider behavior changed. Activity measures tell us that something happened, but they do not necessarily tell us whether the intended outcome was achieved.

AI makes this problem more pronounced because it can generate measurable activity at a scale that is difficult to achieve manually. Organizations can quickly report how many claims an AI system reviewed, how many documents it summarized, how many recommendations it generated, or how many hours it theoretically saved. Those numbers may demonstrate adoption and utilization, but they do not answer the more important question: What changed because we used AI?

Consider time savings. If an AI application reduces a five-hour process to two hours, the organization has created three hours of capacity. That does not automatically mean it saved three hours of labor cost. The value depends on what happens with that capacity. If the employee uses the time to address a backlog, perform additional audits, support more complex investigations, improve monitoring, or absorb increased volume without additional staffing, the organization may have created meaningful value. If the same work simply gets completed earlier, the operational improvement may be real, but the financial benefit may be very different from what was originally projected.

That is the difference between productivity and value realization. Productivity tells us what the technology enabled people to do. Value realization tells us what the organization actually gained as a result.

Start With the Business Case

One of the most important questions leaders can ask about an AI initiative is also one of the simplest: Why did we approve this technology in the first place?

The answer should determine what the organization measures. If the business case was based on reducing audit preparation time, then preparation time should be measured before and after implementation. If the objective was to improve coding accuracy, accuracy should be a primary measure. If the organization expected to reduce denials, it should measure denial rates and the reasons behind those denials. If the purpose was to increase auditor capacity, leadership should determine whether that capacity actually increased and how it was used.

The same principle applies to compliance monitoring. If AI is intended to identify regulatory changes more quickly, the organization should determine whether meaningful changes are being identified, validated, communicated, and incorporated into operations more effectively. If the technology is intended to improve documentation, the organization needs to determine whether documentation actually improved rather than simply measuring how many documents the system touched.

The business case should become the measurement plan.

That is particularly important when evaluating AI ROI in healthcare, because the value of an implementation cannot be separated from the quality, compliance, and operational risks it creates. Organizations should establish the expected outcome before deployment and identify how they will know whether that outcome was achieved.

Otherwise, it becomes very easy to approve AI for one reason and later declare it successful based on something entirely different simply because that alternative metric is easier to count.

AI ROI in Healthcare Has to Include Risk

Traditional return-on-investment calculations still matter, but healthcare organizations cannot evaluate AI based solely on hours saved or dollars generated. An AI implementation can produce an attractive operational result while introducing quality, compliance, privacy, security, or financial risk that was not included in the original calculation.

Consider an AI-assisted coding system that increases productivity by 30 percent. On the surface, that may appear to be a strong result. But if subsequent monitoring shows that the system is also contributing to unsupported higher-level coding, the productivity number does not tell the whole story. The organization may now be dealing with overpayments, refunds, payer scrutiny, corrective action, or other compliance consequences.

The productivity gain has to be evaluated in the context of those consequences.This does not mean AI should never introduce risk. Every technology investment carries risk. The important questions are whether the organization understood the risk, established appropriate controls, and determined that the expected benefit justified the exposure.

For healthcare, I believe the more meaningful question is not simply whether AI made the organization faster or less expensive. It is whether the technology created the value the organization expected at an acceptable level of risk. That is a much more meaningful definition of AI ROI.

Measurement Is Part of AI Governance

Measurement should not happen months after implementation when leadership wants to determine whether an AI investment paid off. It should be part of the governance process from the beginning.

When an organization establishes performance expectations before deployment, monitors results against those expectations, investigates meaningful variances, and takes corrective action when necessary, measurement becomes a governance control. It provides evidence that the technology is performing as intended and creates an opportunity to identify problems before they become larger compliance, financial, operational, or patient-impact issues.

This should be familiar to healthcare compliance professionals. We establish expectations, monitor performance, identify variances, investigate root causes, implement corrective action, validate whether the correction worked, and reassess when circumstances change. AI does not eliminate that discipline. It makes it more important.

That is especially true as AI takes on greater responsibility. A system that assists an employee creates a different measurement challenge than a system that recommends an outcome. A system that makes a decision or takes action creates another level of responsibility. As AI moves through Assist → Recommend → Decide → Act, organizations need increasingly meaningful evidence that the technology remains within its intended purpose, produces the expected outcomes, and operates within its authorized boundaries.

The higher the level of AI authority, the less acceptable it becomes to rely on a general statement that the technology is “working well.” Leaders need evidence.

Look Beyond the AI Vanity Metrics

Healthcare leaders should become increasingly skeptical of AI metrics that sound impressive but provide little insight into whether the organization achieved its intended outcome.

The number of employees using an AI tool may demonstrate adoption. The number of claims reviewed may demonstrate utilization. The number of summaries generated may demonstrate output. Hours saved may demonstrate productivity.

All of those measures can be useful. None of them, by themselves, demonstrates value. The more important question is what changed as a result. Did the organization improve its financial position? Did quality improve? Did compliance risk decrease? Did employees use the capacity created by AI for higher-value work? Did turnaround times improve in a way that mattered to the organization? Did patients, providers, employees, or other stakeholders experience a meaningful improvement?

And particularly in healthcare compliance, did the work remain accurate, defensible, and compliant?

Speed without accuracy is not transformation. It can simply be faster risk.

This is why AI measurement needs to include more than utilization and productivity. Organizations need a balanced understanding of operational performance, financial impact, quality, compliance, and the effectiveness of the controls surrounding the technology.

When the Numbers Change, Investigate Why

Measurement also becomes important when results do not match expectations. A variance should not automatically be treated as a technology failure. The organization needs to understand what caused the difference.

Perhaps the AI system is not performing as expected. Perhaps the workflow changed. Perhaps employees are using the technology differently than anticipated. Perhaps the data is incomplete. Perhaps a vendor update changed system behavior. Or perhaps the original business case was unrealistic.

Those questions are familiar to anyone who has performed root-cause analysis in healthcare compliance. The goal is not simply to identify that performance changed. It is to understand why, determine whether the underlying cause creates additional risk, and decide what corrective action is necessary.

This is another reason measurement should be built into the AI governance process rather than treated as a financial exercise at the end of implementation. Good measurement tells the organization when something has changed. Good governance determines what to do about it.

If the organization expected AI to reduce denials and denials do not decrease, that should trigger investigation. If AI was expected to improve coding accuracy and audit results show otherwise, that should trigger investigation. If the technology was expected to create additional capacity but employees are not using that capacity for higher-value work, that is also important information.

The answer is not always to abandon the technology. It may be to change the workflow, improve the controls, retrain users, adjust the implementation, or reconsider the original business case.

From AI Adoption to AI Accountability

There is tremendous pressure for healthcare organizations to demonstrate that they are adopting AI. Adoption can easily become a performance objective in itself, particularly when organizations are comparing themselves with peers or trying to demonstrate that they are keeping pace with the technology.

I believe healthcare leaders should resist measuring success by adoption alone.

The goal is not to have the most AI, generate the most output, or automate everything that can technically be automated. The objective should be to create measurable, sustainable, and defensible value while protecting the patients, providers, employees, organizations, and data affected by these systems.

That also creates a more meaningful role for compliance. Compliance should not enter the conversation only after an AI implementation produces a problem. Compliance professionals can help define acceptable performance, identify the risks that need to be measured, establish monitoring thresholds, interpret exceptions, perform root-cause analysis, and determine whether corrective action actually restored the intended control environment.

That is a much more valuable role than simply reviewing an AI policy. AI will continue to create opportunities for healthcare organizations to improve productivity and rethink how work gets done. But the ability to deploy a technology quickly does not eliminate the responsibility to determine whether it is actually producing the intended result.

Healthcare organizations have spent decades developing disciplines for measuring performance, managing risk, investigating variance, and correcting problems. Those disciplines still apply. In fact, as AI becomes more capable and more deeply embedded in healthcare operations, they become even more important.

The organizations that create sustainable value from AI will need to demonstrate more than adoption or activity. They will need to show that their investments produced the outcomes they intended, that the results are measurable, and that the risks remained within boundaries the organization was prepared to accept.

Track value, not activity. And measure the value that matters. That is how healthcare organizations move from experimenting with AI to holding it accountable as a serious business capability.