AI Auditing and Monitoring in Healthcare: Governance Should Not Stop at Deployment

Healthcare already has a model for managing risk. We need to apply it to AI.

For more than 40 years, I have worked in healthcare compliance, coding, auditing, payment integrity, and regulatory risk. One principle has remained consistent throughout every major change in healthcare: You cannot manage risk with policy alone.

We develop policies, train people, and implement controls. Then we audit and monitor to determine whether those controls are actually working. When we find a problem, we investigate the root cause, implement corrective action, provide education where necessary, and monitor again to make sure the problem has been corrected.

That is fundamental compliance practice. As I continue studying AI leadership and building AI-enabled workflows, something has become increasingly clear to me: Why would we govern AI any differently? AI may be new technology, but the fundamental principles of effective compliance are not. Healthcare organizations need to apply those principles throughout the AI lifecycle, not simply during implementation.

AI Governance in Healthcare Requires a Different Approach to Risk

One thing I appreciate about continuing my education in AI leadership is that it challenges assumptions, even after decades of experience. AI introduces risks we have not traditionally managed in quite the same way. An AI system may provide a perfectly written answer that is wrong, reach different conclusions when essentially the same question is phrased differently, or fail to recognize an important exception. Its performance may also change after a model update, or it may rely on incomplete, outdated, or biased information.

Perhaps most concerning, AI can sound extremely confident while doing it. That confidence can influence how people respond to its recommendations, particularly when the output appears authoritative or the organization has invested heavily in the technology. Responsible AI therefore cannot be built on trust alone. It must be built on controls that can be tested, monitored, and corrected.

AI Transparency in Healthcare Starts With “Show Me the Source”

Throughout my career, one of my most frequent questions has been: Show me the source. If we are advising a healthcare organization about a regulatory requirement, coding rule, payer policy, documentation standard, or compliance obligation, I want to know where the information came from. Is the source authoritative and current? Does it actually apply to the situation? Has the guidance changed? Is there conflicting information, and can we defend our interpretation?

AI does not change that standard. If anything, it makes the discipline even more important. An AI-generated answer should never become authoritative simply because it sounds authoritative. The confidence of the AI’s language should never be confused with the strength of the evidence behind it.

Transparency, however, is not enough on its own. Providing a source does not automatically make a workflow safe. Someone still needs to determine whether the source is correct and current, whether it applies to the situation, whether AI has interpreted it appropriately, and whether uncertainty or conflicting guidance requires escalation. Transparency needs to become an operational control, supported by source validation, documentation, defined review responsibilities, and traceability.

This is where healthcare compliance professionals have an enormous amount to contribute to AI governance. We already understand the importance of authoritative sources, defensible interpretations, and evidence that supports a decision.

The Healthcare Compliance Framework for AI: Policies, Controls, and Auditing

Think about the basic structure of an effective compliance program. We establish expectations through policies and procedures, educate people through training, and implement controls to reduce known risks. We then conduct auditing and monitoring to determine whether those controls are working as intended.

When something goes wrong, we perform root-cause analysis (RCA), implement a corrective action plan (CAP), and re-audit or monitor to determine whether the corrective action actually worked. This is not a one-time exercise. It is a continuous process of identifying risk, responding to problems, and validating that the response was effective.

That familiar cycle translates remarkably well to AI governance:

Policy → Training → Controls → Auditing and Monitoring → Detection → Root-Cause Analysis → Corrective Action Plan → Re-audit.

Healthcare organizations do not need to abandon established compliance disciplines to govern AI. They need to adapt those disciplines to systems that may change, operate at scale, and increasingly participate in workflows with less direct human intervention. The goal is to make AI governance part of the operating environment rather than a separate policy exercise.

AI Auditing and Monitoring Must Continue After Deployment

Before deploying a high-risk AI system, organizations will understandably spend time testing it. But pre-deployment testing only tells us how the system performed before deployment. It does not tell us how it will continue to perform six months later, after the model changes, the underlying information is updated, employees develop new usage patterns, or the workflow itself evolves.

AI systems, models, data, sources, regulations, workflows, and vendors can all change. Users may find new ways to interact with the technology, and the environment surrounding AI is constantly moving. A system that performed acceptably during initial testing may behave differently as these conditions change.

AI governance therefore cannot be Test → Approve → Deploy → Done. It needs to become Test → Approve → Deploy → Monitor → Audit → Learn → Correct → Re-test. This is a fundamentally different operating model. Deployment is not the end of the governance process; it is the beginning of the organization's ongoing responsibility to understand how the system performs in practice.

What Should Healthcare Organizations Monitor in AI Systems?

The answer depends on the use case and its level of risk. For a regulatory intelligence workflow, for example, I would want visibility into accuracy, source reliability, consistency, bias, escalation, human overrides, and unexpected behavior. The organization needs to know whether AI is interpreting information correctly, relying on authoritative and current sources, and recognizing when it should stop and request human review.

Monitoring should also consider whether the system reaches materially different conclusions when similar questions are asked, whether certain clients, provider types, specialties, payers, or settings are treated differently without a legitimate reason, and whether the technology is doing something the organization did not anticipate. These questions should be translated into measurable criteria appropriate to the workflow, with clear thresholds for investigation and escalation.

Those are not merely technical performance metrics. They are compliance monitoring indicators. They help the organization determine whether AI is operating within its approved boundaries, whether the controls are effective, and whether intervention is necessary before a problem becomes more widespread.

Human Overrides Are Valuable AI Risk Indicators

Suppose an experienced compliance professional repeatedly overrides an AI recommendation. That should tell us something. The question should not simply be why the employee is not following the AI. We should also ask: Why does the AI keep getting overridden?

Perhaps AI is missing an important regulation, a payer-specific rule is absent from its knowledge base, the underlying information is outdated, or the prompt or workflow is poorly designed. The employee may be identifying an exception that the system does not recognize. It is also possible that the employee is mistaken, which is why the organization needs a process for reviewing the disagreement rather than assuming either the human or the AI is automatically correct.

This is why human expertise remains so important. AI can process enormous amounts of information very quickly, but experienced professionals often recognize when something simply does not look right. That judgment should not be designed out of the workflow. It should become part of the control environment.

Organizations should track patterns in overrides, investigate recurring disagreements, and determine whether the findings point to a knowledge gap, a model limitation, an unclear policy, a workflow problem, or a need for additional training. Human intervention can provide valuable evidence about how well the AI system is performing in the real world.

AI Incident Management Requires Root-Cause Analysis

Healthcare compliance professionals already understand root-cause analysis. We do not simply correct an individual error; we investigate why it happened. The same approach should apply to AI. If an AI system produces an incorrect recommendation, the answer cannot simply be, “The AI got it wrong.” Leaders need to understand what caused the failure and whether the same conditions could produce additional errors.

The source may have been outdated, important information may have been missing, or the source itself may have been incorrect. AI may have misinterpreted accurate information, failed to recognize an exception, or produced an unreliable conclusion because of ambiguity in the underlying rule. The cause could also involve a model update, a poorly designed prompt, a bypassed control, excessive human reliance, or an inadequate escalation threshold.

The error is the event. The root cause is what governance needs to understand. Without a structured investigation, organizations risk treating the immediate symptom while leaving the underlying problem in place. Root-cause analysis should identify the contributing factors, establish what needs to change, and provide a documented basis for corrective action.

AI Corrective Action Plans Must Include Validation

Once the cause is understood, organizations need a structured response. A corrective action plan may involve updating the knowledge source, changing the prompt or workflow, adjusting an escalation threshold, adding human review, retraining employees, updating policy, changing access permissions, revising monitoring criteria, working with the vendor, reducing AI autonomy, or temporarily stopping the workflow.

The appropriate action depends on the cause and the level of risk. A documentation gap may require a knowledge update, while a problem involving unauthorized actions may require stronger access controls or a reduction in the agent's authority. Corrective action should address the identified cause rather than defaulting to retraining employees whenever an AI process fails.

Then comes the step organizations sometimes forget: Did the corrective action work? Re-audit, monitor, measure, and verify. A CAP is not complete simply because an action was documented or a policy was revised. The organization needs reasonable evidence that the identified problem has been addressed effectively and that the corrective action has not introduced another risk.

AI Risk Management in Healthcare Must Be Proportionate to Risk

Not every AI application needs the same level of oversight. Using AI to help draft an internal meeting agenda presents a very different risk from using AI to influence patient care, coding, billing, claims, payment integrity, utilization management, compliance decisions, employment decisions, or other consequential activities.

The monitoring structure should reflect the potential impact of the use case. Organizations should consider the consequences of an incorrect output, the degree of autonomy granted to the system, the sensitivity of the information involved, and the ability of a human to identify and correct an error before harm occurs.

I continue to think about AI autonomy as Assist → Recommend → Decide → Act. As AI moves further to the right, governance, monitoring, and auditability should become stronger, and human authority should become clearer. An AI system that assists a professional does not necessarily require the same controls as one that makes a recommendation or takes action across a workflow.

The important point is that governance should reflect what AI is authorized to do and what could happen if it gets something wrong. Capability does not automatically justify authority. Organizations need to define the boundaries before deployment and reassess them as the technology, workflow, or risk changes.

Healthcare AI Policies Must Become Operational Controls

Healthcare organizations have spent years developing compliance policies. Today, most policies are not sitting in binders on shelves. They live in electronic policy-management systems, intranets, shared repositories, and compliance platforms. But the underlying risk remains the same: a policy that does not influence actual behavior provides limited protection.

A beautifully written AI policy that employees rarely use and that does not shape the workflow is not enough. AI governance must move from the policy document into the operating environment. The policy establishes expectations, training helps people understand them, and controls put those expectations into practice. Auditing and monitoring tell us whether the controls are working, root-cause analysis helps explain why they failed, and corrective action addresses the problem.

Re-auditing then helps determine whether the organization actually fixed it. That is governance in action. The objective is not simply to demonstrate that an AI policy exists. It is to demonstrate that the policy is reflected in the way the system is configured, used, monitored, and corrected.

Healthcare Organizations Should Build AI Governance Before an Incident

Healthcare has learned this lesson many times. It is much easier to identify a small compliance issue early than to address a systemic problem after it has affected thousands of claims, patients, providers, or transactions. AI will be no different. A small pattern of inaccurate recommendations today can become thousands of incorrect actions when an AI capability is scaled.

This is why one of my guiding principles for AI transformation continues to be: Don’t wait for perfection, but don’t scale chaos. Organizations should establish a controlled pilot, define success and risk criteria, monitor real-world performance, investigate exceptions, and correct problems before expanding the use case.

The goal is not to delay innovation until every uncertainty has been eliminated. It is to build the governance infrastructure needed to learn safely, respond when something goes wrong, and demonstrate that the system is performing within acceptable boundaries. Pilot, monitor, audit, learn, correct, and then scale.

The ProCode Perspective: AI May Be New, but Good Compliance Isn't

The more I study AI leadership, the more I recognize how much traditional healthcare compliance expertise belongs in this conversation. AI governance does not require us to abandon the disciplines that have protected healthcare organizations for decades. It requires us to adapt them.

Compliance professionals understand controls, auditing and monitoring, documentation, regulatory interpretation, investigations, root-cause analysis, and corrective action plans. Perhaps most importantly, experienced professionals understand that when something does not look right, you investigate before it becomes something much bigger.

AI may identify risk faster than we ever could, analyze more information than a human team could reasonably review, and eventually automate portions of work we cannot imagine automating today. None of that eliminates the need for oversight. It increases it.

The future of responsible healthcare AI will require good technology alongside strong policies, meaningful training, designed controls, continuous auditing and monitoring, clear human accountability, root-cause analysis, corrective action, and verification that those actions actually worked.

AI may be new. Good compliance isn't. One of the greatest opportunities healthcare compliance leaders have right now is to bring decades of proven discipline into the design of the AI-enabled organizations we are building next.