AI Risk in Healthcare: Not Every Risk Can Be Managed the Same Way

Why Healthcare Leaders Need to Distinguish Between Operational, Strategic, and Unexpected AI Risk

Healthcare leaders are moving quickly to understand artificial intelligence—what it can do, where it can create value, and how it should be governed. After more than four decades working in healthcare compliance, I have learned that one of the most important disciplines in risk management is resisting the temptation to treat every risk the same way. That lesson is particularly important with AI because the word risk is being used so broadly that it can lose its meaning. If we simply conclude that “AI is risky” and respond by creating an AI policy, we have not necessarily done anything meaningful to understand or manage the actual exposure.

I believe healthcare organizations need to become much more precise about the type of AI risk they are facing. One framework I have been exploring in my AI leadership coursework separates AI risk into three broad categories: operational risk, strategic risk, and novel or unexpected risk. That distinction resonates with me because it reflects a principle that has guided healthcare compliance for decades: you cannot effectively control a risk until you understand what kind of risk you are dealing with. An operational failure may require a control. A strategic risk may require a leadership decision about direction, dependency, or risk tolerance. An unexpected risk may require something different altogether, the ability to recognize that our original assumptions were incomplete and respond before the issue becomes consequential.

AI makes this distinction especially important because the technology is capable of operating at different levels of influence and autonomy. An AI system may assist an employee, recommend a course of action, influence a decision, or eventually take action itself. The risk associated with the technology therefore cannot be separated from how it is being used, what information it can access, what decisions it influences, and what happens when it is wrong. The governance question is not simply whether an AI model is accurate. It is whether the organization understands the risk it has accepted and has the appropriate mechanisms to manage that risk.

Operational AI Risk: Managing What We Know Can Go Wrong

Operational AI risk is the category most organizations will recognize first because it concerns identifiable failures within the technology or the workflow surrounding it. These risks include inaccurate or fabricated outputs, model failures, privacy or data leakage, bias, cybersecurity vulnerabilities, integration failures, immature AI lifecycle management, retrieval-augmented generation failures, and prompt-injection attacks. These are not abstract technology issues when AI is operating in a healthcare environment. The consequences depend on the workflow, but a technical or process failure can quickly become a compliance, financial, privacy, patient safety, or reputational issue.

Consider an AI coding assistant that recommends an incorrect CPT code. The initial problem may appear to be an inaccurate AI output, but the compliance implications can extend into coding accuracy, billing, reimbursement, audit exposure, and potentially regulatory risk. The same principle applies when a generative AI application relies on outdated Medicare guidance, an AI-supported appeal contains a fabricated citation, or a patient communication tool generates inaccurate medical information. An employee entering PHI into an unauthorized AI application creates a privacy and security concern, while an AI agent acting on an incorrect interpretation of a payer policy introduces an additional level of risk because the technology may move from producing information to initiating an action.

This is where healthcare compliance professionals have an important advantage. We already understand how to build controls around high-consequence processes. Validation, access controls, source verification, testing, human review, audit trails, exception monitoring, security controls, change management, and ongoing auditing are not new concepts simply because AI is involved. What is different is the behavior and scale of the technology to which those controls must now be applied.

For operational AI risk, the objective is therefore relatively clear: identify the failure modes, assess their potential consequences, and establish controls capable of preventing, detecting, or containing them. That requires more than approving a technology. It requires understanding how the technology operates within the actual workflow and verifying that the controls continue to function after deployment.

Strategic AI Risk Requires a Different Conversation

Strategic AI risk is different because the technology does not have to malfunction for the organization to experience significant risk. An AI system may perform exactly as designed and still create a dependency, workforce consequence, regulatory exposure, or reputational problem that leadership should have considered before adopting it.

Vendor dependency is a good example. An organization may integrate an AI platform into a critical workflow because the technology provides substantial value. Over time, however, that workflow may become so dependent on the platform that changing vendors becomes difficult or disruptive. The issue is not that the AI failed. The issue is that the organization may have created a dependency without fully considering its long-term implications.

Workforce impact presents another strategic consideration. AI may change the responsibilities of coders, auditors, clinicians, revenue-cycle professionals, compliance teams, and administrative staff. That does not automatically mean AI adoption is inappropriate. It means leadership needs to understand how the organization will operate as those responsibilities change and, importantly, where human expertise remains necessary to exercise judgment and accountability.

Regulatory uncertainty also belongs in this conversation. Healthcare organizations are adopting AI while laws, regulations, payer expectations, professional standards, and enforcement approaches continue to evolve. An organization making an AI investment today therefore needs to think beyond whether the technology meets today's requirements. It needs to consider whether its governance model, contractual arrangements, data practices, and operational dependencies provide enough flexibility to respond when those expectations change.

There is also an organizational trust component. A technically successful AI initiative can still create reputational damage if patients, employees, providers, or other stakeholders believe the organization adopted the technology without adequate safeguards or without considering its implications. Strategic risk therefore belongs at the leadership table. Leaders should be asking whether an AI use case aligns with the organization's mission, what dependencies it creates, how it may affect the workforce, what regulatory exposure could emerge, and whether the organization is pursuing a particular application because it solves a meaningful problem or simply because AI is available.

Strategic risks cannot always be eliminated. They require leadership to understand the exposure, establish an appropriate risk tolerance, prepare for possible consequences, and make deliberate decisions about where the organization is willing to go.

The Hardest Category: Risk We Did Not Know to Look For

Novel or unexpected AI risk is, in my view, the most challenging category because it exposes a limitation in traditional risk assessment. We generally begin by identifying what we know could go wrong and then designing controls around those risks. With rapidly evolving AI, there may be circumstances that were not reasonably apparent when the system was evaluated.

Models may behave differently under conditions that were not part of initial testing. Capabilities may emerge as systems evolve. AI may interact with other applications in ways that were not contemplated during implementation. A vendor update may materially alter system behavior. Multiple organizations may become dependent on the same underlying models or infrastructure, creating concentration concerns that are difficult to see when evaluating a single implementation. As AI becomes more agentic, the range of circumstances a system can encounter also expands.

We need to acknowledge an uncomfortable reality: we will not identify every AI risk before deployment. I do not believe that should become an argument against using AI. It should instead change how we think about governance.

If we know that some risks will only become visible after deployment, then monitoring cannot be an afterthought. It becomes part of the governance architecture. The organization needs the ability to identify unusual outputs, investigate exceptions, recognize patterns of failure, review near misses, track overrides, monitor complaints, observe changes in vendor or model behavior, and establish clear thresholds for escalation.

The goal is not to predict the unpredictable. The goal is to create an organization that is capable of recognizing when reality has departed from the assumptions on which the original risk assessment was based. That is a very different governance objective.

Control, Position, Watch: A Practical Compliance Framework for AI Risk

The three categories of risk lead to three different leadership responses: control the operational, position for the strategic, and watch for (and contain) the unexpected.

I find this framework useful because it prevents organizations from relying on a single response to every AI concern. Operational risk generally calls for controls designed around known failure modes. Strategic risk requires leadership to make decisions about direction, dependency, workforce implications, regulatory exposure, and risk tolerance. Novel risk requires an organization to build the monitoring, escalation, investigation, and response capabilities necessary to identify problems that were not apparent at the outset.

That third response is particularly important. Traditional compliance programs are often designed around known requirements and identifiable risks. AI governance has to preserve those disciplines while adding the ability to detect signals that something is happening outside the organization's original expectations. Monitoring exceptions, unusual patterns, human overrides, inconsistent outputs, complaints, near misses, vendor changes, and model changes can provide information that was simply unavailable during the initial assessment.

This is where my principle of “Build the brakes before you need them” becomes more than a metaphor. Before an AI system is given meaningful authority, the organization should know what conditions require intervention, who has the authority to intervene, how that intervention occurs, and what happens afterward. A control that exists only on paper is not enough. The organization needs to know that the control works.

Human Oversight Cannot Be Reduced to an Approval Step

As AI becomes more autonomous, the phrase “human in the loop” is likely to become increasingly common. I agree that qualified human oversight is essential, but I am concerned when human involvement is treated as satisfied simply because someone is required to click an approval button.

Meaningful oversight requires more than presence. Does the reviewer have the expertise necessary to recognize an error? Can the reviewer see the source supporting the AI's recommendation? Do they understand the limitations of the system? Can they challenge the output? Do they have the authority to override it? Do they know when escalation is required? Can they stop the workflow when the circumstances warrant it?

These questions are particularly important when AI moves from assisting to recommending, from recommending to deciding, and from deciding to acting. The more authority the technology receives, the more deliberate the governance surrounding that authority needs to become.

That is why I increasingly think in terms of Human-in-Governance™, not simply human-in-the-loop. Human-in-Governance means that qualified people retain authority over the boundaries within which AI operates. It means decision rights are established in advance, exceptions have defined escalation paths, and there is a person or function with the authority to restrict, override, or stop the system when necessary. Human accountability should become stronger as AI autonomy increases, not weaker.

“Show Me the Source” Is an AI Control

One of my strongest instincts from decades of healthcare auditing and compliance work is simple: Show me the source.

AI can produce an answer that sounds authoritative without providing a reliable basis for that conclusion. If an AI system tells a healthcare organization that a particular service can be billed, that documentation supports a code, that a regulation requires a particular action, or that a payer policy permits something, I want to know where the conclusion came from.

Is the source authoritative? Is it current? Does it apply to the specific payer, provider, jurisdiction, date of service, and circumstances involved? Did the AI interpret it correctly? Can a qualified person independently verify the conclusion?

Those questions are not academic. In healthcare, the difference between an authoritative source and an outdated, incomplete, or incorrectly interpreted source can have real consequences. The higher the risk associated with the decision, the more important traceability becomes.

For high-risk healthcare applications, therefore, source traceability should be treated as a control, not simply a desirable feature. An organization should be able to establish the evidentiary basis for an important AI-supported conclusion and determine whether that evidence was appropriate for the decision being made. Fluency is not evidence. Confidence is not validation.

AI Risk Does Not Stay in One Category

Another reason this framework is useful is that AI risks can change category over time. A behavior that initially appears novel may eventually become sufficiently understood to manage through an operational control. Conversely, a seemingly minor operational problem may reveal a much larger strategic issue. A vendor update may introduce new capabilities that materially change the organization's risk exposure even though the organization has not intentionally changed the use case.

This is why I do not view an AI risk assessment as a document that can be completed once and filed away. The assessment needs to be revisited when the technology changes, when the workflow changes, when the level of autonomy changes, when the vendor changes the system, or when monitoring identifies behavior that was not part of the original risk assumptions.

The process I would use is: Identify → Classify → Control → Monitor → Detect → Investigate → Correct → Reassess. That cycle should look familiar to healthcare compliance professionals. The technology is new, but the underlying discipline is not. What has changed is the speed, scale, interconnectedness, and potential autonomy of the environment in which we are applying those disciplines.

Do Not Wait for the Incident to Decide Who Is in Charge

One of the worst times to determine who has authority to stop an AI system is after something has gone wrong.

For higher-risk AI applications, those decisions should be made before deployment. The organization should know who owns the system, who monitors it, what constitutes an exception, what triggers escalation, who investigates, and who has authority to restrict or shut down the technology. There should also be clear expectations for when compliance, legal, privacy, security, clinical leadership, senior leadership, or the board becomes involved.

The organization also needs to understand what happens after an incident. How will affected decisions be identified? How will the organization determine whether the problem was isolated or systemic? What root-cause analysis will be performed? What corrective action will be implemented? How will the organization validate that the corrective action actually addressed the underlying problem?

Saying “the AI was wrong” does not constitute root-cause analysis. The organization needs to determine why the AI produced the wrong result and why the control environment did not prevent or detect the problem. The failure may involve data, the model, the source material, system configuration, workflow design, human interaction, escalation, vendor management, or another component of the control environment.

That distinction matters because the purpose of RCA is not to assign blame to the technology. It is to understand the system well enough to prevent recurrence.

Healthcare Compliance Has More of the AI Governance Playbook Than It May Realize

The more I study AI governance, the more convinced I become that healthcare compliance professionals have an important role to play in this transformation. We have spent decades managing complicated, changing, high-consequence environments in which errors can affect patients, reimbursement, privacy, regulatory obligations, and organizational reputation. We understand risk assessment, policy development, education and training, internal controls, auditing and monitoring, investigations, root-cause analysis, corrective action, and validation.

AI does not make those disciplines obsolete. It makes them more important.

What changes is the speed and scale of the environment. A human may make one incorrect decision. An automated system may replicate an incorrect decision across thousands of transactions or interactions before the organization recognizes that a pattern exists. When AI begins influencing or executing decisions at scale, the organization needs assurance mechanisms capable of operating at that same scale.

This is why I do not believe healthcare needs to reinvent compliance in order to govern AI. We need to apply what we already know while adapting our methods to a technology that can change rapidly, operate across systems, generate outputs that require validation, and increasingly exercise greater levels of autonomy.

A Better Question for Healthcare Leaders

I would encourage healthcare leaders to move away from the broad question, “What are the risks of AI?” That question is understandable, but it is too general to drive an effective response.

A more useful question is: “What kind of AI risk are we dealing with—and what response does that risk require?”

If the risk is operational, we should control it. If it is strategic, we need to prepare and position for it. If it is novel or unexpected, we need to watch for it, contain it, learn from it, and adapt our governance accordingly.

Across all three categories, the organization should maintain qualified human accountability, require transparency where decisions matter, monitor the technology and the surrounding workflow, investigate meaningful failures, implement corrective action, and reassess controls as the technology evolves.

Responsible AI governance is not about creating a system that predicts every possible failure. We cannot do that. It is about creating an organization that can recognize risk early, respond intelligently, contain emerging problems, and learn from what the technology is telling us about the limitations of our original assumptions.

We may not know every risk AI will create. But we can decide today how prepared we will be when one appears.