Responsible AI in Healthcare: Principles Aren’t Enough, You Need Controls

Why healthcare organizations need to move responsible AI from good intentions to an operational compliance framework

Artificial intelligence is becoming increasingly integrated into healthcare, and organizations are appropriately paying greater attention to responsible AI. Much of the conversation has focused on principles such as fairness, transparency, accountability, privacy, security, ethical use, and equity. Those principles matter, but principles alone do not create a governance program.

After more than four decades working in healthcare compliance, auditing, coding, revenue integrity, and regulatory risk, I have learned that the real challenge is translating an obligation or expectation into something an organization can actually implement, measure, audit, and enforce. Healthcare has spent decades developing processes for doing exactly that: translating requirements into policies and procedures, workforce education, operational controls, monitoring, audits, investigations, corrective action, and accountability. AI requires the same discipline.

The question for healthcare leaders should therefore extend beyond whether their organizations believe in responsible AI. A more meaningful question is whether they can demonstrate, through evidence and operational controls, that the AI systems they use are functioning within defined boundaries and that someone is accountable when they do not.

Responsible AI in Healthcare Must Become Operational

It is relatively easy for an organization to publish principles stating that AI should be fair, transparent, secure, and accountable. The harder task is determining what those principles actually mean when AI is incorporated into a healthcare workflow.

A principle has value only when an organization can translate it into expectations, controls, and measurable outcomes.

Consider an AI application used to assist with coding. If fairness is a governing principle, how will fairness actually be evaluated within that application? If transparency is required, what information supports the system's recommendations, and can those recommendations be meaningfully evaluated by a qualified person? If the system produces an inaccurate recommendation, who is responsible for identifying the problem, investigating it, determining whether the issue is isolated or systemic, and deciding what corrective action is necessary?

The same questions apply to privacy, security, and accountability. What patient information does the system access? Where does that information go? What happens if the system repeatedly recommends a higher-paying code? How would the organization identify that pattern, who would investigate it, and who has the authority to suspend or modify the system if the risk becomes unacceptable?

These are no longer abstract questions about the ethics of artificial intelligence. They are practical governance and compliance questions that need to be answered if an organization is going to demonstrate that its use of AI is responsible.

AI Fairness in Healthcare Requires Testing and Monitoring

Healthcare organizations should not assume that an AI system is objective simply because a machine generated the result. AI systems rely on data, rules, models, and human-designed processes, all of which can reflect bias, incomplete information, or limitations in the underlying data. In healthcare, those limitations can have meaningful consequences because AI may be used in circumstances that affect patients, providers, specialties, geographic populations, payer populations, and other groups.

For that reason, fairness needs to be treated as an operational requirement rather than a statement of intent. Organizations should consider whether the outcomes produced by an AI system are consistent across relevant populations and whether unexpected differences emerge over time. When disparities are identified, the organization needs a process for determining whether the variation has a legitimate explanation or represents inappropriate or unintended bias.

This means fairness cannot be treated as a one-time test performed before deployment. It requires ongoing monitoring and a defined response when results fall outside established expectations. Fairness becomes something that can be evaluated through testing, monitored through performance data, investigated when disparities emerge, and addressed through corrective action.

The principle may be fairness, but the evidence of responsible implementation is found in the testing and monitoring that demonstrate how fairness is being maintained.

AI Transparency in Healthcare Requires Traceability

One of my longstanding compliance principles is simple: Show me the source.

That principle becomes particularly important when AI is influencing a regulatory, coding, reimbursement, compliance, or operational decision. A polished recommendation is not sufficient evidence that the recommendation is correct. The organization needs to understand what information supports the result and whether that information is authoritative, current, applicable to the specific circumstance, and accurately interpreted by the system.

This distinction is increasingly important as generative AI becomes more capable of producing answers that appear authoritative. An AI system can generate a highly fluent and professionally worded response while still relying on incorrect information, outdated material, or an inappropriate interpretation.

Fluency is not evidence.

Responsible AI therefore requires a level of traceability appropriate to the consequences of the use case. Qualified individuals should be able to validate consequential recommendations and, when necessary, understand how a decision was reached. Depending on the application, this may require source validation, explainability, documentation, version control, and audit trails.

The objective is not necessarily to make every AI system completely transparent at the technical level. Rather, organizations need enough visibility into the system's inputs, outputs, supporting information, and decision process to exercise meaningful oversight.

AI Accountability in Healthcare Requires Clear Ownership

Accountability is one of the most important—and potentially most difficult—elements of AI governance.

When an AI system contributes to an undesirable outcome, responsibility can become unclear. Is the responsibility with the employee using the system, the department that implemented it, IT, compliance, the vendor, the model developer, or executive leadership?

That question cannot wait until an incident occurs.

Significant AI use cases should have clearly defined ownership before the technology is placed into operation. Someone needs to be responsible for approving the use case, establishing its boundaries, monitoring performance, reviewing exceptions, responding to failures, and determining whether the system remains appropriate for continued use.

I keep coming back to a principle that has become central to how I think about AI governance:

AI may participate in a decision, but it cannot own accountability for the decision.

The organization remains responsible for establishing the governance structure within which the technology operates. Clear ownership and defined decision rights are therefore not administrative details. They are fundamental controls that establish who has authority to act when the technology performs outside established expectations.

Healthcare AI Privacy and Security Require More Than a HIPAA Checklist

Healthcare AI also raises significant questions about information management.

Organizations need to understand what data an AI system can access, whether that data includes protected health information or personally identifiable information, where the information is transmitted, whether it is retained, who can access it, and whether organizational information is used to train or improve a model. These questions become even more complicated when third parties are involved or when employees enter information into external AI tools.

HIPAA remains critically important, but responsible AI governance requires organizations to consider the entire data lifecycle rather than treating HIPAA compliance as the endpoint of the analysis. Data governance, cybersecurity, access controls, data minimization, retention, vendor relationships, permitted uses, and monitoring all become relevant depending on the technology and the workflow in which it operates.

The emergence of AI agents introduces additional considerations because systems may be capable of moving information between applications or taking actions across multiple systems. The governance question therefore extends beyond where information is stored. Organizations need to understand how information moves through the AI-enabled environment, what authority the technology has to access or transmit it, and what controls exist to prevent inappropriate use.

Ethical AI in Healthcare Requires Defined Boundaries

One of the most important principles in AI governance is that capability is not authority.

The fact that an AI system can perform a particular task does not mean that the organization should automatically permit it to do so. That distinction becomes particularly important as AI systems become increasingly capable of acting with less direct human involvement.

An AI system that retrieves regulatory guidance for a compliance professional has a relatively limited role. A system that interprets that guidance and recommends an organizational response has greater influence. A system that independently changes a workflow or initiates an action represents a substantially different level of risk because authority has shifted closer to the technology itself.

Healthcare organizations therefore need to establish deliberate boundaries around what AI may assist with, what it may recommend, what it may decide, and what it may execute. They should also identify which activities require human approval and which decisions should never be delegated to AI.

Those boundaries should be supported by approval thresholds, escalation pathways, defined authority limits, and hard stops where appropriate.

This is where my philosophy of “build the brakes before you need them” becomes particularly practical. Controls should not be designed only after a system has demonstrated that something can go wrong. The organization should establish the mechanisms for intervention before the technology is given authority to create a consequential outcome.

AI Equity in Healthcare Requires Looking Beyond Accuracy

Accuracy is an important measure of AI performance, but it is not sufficient to demonstrate that an AI system is operating equitably.

A system can perform well when evaluated across an aggregate population and still produce materially different outcomes for particular groups or in particular settings. In healthcare, that distinction deserves careful attention because the consequences of an AI error may not be distributed evenly.

Organizations should therefore consider who benefits from an AI system, who is more likely to experience an error, whether relevant populations are adequately represented in the underlying data, and whether the system performs differently across care settings or populations. They should also consider whether automation could unintentionally create new barriers to access or service.

The appropriate question is not simply whether the technology works. For whom does it work, under what circumstances, and where might it fail? Equity requires organizations to evaluate those questions over time through impact assessments and ongoing monitoring rather than assuming that an acceptable overall accuracy rate necessarily means acceptable outcomes for everyone affected by the system.

Healthcare AI Governance Requires Auditing and Monitoring

This is an area where healthcare compliance professionals can bring significant value to AI governance because the fundamental discipline is already familiar.

Effective compliance programs do not rely on policies alone. Organizations train their workforce, implement controls, audit performance, monitor activity, investigate anomalies, identify root causes, implement corrective action, and then monitor again to determine whether the corrective action was effective.

AI should be governed with the same discipline. An AI system may perform appropriately during initial testing and behave differently later. The model may change, the underlying data may change, regulations may change, a vendor may release an update, or the organization's use of the technology may expand. Employees may also begin relying on the system in ways that were not contemplated when the original use case was approved.

These changes matter because an error that occurs once may be relatively contained, while a small error that is repeated at machine scale can become a significant compliance, financial, operational, or patient-impact issue.

This is why deployment should not be viewed as the end of AI governance. Deployment is the beginning of the organization's responsibility to monitor the technology.

AI Incident Management Requires Root-Cause Analysis

Healthcare compliance professionals understand that correcting an individual error is not the same as correcting the underlying problem.

When an AI system produces an inaccurate or inappropriate recommendation, the organization needs to understand why the failure occurred. The root cause could involve outdated information, an inappropriate source, a failure to interpret accurate information correctly, missing context, a model update, operation outside the system's intended scope, inadequate human review, a failed escalation process, or excessive autonomy.

Simply concluding that “the AI was wrong” does not constitute a root-cause analysis.

The organization needs to determine what allowed the incorrect result to occur, whether the issue is isolated or systemic, and whether similar problems may exist elsewhere in the workflow.

Once the underlying cause has been identified, the organization can develop a Corrective Action Plan, implement the appropriate changes, and monitor whether those changes actually resolved the problem. This process is familiar territory for healthcare compliance. AI does not change the fundamental need to understand why something went wrong and to verify that corrective action was effective.

A Healthcare AI Compliance Framework for Responsible AI

Rather than treating responsible AI as an entirely separate discipline, healthcare organizations can build on the compliance infrastructure they already understand.

Applied to AI, this means first identifying where AI is operating and risk-ranking the relevant use cases. Organizations can then establish controls appropriate to those risks, train the people who interact with the technology, monitor actual performance, investigate unexpected outcomes, implement corrective action, and validate whether those corrective measures were effective.

The process should not end there. As technology, regulations, vendors, data, and organizational workflows change, the risk assessment may need to be revisited. Responsible AI is therefore better understood as an operating discipline than as a one-time certification that an organization has adopted appropriate principles.

The objective is to create a system in which responsible AI can be demonstrated through evidence. Policies establish expectations, but controls demonstrate implementation. Monitoring provides evidence of performance, and corrective action provides a mechanism for addressing failures.

Together, these elements transform responsible AI from a statement of organizational values into an operational compliance framework.

Human-in-Governance™: Making Human Oversight Meaningful

There is another principle that healthcare organizations should consider carefully as AI becomes more autonomous: the distinction between having a human in the loop and maintaining meaningful human governance.

Human oversight is important, particularly for higher-risk applications, but simply placing an individual somewhere in an automated workflow does not necessarily provide effective oversight.

The more important question is where human authority resides.

Who establishes the AI system's boundaries? Who determines acceptable risk? Who approves increased autonomy? Who reviews exceptions? Who has the ability to override or stop the system? Who owns corrective action when something goes wrong? And, ultimately, who remains accountable for the outcome?

I think of this as Human-in-Governance™. Humans do not necessarily need to perform every task that AI can perform. However, human judgment, authority, and accountability need to remain embedded in the governance structure surrounding the technology. The purpose of human governance is not to prevent appropriate automation. It is to ensure that the organization consciously determines the conditions under which automation is permitted and maintains the authority to intervene when those conditions are no longer being met.

From Responsible AI Principles to Proof

Healthcare organizations should not have to choose between innovation and governance. Artificial intelligence has significant potential to reduce administrative burden, improve access to information, identify risks earlier, support healthcare professionals, and allow highly trained individuals to focus more of their time on higher-value work. Realizing those benefits responsibly, however, requires organizations to move beyond statements of intent.

When an organization tells me that it has adopted responsible AI principles, my compliance perspective naturally leads to the next question: How do you know?

Show me the policy. Show me the risk assessment. Show me the source behind the recommendation. Show me who is accountable. Show me where the human intervenes. Show me the audit trail. Show me how performance is being monitored. Show me what happens when something goes wrong.

Most importantly, show me the controls that demonstrate how the organization has translated its principles into practice.

Responsible AI in healthcare cannot simply be something an organization says it believes in. It must be something the organization can demonstrate through its policies, processes, evidence, oversight, monitoring, and accountability. Principles provide the direction, but controls provide the mechanism for putting those principles into practice.

That is the distinction healthcare leaders need to understand as AI becomes increasingly embedded in the work. Responsible AI is not established by good intentions alone. It is established when an organization can demonstrate that its technology is operating within defined boundaries and that meaningful governance exists when it does not.