Human + AI in Healthcare: Why the Future Isn’t Human or Machine

Why healthcare leaders need to design the human-agent relationship as carefully as the technology

Artificial intelligence is moving quickly from tools that help us complete individual tasks to agents capable of managing increasingly complex workflows. That shift creates tremendous opportunity for healthcare. AI can monitor regulatory changes, organize information, perform first-pass analysis, identify patterns, flag potential compliance risks, support audit preparation, and reduce hours of manual work.

But as I continue my work in AI leadership and develop AI-enabled compliance capabilities, I keep returning to an important question: Are we spending as much time designing how humans will work with AI as we are designing the AI itself?

Building an intelligent agent is only part of the challenge. We also need to determine how people will understand it, trust it, challenge it, intervene when necessary, and remain accountable for the decisions that matter. That is where AI moves from interesting technology to a sustainable operating model.

Human and AI Collaboration in Healthcare Requires More Than Automation

There is a natural tendency with automation to focus on how much work technology can take away from people. As AI becomes more capable, however, I think healthcare leaders need to think beyond automation and focus on how humans and AI will work together.

The important question is not simply whether AI can perform a particular task. It is whether the workflow has been intentionally designed around the respective strengths of the technology and the people using it. AI can process enormous amounts of information, recognize patterns, monitor activity, and perform repetitive analysis at a scale that would be difficult for an individual professional to match. Humans bring context, experience, professional judgment, relationships, ethical reasoning, and accountability.

That means the future of healthcare AI should not be about choosing between human expertise and artificial intelligence. It should be about determining where each contributes the most value and how the two operate together within a governed workflow. The human-agent relationship therefore becomes part of the design. It cannot be treated as something that will naturally work itself out after the technology has been deployed.

AI Autonomy in Healthcare Should Not Be the Goal

When organizations evaluate automation, the natural question is often, “How much of this process can we automate?” I think healthcare leaders should ask a different question: How much autonomy should we give AI in this particular workflow? Those are very different questions.

Just because AI can perform a task does not mean it should perform that task independently. In healthcare compliance, the consequences matter. An AI agent may be well suited to continuously scan regulatory sources, identify relevant changes, compare information, or conduct an initial review. Interpreting what a regulatory change means for a particular organization, determining whether an audit finding creates an overpayment obligation, or providing consequential guidance is different.

The more consequential the decision, the more clearly we need to define where human authority begins. That is why I do not view maximum autonomy as the objective. The goal should be trusted autonomy—AI that can operate with an appropriate level of independence while remaining transparent, governable, and subject to meaningful human intervention.

Autonomy should be based on evidence that the workflow can be operated safely and responsibly at that level. It should not be granted simply because the technology is capable of doing more.

Human Oversight of AI Starts With “Show Me the Source”

Anyone who has worked with me knows one of my most common questions: Show me the source.

After more than 40 years in healthcare and compliance, that discipline is deeply ingrained in how I work. If we are going to advise a healthcare organization about a regulatory requirement, coding rule, audit finding, payer policy, or compliance risk, I want to know where the information came from, whether it is authoritative and current, whether it applies to the situation, whether conflicting guidance exists, and whether we can defend our interpretation. AI should not lower that standard. It should help us strengthen it.

In a well-designed human-agent workflow, users should be able to see the evidence the AI relied upon, understand the basis for a recommendation, recognize relevant risk signals, and challenge the conclusion when something does not look right. Source visibility and traceability become particularly important as AI moves beyond generating information and begins participating in decisions and workflows.

AI can be incredibly convincing. A polished answer is not necessarily an accurate answer. Fluency is not evidence.

For healthcare organizations, trust therefore cannot be based simply on whether an AI system sounds confident or produces useful-looking results. The information needs to be verifiable, and the person responsible for the outcome needs to be able to evaluate it.

Design the Human-Agent Relationship Before Deployment

One of the most important parts of AI workflow design may ultimately be the handoff between machine and human.

Healthcare leaders need to determine where an agent can proceed independently, where it should pause, what requires review or approval, what must be escalated, who can override the system, and what happens when AI encounters something outside its established boundaries. These questions should not be answered after deployment. They should be part of the design.

Consider a regulatory-intelligence workflow. An agent might monitor regulatory sources, identify relevant changes, retrieve the underlying information, compare requirements, summarize the findings, and risk-rank what it discovers. A human compliance professional could then validate the information, interpret what it means for the organization, determine the appropriate response, and provide the advice.

That division of responsibility can evolve. Evidence may eventually demonstrate that AI can safely assume greater responsibility for portions of the workflow. If that happens, expanding its authority may make sense.

But increased autonomy should be earned through evidence, not granted simply because the technology is capable of performing the task.

Why Human-in-the-Loop AI Is Not Enough

I have become increasingly cautious about the phrase “human in the loop.” It sounds reassuring, but where exactly is the human?

If someone is reviewing hundreds of AI-generated recommendations and simply clicking approve, there is technically a human in the loop. But is there meaningful human judgment?

Effective human oversight requires more. People need the authority to pause, question, override, correct, and escalate the AI without fighting against the workflow. They need access to the underlying information and enough expertise and time to evaluate what the system is producing.

That connects directly to something I have been developing throughout my AI work: Human-in-Governance™.

For me, Human-in-Governance™ goes beyond putting a person at the end of an automated process. Human judgment needs to be built into the governance structure itself. Humans establish the boundaries, determine acceptable risk, define escalation thresholds, oversee exceptions, monitor performance, and remain accountable for consequential decisions.

The question is therefore not simply whether there is a human somewhere in the workflow. The question is whether the right human has the information, expertise, authority, and opportunity to exercise meaningful judgment.

AI Will Change Healthcare Work, but Human Expertise Still Matters

There is another part of this transformation that healthcare leaders need to address openly: the workforce.

People are understandably asking what happens to their jobs when AI can perform work they have traditionally done. In healthcare compliance, I think there is an opportunity to use AI differently—not simply to eliminate human activity, but to shift human expertise toward the work that requires it most.

Consider how much time compliance professionals currently spend searching websites, gathering regulations, organizing documents, preparing audit files, comparing policies, and performing other repetitive work. If AI dramatically reduces that burden, the compliance professional does not become less important. Their expertise can become more concentrated on the work that requires expertise. 

Instead of spending hours finding information, professionals can spend more time determining what it means, where the risk lies, whether the result makes sense, what may be missing, how it affects the organization, and what should happen next. That is higher-value work.

The opportunity is not to remove people from the workflow simply because technology can perform more of it. It is to allow experienced professionals to spend less time on repetitive information processing and more time applying judgment, managing exceptions, addressing risk, and making decisions that require human expertise.

Human Experience Can Become an AI Governance Control

There is something else I do not want organizations to lose as they automate more work.

Experienced professionals develop instincts. After years of reviewing claims, regulations, audits, documentation, investigations, and compliance issues, you sometimes look at something and think: That doesn’t look right.

You may not immediately know why, but you know enough to stop and investigate. That professional skepticism is incredibly valuable in an AI-enabled organization, and we should not design it out of the workflow. Human experience itself can become part of the control environment.  AI can process more information than we ever could. Experienced humans, however, understand context, nuance, organizational history, relationships, consequences, and exceptions that may never be completely represented in the data.

That is one reason I believe human expertise becomes more—not less—important as AI capabilities increase. The technology can help surface information and patterns, but the experienced professional may recognize when something does not fit the expected pattern or when a seemingly reasonable result warrants another look.

In a strong governance environment, that skepticism is not an obstacle to automation. It is a control.

Responsible AI in Healthcare Requires Designed Trust

Organizations sometimes talk about getting employees to “trust AI.” I think that puts the emphasis in the wrong place.

We should not ask people to trust AI simply because leadership purchased it. The system should earn trust.

Trust comes from transparency, accuracy, consistency, traceability, privacy protection, fairness, clear accountability, and the ability to intervene when something goes wrong. From a compliance perspective, I would add one more: verification. 

• Can we see the source?

• Can we understand the basis for the recommendation?

• Can we identify what information the system relied upon?

• Can we challenge the result?

• Can we determine what happened after the recommendation was made?

Those questions are part of responsible AI governance because trust should be based on evidence, not assumption.

Healthcare AI Governance: Don’t Scale Chaos

I also strongly believe in an iterative approach to AI implementation.Organizations do not need to solve every AI governance problem before they begin. They can start with a meaningful workflow, establish controls, define human and agent responsibilities, test the process, monitor it, learn from it, improve it, and then scale when the evidence supports doing so.

That approach matters because a successful pilot does not automatically mean an AI workflow is ready to operate at enterprise scale. Organizations need to understand what happened during the initial deployment. Where did the AI perform as expected? Where did humans need to intervene? What exceptions emerged? Did people understand their responsibilities? Were there situations where the system produced a result that looked reasonable but required further investigation?

Those lessons should inform the next stage of deployment. My principle is simple: Don’t wait for perfection. But don’t scale chaos. Moving too slowly carries risk. Moving too quickly without governance carries risk too. Responsible AI leadership requires balancing both.

The Future of Healthcare AI Is Human + Machine

After more than four decades in healthcare, I do not view AI as a replacement for human expertise. I see an opportunity to remove enormous amounts of repetitive work so that experienced professionals can focus more of their time where their judgment creates the greatest value.

AI brings speed, scale, pattern recognition, and extraordinary processing capability. Humans bring experience, context, professional judgment, relationships, ethical reasoning, and accountability. The opportunity is to combine these strengths intentionally. 

That requires healthcare organizations to design the human-agent relationship as carefully as they design the technology. Make the evidence visible. Define authority. Create meaningful checkpoints and escalation pathways. Train people for new oversight roles. Audit and monitor performance. Preserve the ability for a qualified human to question the system when something does not look right.

The future of responsible AI in healthcare is not human versus machine.

It is human intelligence amplified by artificial intelligence, with governance connecting the two.

That is the future I believe we should be building.