Human in the Loop Is Not an Operating Model

“Human in the loop” tells you where a person sits. It does not tell you what they do there.

Over the past month I have written about four situations that came up in our work. Read together, not one of them was decided at the review step.

Information that should have stopped a refill arrived on a phone call about something else. There was no output to review. Someone had to be positioned to hear it.

A physician encoded her treatment plan into software. Her judgment was sound. The open question was whether anyone could check that the rule ran correctly on a given patient.

A patient reported chest pain and the escalation went unanswered for more than five minutes. The protocol had worked as written. A person noticed where it ran out and stayed on the line.

A hypothetical about a patient lost its conditional over a couple of handoffs and arrived as a death. What fixed it was someone going back to the original exchanges and reading what was actually said.

These are examples of hearing and checking and noticing and going back to the source. Four different jobs, and only one of them looks like reviewing an AI output. Even that one requires an ability not every reviewer has.

When an AI implementation says it keeps a human in the loop, it usually means one person approving what the system produced. That covers some of the checking. It does not cover the rest.

So the question I would ask of any AI deployment, ours included, is not whether there is a human in the loop. It is which of these jobs someone is doing and whether they can do it. And who answers when one gets missed.

That’s an operating model, not a checkpoint. It’s what we are building toward.