There is a sentence that appears in almost every AI policy, every vendor deck, every regulator’s advisory, and almost every board deck I have seen this year. It reads, in some variation: “We keep a human in the loop.”
It is meant to be reassuring. It is meant to signal that no matter what the system does, a person will see it before it acts, and that person will decide.
Read carefully, the sentence does not say that. It describes a diagram, not a discipline.
Human in the loop is an interface pattern. It tells you where the human’s screen is in the flow of work. It does not tell you whether the human can refuse the flow, or what happens to them if they do.
What the phrase actually specifies
Strip the phrase down to its literal claim, and here is what it commits to:
- There is a step in the workflow where a person can see the system’s output.
- That person can, in principle, take an action other than “accept.”
That is the entire specification. Everything a governance model would need to add is missing.
It does not say how much time the human has. A recommendation displayed for four seconds on a shift-timer screen and a recommendation displayed for two hours on a legal reviewer’s desk are both “in the loop.” One of them is not a governance step.
It does not say what the human is measured on. If the reviewer’s manager tracks throughput and treats overrides as variance to explain, the human in the loop is in the diagram but not in the decision.
It does not say whether the human has authority to change the input the system used, or only to accept or reject the output the system produced. Reviewing a decision the system already framed is not the same as forming a decision the system helped inform.
It does not say what the human has already lost by the time the loop reaches them. If the system has drafted the sentence, ranked the queue, summarized the file, and displayed a confident recommendation, the human is not entering the decision. They are entering the last edit of a decision that has already taken shape.
The book’s version of the question
In Chapter 4, working through the story of a benefits case, we arrive at this formulation:
The design question is not, “Did we keep a human in the loop?” The design question is, “What kind of human authority did we preserve?”
That reframing is the whole difference. Loop is a topology. Authority is a capacity. The first is trivial to satisfy. The second is much harder, because it forces the design to answer questions the phrase lets you skip.
Chapter 6 goes further:
The answer should not always be, “keep the human in the loop.” That phrase has become too easy. It can mean a person sees the output, clicks approve, or stands near the machine while it acts. Human presence is not enough. The question is whether the human is truly authoritative.
“Truly authoritative” is a governance test that decomposes into things you can actually check.
The five tests a real governance model applies
Instead of the loop question, apply these:
1. Can the person say no without becoming the problem? A reviewer who slows down a file is either exercising judgment or generating a variance report. Which of the two she is doing depends on how the organization treats the pause. Governance begins with knowing the answer, in advance and out loud.
2. Can the person see what the system did not put on screen? The system chose an emphasis. It selected a category, a summary, a recommendation. What did it leave out? A reviewer who cannot pull the omitted context is not reviewing; she is confirming.
3. Can the person hear from the person affected? Some decisions are impossible to make well from the file alone. A candidate ranking, a benefits denial, a safety finding — each carries information that the system’s inputs could not have captured. Governance decides in advance which decisions require a channel back to the person the decision is about, and how that channel is protected against the queue’s economics.
4. Does the person have time enough for the type of decision being made? Not endless time. Enough. A routine confirmation may need seconds. A consequential denial may need an hour. If the loop presents both at the same pace, the loop is not the governance model — it is the throughput model.
5. Is the person a plausible author of the sentence that will leave the company? Voice rights, from earlier in the book: could the person be named, in public, as the human who owns the language before it travels? If the answer is no, the human is standing near the sentence, not signing it.
Any decision class that fails one of these tests still has a human in the loop by the vendor’s definition. It does not have human authority in the governance sense. Which one you were counting on will determine what happens when the decision goes wrong.
What governance actually needs to be
Governance for AI-enabled work is not a diagram of where humans stand. It is a specification of what humans are allowed to do at the point where the company acts.
It has to name:
- The classes of decision the company makes with system help, from routine to consequential.
- For each class, the authority the human must have — the time, the context, the tools, the escalation path, the freedom to refuse — and the standard against which that authority will be measured.
- The visible signal to the person outside the company that a decision was owned inside it — a signature, a named reviewer, a documented reasoning trail — and the practice inside the company that makes that signal true.
- The operational tests that would fail if authority were quietly being eroded — override rates, average review times on consequential cases, escalation frequency, the willingness of managers to protect a reviewer who pauses a file.
Human in the loop can be preserved while every one of these degrades. That is precisely the failure mode this book returns to. A company can pass its AI audit, keep its policy in place, populate the diagram exactly as promised, and still lose the human authority it thought it was protecting.
The sentence to strike
If your governance model uses the phrase “human in the loop” as a load-bearing commitment, strike it. Replace it with a sentence in the form:
“For decisions of class X, a named human with authority Y has time Z to form or refuse the answer, and can be identified afterward by anyone the decision touched.”
That sentence is longer. It is also actually a governance model.
The other one is a picture of one.
This essay draws on Chapters 4 and 6, and Interlude III, of AI in the Org Chart.