Definition
The phrase now names two unrelated things. In machine learning it means a person labeling data so that a model can be trained. In agent systems it means a person deciding whether a running agent may perform a specific act. The words are identical. The human, the timing, the artifact, and the failure mode are not.
This page is about the second one, and it starts with the distinction because the conflation is not harmless. The newer meaning inherited the older one’s vocabulary and none of its assumptions, and that is a large part of why the tooling for it is so thin.
The disambiguation
Human in the loop entered wide use describing supervised machine learning: a person in the training cycle, correcting a model’s output so the next version is better. That work is real, it is large, and it owns the term. Agent systems then borrowed the phrase for something else entirely, and the borrowing is where the trouble starts.
| Training senselabeling, rating, preference data | Runtime senseauthorizing an act an agent is about to take | |
|---|---|---|
| Who the human is | An annotator, rater, or domain reviewer, usually working through a queue of examples. | The person accountable for the consequence of one act that has not happened yet. |
| When it happens | Before the model is used, or between training runs. The output is a better model. | While an agent is mid run and blocked. The output is a decision about a single act. |
| How long the human has | As long as the labelling task takes. Nothing is waiting. | As long as the agent is stopped, which in practice is seconds. Something is waiting. |
| What goes wrong | Noisy labels, drifting guidelines, disagreement between annotators. | An act is performed whose consequences nobody at the keyboard could see. |
| What the record is for | Improving the model. The individual label is not evidence. | Answering, later, who decided this and on what basis. The individual decision is the evidence. |
The consequence of reading the second column with the first column’s habits is a queue. Labeling work is properly a queue: many small items, throughput is the goal, and the reviewer’s job is to keep moving. Runtime authorization is not a queue. There is one item, it is not equivalent to the last one, and throughput is the failure mode rather than the objective.
The definition
Human in the loop for AI agents is the moment an autonomous agent stops short of an act it is not authorized to take alone and hands a named person the decision, together with the evidence that decision requires.
Almost every implementation satisfies the first clause and skips the second. The agent stops, a person is asked, and what the person receives is the name of an act and two buttons. That is a human in the loop by the letter of the phrase. It is not a human deciding, because nothing was supplied that could have changed the answer.
The gap between those two needs a name, because the whole difficulty of the field sits inside it, and an unnamed distinction is one that keeps getting argued from scratch.
The term this page proposes
Informed authorization
Authorization given by a person who could have refused on the evidence in front of them. Anything short of that is a signature that was collected rather than a decision that was made.
The echo of informed consent is deliberate. In research and medical ethics, comprehension is an element of consent rather than a courtesy: a signature from someone who did not adequately understand what they were agreeing to is a defective consent, and it is treated as one. The same argument applies to an agent asking for permission, and it is not currently made.
It is not a novel argument either, which is the point. The law that governs high risk systems in Europe already writes comprehension into the oversight obligation, in the section quoted further down this page. Nothing here asks for a new principle. It asks for the one that already exists to be buildable.
How often this comes up
There is no public census of where agent systems need a human, so one was built for this work. It enumerates 192 distinct moments across 20 domains, from coding agents and payment release through security operations, moderation, legal review and agent fleet supervision, and assigns each to one of seventeen interaction families. It is an enumeration rather than a survey: it is reproducible, and it is nobody’s telemetry.
43%
Nearly three times the next largest family, which is review at 27 moments. The single most common reason an agent needs a human is to authorize one specific act, and that is exactly the moment the industry answers with a modal.
6
Three percent, and the reason is not that people rarely want to. It is that in mainstream agent stacks the primitive mostly does not exist, so the moment never appears in a workflow to be counted.
0
Irreversible acts with extreme consequences and cheap reversible ones are presented in the same widget, with the same affordance. Severity is computable from what the act will touch. Across the census, nothing computed it.
The practical reading of the first number: build for one act at a time and you have addressed nearly half of everywhere a person is needed. The practical reading of the third: the cheapest available differentiator is to make a destructive act look different from a reversible one, because at the time of counting nothing did.
The anatomy
This is the substantive claim of the page. A riskier act is not made safe by asking someone about it. It is made accountable by giving it a shape, and the shape has parts that can be listed. An approve button accepts the identical underlying risk with no shape at all, which is why it feels lighter and why it is not.
Not a category, not a session, not a tool name. The single thing that will happen if the answer is yes. A request to approve a capability is a request to approve everything that capability can ever do.
A stable identity for that exact act, computed from what the act will do. Without one, a yes can be replayed against a different act, and the record cannot prove which act was actually authorized.
Two million one hundred forty seven thousand and three rows, not a large table. A count and an addressable sample. Every approval surface on the market describes the act; almost none describes the consequence, and the consequence is what the person is being asked about.
Clean, costly, or irreversible, with the cost named. Irreversibility is not correlated with how heavy the interface is today: an unrecoverable act and a trivially undoable one arrive in the same widget, with the same button.
Attached, not linked. A claim the person can dispute in place, and a correction that applies to the thing being decided rather than to a copy of it. Evidence a reader cannot challenge is decoration.
Authority with no end is not authority for an act. It is a standing grant, and standing grants are the ones nobody ever reviews. If the act does not happen inside the window, the answer has to be asked again.
Settled by a person, with the record naming which person and when. Held is a real answer and needs to be expressible: the evidence is not sufficient, and saying so is a decision, not a failure to decide.
The decision, the evidence as it stood, the challenges raised, and the outcome, none of them editable afterwards. A record that can be tidied is a record that cannot settle an argument.
One rule governs all eight: default accept is prohibited. No preselected yes, no enter key default, no accept all across a set without an explicit target predicate. A default answer converts a decision into an interruption, and people become fast at clearing interruptions. That speed is usually reported as an efficiency gain.
What the frameworks call it
The runtime sense of the phrase does have a de facto name, and it is the phrase itself: at almost every agent framework, the documentation page for this is titled human in the loop. The right hand column is the more interesting one. It is the complete set of answers each implementation lets a person give.
| Project | What it calls this | Answers a person may give |
|---|---|---|
| LangChain and LangGraph | Human-in-the-loop | approve, edit, reject, respond |
| OpenAI Agents SDK | Human-in-the-loop, approval gates | approve, reject |
| Vercel AI SDK | Tool execution approval | approved, denied |
| Claude Agent SDK | Permissions and approvals, not human in the loop | allow, deny, ask |
| LlamaIndex | Human in the loop | input required, human response |
| CrewAI | Human-in-the-loop (HITL) | pending human input, then feedback |
| Model Context Protocol | Elicitation | accept, decline, cancel |
Read the column downwards. Not one of these vocabularies contains a word for the evidence the decision rests on, a word for disagreeing with that evidence, or a word for who owned the decision afterwards. The verbs are all about disposing of a request. That is not an oversight in seven independent places; it is what happens when a decision is modelled as an interruption to be cleared.
The absence is the opening. Several efforts are now building the layer underneath, and doing it well: signed pre execution receipts, agent identity, authenticated delegation, protocols that can prove a named human authorized a specific act. All of that answers whether an authorization happened. None of it answers whether the authorization was worth anything, which is a question about what the person could see. That is the question this page names.
Why the button is worse than it looks
The failure mode has a name and it is not carelessness. People asked to check a machine that is usually right become worse at checking it, in proportion to how right it usually is. An interface that supplies a highlighted yes, a keyboard default, and no way to interrogate the claim is not neutral about that tendency. It is the mechanism of it.
Article 14 of the EU AI Act is worth reading closely by anyone building this, regulated or not, because it describes the capability rather than the widget. Three of its five clauses cannot be satisfied by an approve button at all.
Natural persons to whom human oversight is assigned must be enabled to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias).
And to correctly interpret the high-risk AI system’s output, taking into account the interpretation tools and methods available.
And to decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output.
Correctly interpret is a comprehension requirement. Disregard, override or reverse requires that reversal exist as a thing a person can do, and in most agent stacks it does not exist at all, which is why it almost never appears in a workflow. A surface that offers approve and deny and nothing else fails the second and third of these on its face, and arguably causes the first.
The neighbouring words
Each of these is a real thing with a real definition, and each gets substituted for the others in practice. The substitutions are not equivalent, and a specification written with the wrong one produces the wrong system.
How to build it
Everything below can be done inside whatever agent framework is already in use. Most of it is a few hours of work, and the reason it is rarely done is not difficulty. It is that the approve button looks finished.
Ask about the act that is about to happen, with its arguments resolved. A yes attached to a tool name authorizes every future call of that tool, which is a standing grant wearing the clothes of a single decision.
Run the query that counts the rows, resolve the recipient list, expand the cohort predicate. This is nearly always cheap, and it is the difference between asking someone to accept a risk and asking them to accept a described risk.
No highlighted accept, no enter key default, no accept all across a set without an explicit target predicate. A default answer converts the decision into an interruption to be cleared, and clearing interruptions is what people become fast at.
Severity is computable and almost nobody computes it. An irreversible act on production and a reversible act on a scratch branch should not look the same, and if they do, the interface is training the reader to treat them the same.
Bind the yes to a fingerprint of the exact act and to a window it is valid in. Both are a few lines. Together they are what makes the record able to answer a question afterwards.
The most useful thing a person can say is that the evidence is not sufficient. If the only two buttons are approve and deny, that answer has to be smuggled in as a deny, and the reason is lost.
A person who can only accept or refuse cannot improve the decision. A person who can dispute a claim and have the correction land on the thing being decided can. This is the part that is hard, and it is the part that separates deciding from signing.
The person accountable for the act is frequently not the person watching the process. A prompt that can only be answered by whoever is at that keyboard selects the approver by proximity rather than by authority.
Where Modyl sits
Modyl is built for the runtime sense of the term and for one part of it in particular: the moment where the evidence has to be present before the person arrives. An agent brings a decision into a room, the material is assembled around it, claims can be challenged and corrected in place, and the decision is settled with a named owner and a time, or held because the evidence is not there.
The bound is worth stating plainly: Modyl is in a gated alpha, it takes no action on anyone’s behalf, and a proposed next step always waits for a person to confirm it. The record it keeps is the product. The room is where the record gets made.
Where this shows up in practice