Human in the loop is an architecture decision

Every AI program reaches a meeting where someone asks whether a person will check the output. Answer yes to all of it and the throughput case disappears, because a reviewer who checks everything is the bottleneck the system was supposed to remove. Answer no to all of it and the first bad outcome ends the program.

Both answers come from asking the question at the wrong level. Review is not a property of the system. It is a property of each effect the system can produce.

Sort effects, not conversations

• Reads inside the caller's scope. No review. Log them.
• Reversible writes below a value threshold. Execute, keep a one-step undo and a window.
• Reversible writes above the threshold. Execute, notify the owner, keep the undo.
• Irreversible effects. Human confirmation before, always.
• Anything touching money leaving the company, or a customer commitment. Confirmation, regardless of value.

# The gate belongs on the effect, not on the conversation.
CONFIRM_ABOVE_CENTS = 50_000

def apply(effect: Effect, caller: Principal) -> Outcome:
    if effect.reversible and effect.value_cents < CONFIRM_ABOVE_CENTS:
        return execute(effect, caller)                     # log, do not ask

    if effect.reversible:
        return execute_with_undo(effect, caller, window=timedelta(hours=24))

    return queue_for_review(
        effect,
        summary=describe_in_business_terms(effect),        # not a serialized tool call
        sla=timedelta(hours=4),
    )

Make the confirmation worth someone's attention

A confirmation screen showing a serialized tool call gets approved without being read within about a week. Show the effect in business terms: what will change, for whom, what it costs, and what the system based it on. If a reviewer cannot disagree with it in ten seconds, the gate is theatre.

Plan to move the line

The boundary should tighten as evidence accumulates. Start with confirmation on a class of effect, measure the override rate for a month, and if it is low, move that class down a level with the data attached to the decision. That gives the risk committee something to approve other than a promise, and it gives the team a path to the throughput case that does not depend on anyone's confidence.

More field notes