Human-in-the-Loop
A design where a human reviews or approves an AI system's output before it is acted on. Regulators expect it for adverse decisions; a few statutes require it.
Human-in-the-loop means that an AI system’s output is reviewed by a human before it is used to make or execute a decision. In insurance it is what regulators expect to find behind high-risk decisions such as coverage denials, prior-authorization rejections, and claims denials, though only a few statutes actually compel it.
Regulators look past the formal setup. A human reviewer who almost always agrees with the model, or who has no time to understand the recommendation, is not meaningful oversight. The NAIC evaluation tool has no override-log question; for each high-risk system it asks whether the AI automates, augments, or supports the decision, and it defines augmentation as advising the person who decides. Our test for a system a carrier calls augmented is whether the record shows humans sometimes deciding otherwise, because review that leaves no trace of disagreement is a rubber stamp.
California’s SB 1120 makes the requirement explicit for medical-necessity decisions: a licensed clinician must make the final determination. Even without such a law, carriers should design escalation paths so that consequential AI recommendations can be challenged by a person with the right authority and expertise. See our guide to agentic AI in claims.