NAIC AI Evaluation Tool

The NAIC AI Systems Evaluation Tool is a set of exhibits state insurance regulators can send an insurer to structure what they learn about its AI systems. The document describes itself as optional and supplemental, tells regulators to trim it to the inquiry at hand, and still carries a DRAFT header on every page at Version 4.0. Regulators are told to start with Exhibit A and decide from those answers whether to go further, so the other three branch off those counts rather than off each other:

• Exhibit A, quantifying the entity’s use of AI systems: counts by operational area, plus how many reach consumers directly, how many are financially material, and how many launched within the last year.
• Exhibit B, the governance and risk assessment framework, answerable as a narrative or as a checklist: who maintains the framework, board reporting, validation and testing policy, and oversight of vendors and professional service providers.
• Exhibit C, high-risk AI systems in detail, model by model: type, implementation date, build-or-buy, risk classification, validation and monitoring, last testing date, and any regulatory action taken over it.
• Exhibit D, the data details: a checklist of data-element categories, marking which ones feed the models and, for every one you check, whether the source is internal or a named vendor.

The tool is not itself a regulation, and it is not exam procedure. It is a supervisory aid, and the instruction to regulators is to send only what a given inquiry needs. So the four exhibits together mark the outer limit of what an insurer might be asked about its AI, and any particular examiner may stop well short of it. Our guide to the NAIC AI Evaluation Tool walks through the exhibits one at a time.