NAIC AI Evaluation Tool Guide to Exhibits A-D

What the NAIC AI Systems Evaluation Tool asks across Exhibits A to D, in what order, and where carriers tend to fall short before an exam.

Hand a carrier the NAIC AI Systems Evaluation Tool and the first instinct is to read it like a supply list. Bias testing, we have that. Vendor files, somewhere. A model inventory, more or less.

The tool does not read like a supply list. It reads like a sequence, where each of the four exhibits, A through D, quietly assumes you can already answer the one before it. You cannot describe governance over systems you never counted. You cannot detail a high-risk model you never flagged as high-risk. By the time an examiner reaches the data questions in Exhibit D, they are standing on three exhibits’ worth of answers you were supposed to have ready. The order of the exhibits is not housekeeping. It is the design of the exam.

This article walks each exhibit in turn: what it asks for, what documentation it expects, and where the gaps usually open up.

What the tool is

The NAIC AI Systems Evaluation Tool is a supplemental examination framework for state regulators, developed by the NAIC Big Data and Artificial Intelligence Working Group and released as Version 4.0 in 2026. It exists to help regulators find and assess AI-related risk during market conduct exams, financial condition exams, and financial analysis.

It does not create new requirements. It bolts AI-specific questions onto the examination handbooks regulators already use. If your state has adopted the NAIC Model Bulletin or sits in the 12-state pilot, this tool is how an examiner is likely to come at your AI during the next exam.

The four exhibits break down like this:

Exhibit	Purpose	Format
A	Quantify AI Systems use	Quantitative table
B	AI Governance Risk Assessment	Narrative or checklist
C	High-Risk AI Systems Details	Detailed model questionnaire
D	AI Systems Model Data Details	Data source inventory

Which exhibits you actually face depends on your risk profile. A carrier with few AI systems and low consumer impact may only work through A and B. A carrier running high-risk AI in claims or underwriting should expect all four.

FIG. 1 — THE FOUR EXHIBITS ARE A SEQUENCE, NOT A CHECKLIST

Exhibit A: quantify your AI usage

Exhibit A is where most examiners start, for the same reason the bulletin starts with governance: you cannot demonstrate control over systems you have not identified. The exhibit asks for a quantitative inventory of every AI system, sorted by operational area.

It helps to know how broadly the tool defines its terms. An AI system, in its own words, is “a machine-based system that can, for a given set of objectives, generate outputs such as predictions, recommendations, content (such as text, images, videos, or sounds), or other output influencing decisions made in real or virtual environments.” AI Systems are described as operating with varying levels of autonomy, from supportive to augmented to automated ¹. That definition is wide on purpose. It is meant to catch the embedded tool and the vendor feature, not just the model the data science team thinks of as a model.

The operational areas the exhibit tracks run well past underwriting and claims ¹:

Marketing
Premium quotes and discounts
Underwriting and eligibility
Ratemaking and rate classification
Claims and adjudication, including salvage and subrogation
Customer service
Utilization management and prior authorization
Fraud, waste, and abuse detection
Investment and capital management
Legal and compliance
Producer services
Reserves and valuations
Catastrophe triage
Reinsurance
Other insurance practices

For each area, you are asked to provide the number of models in use, whether they directly affect consumers, whether they carry material financial impact, whether they went in within the past 12 months, and the specific use cases.

If the inventory is incomplete, this is where the examination stalls before it really begins. The practical move is to start with the systems that touch consumers directly and work outward into the back office from there.

Exhibit B: assess your governance framework

Exhibit B tests whether your AI governance is documented, active, and wired into your broader risk management. The tool gives you two ways to answer: a narrative response or a checklist.

The narrative format covers five core governance areas:

Governance framework: roles, board reporting, organizational integration, effectiveness assessment, responsibility assignment, and ORSA/ERM integration.
Material AI uses: financial transactions, consumer impact, and risk and control assessments, along with development, testing, and implementation details, including any deviations from established IT protocols.
Vendor and third-party oversight: validation and testing procedures for internal and third-party AI systems, with testing frequency, scope, and methodology.
Professional service provider oversight: actuarial, claims, MGA, and audit functions, with the testing and verification behind them.
Additional framework design: responsible units, assessment approach, and frequency.

The tool also floats a suggested additional question, worth reading in full because of how pointed it is: “How does the insurance company assess autonomy, reversibility, and reporting impact risk of AI Systems?” ¹

The checklist format asks whether you have a written AI Systems Program, when it was adopted and how often it is reviewed, how board and management are involved, and then runs through fourteen specific governance elements, labeled 3a through 3n, covering items such as unfair trade practice risk assessment, state and federal compliance, and adverse consumer outcome tracking ¹.

The choice between formats is itself a small strategic decision. The checklist is faster for an examiner to review, and if your program maps cleanly onto the fourteen elements, that speed works in your favor. If your program has soft spots, the narrative format gives you the room to explain them on your own terms rather than leaving a blank box on a checklist.

Exhibit C: detail your high-risk AI systems

Exhibit C is where the examination goes deep. It concentrates on systems that could do real consumer or financial harm, especially in claims, underwriting, and pricing.

The tool treats a system as high-risk when it ¹:

Makes or materially influences decisions about coverage, pricing, or claims settlement
Is an agent-facing system that feeds recommendations to producers
Operates with limited human oversight or reversibility

For each one, you are expected to document the decision logic and model architecture, the human override mechanisms, the monitoring logs and performance metrics, the validation and testing records, the bias and disparate impact analysis, and how often the model is revalidated ¹.

The recurring theme is provability rather than accuracy. A model can be accurate and still be ungoverned, and Exhibit C is built to surface the second condition. Claims is where this exhibit tends to bite hardest, both because autonomy escalates fastest there and because the documentation is thinnest, and we take that case apart on its own in a separate article on Exhibit C and agentic AI in claims.

Exhibit D: document your AI data inputs

Exhibit D turns to the data feeding your systems. It asks for an inventory of data sources, your proxy discrimination screening, and, new in Version 4.0, a field for “reasonable accommodations or policy modifications” data.

The elements it tracks ¹:

Primary data sources, internal and external
Proxy variables that may stand in for protected class status
Data quality and validation procedures
Frequency of data updates
Whether data sources are audited or certified

Per Fenwick, Version 4.0 also simplified the earlier direct and indirect impact references, reinstated references to unfair trade practices for model testing, and added that new accommodations data field ². Several questions are still open as of June 2026, including how materiality gets defined, whether generalized linear models fall in scope, and the exact terminology the tool will settle on.

Data is usually the weakest link, and the reason is mundane: it is the documentation nobody kept. If you cannot say where a model’s training data came from, or show that it has been screened for proxy discrimination, Exhibit D is where that absence becomes visible.

The documentation an exam actually wants

Across all four exhibits, the underlying demand is the same. Regulators want evidence that governance is operating, not a statement that it exists.

WaterStreet, an insurance technology vendor, put a usable shape on this in a May 2026 guide for carriers. It lists five documentation requirements worth keeping ready: a written AI Systems Program, a complete model inventory with risk tiering, dated testing and validation records that include bias analysis, third-party vendor oversight files with audit rights, and adverse consumer outcome tracking with incident response protocols. The guide’s blunt version of the point is that outsourcing the AI does not outsource the compliance obligation ³. If a technology vendor cannot hand you model documentation, validation records, or audit support, that gap follows you into the exam, not the vendor.

That lines up with the bulletin itself, which is explicit that the AIS Program has to cover AI systems whether the insurer developed them or acquired them from a third party, and that the due diligence and audit rights stay with the insurer ⁴.

Getting ready before the request lands

The evaluation tool is a present documentation requirement, not a future risk. If you operate in an adopting or pilot state, the safe assumption is that an exam could include these questions. Three moves get you most of the way to knowing where you stand.

Build the inventory first. Work across every operational area, not just the obvious two, and capture function, data inputs, decision context, owner, vendor, and last validation date for each system. Everything downstream in the tool depends on this list being honest and complete.

Map your governance to the fourteen checklist elements. Walk 3a through 3n and mark which you can already evidence and which you cannot. The gaps become a short list with owners and dates attached, which is a far better thing to hand an examiner than a surprised silence.

Pressure-test your high-risk and data documentation together. For any system that materially shapes coverage, pricing, or claims, pull the decision logic, the override records, and the data lineage, and see whether they actually exist in a form you could produce. The question to ask yourself is not whether the documentation is perfect. It is whether you could put your hands on it without a scramble.

Most carriers do not fail this tool on any single exhibit. They fail on the seams between them, where the inventory does not match the governance narrative, or the high-risk system has no data lineage behind it. The exhibits are built to test those seams in order, which means the work of preparing is mostly the work of making the four answers agree with each other before someone else checks whether they do.

InsureAI Wire tracks NAIC and state-level AI governance developments weekly. Subscribe for updates on the evaluation tool pilot, state adoption maps, and compliance checklists.