The NAIC AI Risk Evaluation Supplement, Exhibit by Exhibit

The NAIC AI evaluation tool currently exists in two texts. Version 4.0, whose footers carry the name AI Systems Evaluation Tool, is the text the twelve pilot states have used since March across market conduct exams, financial exams, financial analyses, and general regulatory inquiries.¹²³ Version 5.0, the Artificial Intelligence (AI) Risk Evaluation Supplement, was exposed for written comment after the Big Data and Artificial Intelligence (H) Working Group’s August 31 session.⁴⁵

The supplement sits inside the broader insurance AI governance map.

Status, verified October 9, 2026. Nothing has been adopted. The working group’s page exposed version 5.0 “for a 30-day comment period ending Tuesday, September 29, 2026,”⁵ and that window has closed. The page links the comments it received, posts the agenda for an October 8, 2026 public meeting on the supplement, and lists the next public meeting for November 2, 2026; no version 6.0 is posted there.⁵ The posted file still carries DRAFT in its footers and prints no version number of its own.⁴ The August 31 meeting and the exposure posting are covered in our news item on the exposure.

Two texts, one instrument

Version 5.0 never uses the old name in its body.⁴ The working group’s summary of its August 13 session connects the two: in the July 22 minutes it adopted, it records an update on the “AI Risk Evaluation Supplement (formerly known as the AI Systems Evaluation Tool)” pilot process.¹ The exposure notice calls 4.0 “the prior version.”⁵

The pilot adds a third layer. Each participating jurisdiction has the authority to modify the tool during the pilot,⁶ so a request received this year may not match either posted text line for line. Cite the exhibit and question you are answering, and treat the request itself as the controlling document.

What the supplement is, and what it is not

Version 5.0 keeps 4.0’s self-description. It supplements existing market conduct, financial analysis, and financial examination review procedures. Regulators are to keep treating existing NAIC resources as authoritative, and inquiries under it are coordinated with the Market Regulation Handbook, the Financial Condition Examiners Handbook, and the Financial Analysis Handbook.⁴ Its exhibits are optional. A new paragraph says some context is deliberately left out, “particularly related to deciding which companies are in scope of inquiry,” because the Handbooks govern those decisions.⁴ Staff’s Summary of Changes explains that earlier comment periods raised the scope of the document’s use.⁷

Three consequences follow.

Requests will be trimmed. Every exhibit still tells regulators to customize it “to limit information requested to more targeted inquiries for use in a limited scope exam.”⁴ Version 5.0 adds an example. A regulator reviewing claims practices who concludes from Exhibit A that the company’s AI use is out of scope for a claims-related targeted exam may rely on the Market Regulation Handbook for the remaining work.⁴ Answer the request you receive rather than the public file, and do not present a carrier-built worksheet, including ours, as an official regulator form.

Prior work can count. If the information has already gone to that department or any other state insurance department, the response should say so, and regulators may accept it if it is still current and applicable.⁴ That is permission rather than reciprocity, so an answer given in one state has to survive being read next to one given in another.

Answers shape scope, and the authority comes from elsewhere. Responses feed the regulator’s view of the insurer’s inherent risks and should affect the nature, timing, and extent of further procedures.⁴ Version 5.0 adds that the information “will be held confidential, consistent with the requesting state’s relevant authority,” and keeps the instruction that regulators cite examination or other authority, and the relevant confidentiality statutes, when they ask.⁴ The obligation to respond and the protection for what is filed both come from that cited authority. The exhibits supply the questions.

Guidance that now comes before the exhibits

Version 5.0 inserts a General Guidance section ahead of Exhibit A, and three parts of it change how the exhibits read.⁴

GLMs. The section names generalized linear models (GLMs) as one machine learning technique. It says they are “not without risk of causing unfair discrimination or other adverse consumer outcomes” and need governance from data quality through validation, monitoring, and legal compliance. Exhibit A asks about specific model types “to give regulators flexibility in determining their scope of interest.”⁴ Read plainly, a GLM is inside the draft’s scope, and the model-type columns let a regulator decide how far to go after seeing the counts.

Materiality. Regulators may either specify a materiality threshold or rely on the company’s own, and the concept borrows the financial examination definition in the Financial Condition Examiners Handbook (FCEH).⁴ A company’s internal threshold can be relied on when it comes with supporting information, including why that threshold was considered appropriate. Materiality is now defined as “the dollar amount above which the examiner’s perspective of an insurer’s financial position will be influenced.” Material Financial Impact rests on either the company’s disclosed threshold or one the regulator provides.⁴

Risk assessment. Where the supplement relies on a company’s risk assessment, the focus is inherent risk: the risk of economic loss or inaccurate financial reporting before considering internal controls.⁴ Both passages end by saying nothing in the supplement supersedes FCEH guidance.

How a regulator decides what to send

Version 4.0 opened with a routing table headed “Which Exhibit to Use?”² Pilot requests have been built on 4.0, so the table is still the best key to reading one.

Only the adverse consumer outcome inquiry calls for all four exhibits. Our reading is that the shape of a request carries information: a lone Exhibit D asks a data question, a B and C pairing asks a controls question, and the full set points to the consumer outcome track. That is an inference from the table, not a rule.

The version 5.0 text contains no such table, and the Summary of Changes does not mention one.⁴⁷ Routing in the draft runs through Exhibit A. Regulators “may wish to first use Exhibit A to determine if further inquiry using Exhibits B-D is necessary,” and limited or low-risk AI use may not warrant further inquiry at all.⁴ Staff say splitting the counts by model type shows whether a company’s AI use is limited to one type, which may limit further inquiries.⁷

Exhibit A: counts by model type, then a request list

Exhibit A keeps its title, “Quantify Regulated Entity’s Use of AI Systems,” and its job. Its counts help regulators decide whether to ask about the AIS Program in Exhibit B, high-risk models in Exhibit C, and data types in Exhibit D where there is risk of adverse consumer outcomes or material adverse financial impact.⁴ It now comes in two parts, and they ask for different things.

Part One is a grid of counts and use cases, not a list of models. Its rows are thirteen operational or program areas, from marketing and underwriting through claims, utilization management, fraud, reserves, and reinsurance, ending in a catch-all row.⁴ Version 4.0 had fifteen rows.² Version 5.0 counts AI Systems in use and those implemented in the past X months, with the regulator choosing X. It then counts AI Models twice, once for direct consumer impact and once for material financial impact, each split into GLMs, generative or agentic AI models, and other AI or ML models. A final column asks for the use cases in each area.⁴ Version 4.0 asked four flat counts with a fixed twelve-month window.²

The instructions hold two changes a 4.0 reader would not expect. The regulator either sets a materiality threshold or asks for the company’s, and the header has a line for it. And: “If a company has an existing AI Systems inventory, it may suggest submission of the inventory in lieu of completing Exhibit A.”⁴ The verb is suggest. The company can propose the swap; the text does not say the regulator must accept it.

The direct consumer impact rule survives: models that augment or automate decision-making related to consumers count.⁴ New is a formal definition behind it. Direct Consumer Impact covers an AI System that augments or automates human decision-making “whose output has the potential to create final consumer outcomes,” such as underwriting, premium, ratemaking, and claims decisions. It “has less applicability to AI Systems used for internal analytics, model monitoring, and back-office tooling with no consumer-visible output.”⁴ The autonomy definitions are unchanged.²⁴ Two carriers running the same technology can still report different counts without either misreporting, so the classification rule belongs on paper before the number does.

Part Two is where lists come in. It is a request list that, in staff’s account, makes explicit several requests 4.0 only implied.⁷ It asks for an explanation of the company’s materiality calculation, documentation of its AI risk assessment process, and a model inventory for program areas the regulator names. For each AI Model, the inventory or documentation alongside it gives the name, how the model is used, the broader use case, the program area, the inherent risk level under the company’s own risk assessment, and its consumer and financial impact.⁴ Answering Part One still means supplying numbers and use cases by area. Part Two is a separate request, and the AI inventory playbook covers building a register that can satisfy it.

Exhibit B: the AIS Program, two ways

In 4.0, Exhibit B was the “AI Systems Governance Risk Assessment Framework.”² Version 5.0 titles it “AIS Program (Two Options: Narrative or Checklist).” It defines the AIS Program, drawing on the Model Bulletin, as “the controls and processes that an insurer adopts and implements to ensure the responsible use of AI Systems.”⁴⁷ Its preface says the questions “are not intended to be interpreted as creating new requirements for AIS Program.”⁴

The narrative version starts where 4.0 did. The company provides its program, then discusses who maintains it, board reporting, integration and remediation, effectiveness reviews, assigned responsibility, and how AI Systems enter Own Risk and Solvency Assessment (ORSA) and enterprise risk management (ERM) work.⁴ The question on assessing autonomy, reversibility, and reporting impact risk was marked in 4.0 as a suggested additional question. In 5.0 it sits in the list without that label.²⁴

Two additions change what a good answer looks like. For uses with direct consumer impact, the response should now cover “whether there is a human in the loop, whether there are high volume transactions, how any exceptions or errors are found and resolved, and whether there were any staffing reductions as part of the AI System implementation.”⁴ A new item asks how the company ensures explainability and transparency into AI models recommending or making decisions.⁴⁷ Other questions cover financial uses, deviations from established IT practice, vendor oversight, independent validation, and professional service providers such as actuarial, claim, MGA, and audit firms.⁴

The checklist asks whether a written AIS Program has been adopted, when, how often it is reviewed, and what role the board or management plays. Item 3 then asks where the program addresses sixteen elements, 3a through 3p, with a column for the document name and page number.⁴ Version 4.0 listed fourteen.² The new elements are 3o, how the company determined materiality and what threshold it used, and 3p, how it governs, monitors, tests, and ensures transparency of AI Systems developed by vendors.⁴⁷

The page column is the part worth rehearsing. Companies citing a page are told to name the document and include it in the submission.⁴ An element nobody can point to reads as a gap however well the practice works. The framework implementation guide turns these areas into an operating sequence, and the Model Bulletin guide explains the authority behind them.

Exhibit C: fifteen requests per high-risk model

Exhibit C is now “High-Risk AI Models.” It asks about “in-production high-risk AI models,” for example models making automated decisions that could cause adverse consumer outcomes or material financial, consumer, or financial reporting impact.⁴ The scope rule stays with the insurer: “AI model risk criteria are set by the insurance company.” The draft adds: “Note that materiality should be a consideration in determining the level of risk.”⁴ Which systems fall inside that company-set scope is a separate question, and screening the inventory is where it gets answered.

Fifteen requests follow for each model, where 4.0 had thirteen.²⁴ Five identify the model: name and version, use cases and purpose, model type, implementation date, and how it was developed. Model type now carries the examples GLM, generative or agentic AI, or other AI/ML algorithm. Development now distinguishes internal, “owned” third party, true third party, or a mix, with the vendor name.⁴⁷ A new sixth asks whether the company bought the model, whether it was built for the company, and whether the company can modify it.

Four ask how the company classifies the model. They cover risk classification, on a scale that now reads high, moderate, low; model risks, framed as potential adverse consumer impact; limitations, including inputs outside the training data’s domain; and whether it automates, augments, or supports.²⁴

The last five ask for evidence and legal history. The first covers output testing for drift, accuracy, unfair trade practices, unfair discrimination, and performance degradation, plus validation and monitoring. The others ask for the last test date, the effect on financial statements or controls, the compliance review against state and federal law, and, where law permits, any action taken against the company over the model.⁴

One citation trap: the Summary of Changes describes the risks and limitations split as fields 6 and 7, while the draft’s own table numbers them 8 and 9.⁴⁷ Answer against the table.

Exhibit C still has no human-override field. Human involvement is asked about in Exhibit B instead, so a claims model’s review design has to read the same in both exhibits. Agentic AI in Claims examines what that looks like in practice, and deployed-model thresholds belong to model monitoring.

Exhibit D: data categories tied to named models

Exhibit D, now “AI Model Data,” asks for “the source(s) and type(s) of data used in an AI model(s)” so regulators can see where consumer, unfair trade practice, and financial risk may sit.⁴ Its grid keeps 4.0’s twenty-five categories in the same order, from aerial imagery and age, gender, ethnicity/race through driving behavior and income to reasonable accommodations, voice analysis, and weather. The twenty-fifth is an open row for non-traditional data, which the regulator now specifies.²⁴

The form around the grid changed. A header line asks the company to list the AI System or AI models relying on the data described, which let staff delete a column that repeated that question on every row.⁴⁷ Each category then asks how the model uses the data, whether the company sources it in-house, and which outside vendor supplies it if not.⁴ Rows stay blank for data not used in developing a model, and one response is expected per line of business. Regulators may now tailor the exhibit by line of business and, depending on responses, ask for a data dictionary.⁴⁷

The exhibit maps sources. It asks nothing about proxy testing, data lineage, or overrides, so material on those topics belongs under whatever obligation requires it. The Exhibit D playbook owns the field-by-field preparation; it is written against the 4.0 grid, whose categories carry over.

The new definitions

Staff record eight added definitions and one removal, “Degree of Potential Harm to Consumers.”⁷ Beyond those already covered, two shape answers.

Agentic AI means AI Systems that can “pursue objectives across multiple steps without requiring human input at each stage.”⁴ Because Exhibit A’s columns and Exhibit C’s model-type field put generative and agentic AI in one bucket, a carrier tracking agentic deployments separately will fold them together to answer.

AI Model is “an integral component of an AI System that learns from data to perform tasks via computational, statistical and/or machine learning techniques.”⁴ It is the unit behind Exhibit A’s model counts, Exhibit C’s requests, and the model list atop Exhibit D. AI System stays the unit for Exhibit A’s first two count columns.

The seams a reviewer can test

The exhibits describe one program from four angles, and version 5.0 adds a thread, materiality, that runs through three of them.

A polished answer still fails when another department produces a record that contradicts it.

Where the draft goes from here

The pilot began in March and was scheduled to run through September, though the working group notes that not every state’s pilot may finish by September 30.¹³⁶ Version 5.0 is still a draft. Written comments on it were due September 29,⁵ and adoption has not happened.

The working group’s timeline slide plans an October meeting for comment on the current draft and a version 6.0 exposed for 14 days that month, then a version 7.0 to be considered “for adoption at Fall NM.”³ The 2026 Fall National Meeting runs in Dallas, November 14 through 17.⁸ A slide records a plan, and the step this one names for November is consideration, not adoption. Even an adopted version would remain optional supplemental material whose force in any request comes from the authority the regulator cites. Adoption by the NAIC would also leave each state free to use the exhibits or not, much as the Model Bulletin itself has been adopted state by state.

Preparing in the order the draft implies

Under 4.0 the register came first because the counts depended on it. Under version 5.0 one number comes before the register. The Exhibit A header asks for a materiality threshold, and Part Two asks for the explanation behind it. Checklist item 3o asks how it was determined, Exhibit C says materiality should inform risk levels, and Material Financial Impact is defined by it.⁴ A threshold chosen separately for each exhibit will produce answers that disagree with each other.

Then the inventory, with the Part Two fields completed for every model. Then the Part One counts, drawn from that inventory and split by model type. Then the actual request, read against the routing table while pilot requests still rest on 4.0. Narrative answers come last, after the numbers agree. Organization-wide exam packaging is covered in the exam readiness guide.

Running that sequence against your own records shows which version 5.0 mechanics your company can work with. The working group’s exposure notice named Scott Sobel and Miguel Romero of NAIC staff as the contacts for written comments, which were due by close of business Tuesday, September 29.⁵ Two mechanics are new in this draft and deserve the closest test. One is whether a company-disclosed materiality threshold can carry the weight the exhibits now put on it. The other is whether an existing inventory offered in lieu of Exhibit A actually holds the fields Part Two asks for.

Footnotes

1. National Association of Insurance Commissioners, Big Data and Artificial Intelligence (H) Working Group meeting summary report, August 13, 2026, 2026 Summer National Meeting, Columbus, Ohio, circulated as Attachment Two to the Innovation, Cybersecurity, and Technology (H) Committee, August 14, 2026. One page; item 1.B for the former name and item 2 for the pilot’s start in March, twelve participating states, uses, and continuation through September. ↩ ↩² ↩³

2. National Association of Insurance Commissioners, Artificial Intelligence Systems Evaluation: Optional Supplemental Exhibits for State Regulators, posted by the Big Data and Artificial Intelligence (H) Working Group as the clean version 4.0. Page footers read “AI Systems Evaluation Tool” and every page carries a DRAFT watermark. Locators used here: “Which Exhibit to Use?”, p.3; Exhibit A, pp.4 to 6; Exhibit B narrative, pp.7 to 8, and checklist, pp.9 to 10; Exhibit C, pp.11 to 12; Exhibit D, pp.13 to 15; Definitions and Appendix, from p.16. Retrieved August 28, 2026. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹

3. National Association of Insurance Commissioners, Big Data and Artificial Intelligence (H) Working Group, materials for the August 31, 2026 meeting, 40 pages. Attachment 2 slides: pilot timeline with its footnote on state timing, p.10; the pilot slide summarizing changes from v4.0 to v5.0, p.11. Draft supplement, pp.12 to 34. ACLI letter of July 21, 2026, p.37, which describes v4.0 as “the version of the Tool used in the pilot.” Retrieved September 18, 2026. ↩ ↩² ↩³

4. National Association of Insurance Commissioners, Artificial Intelligence (AI) Risk Evaluation Supplement, version 5.0, clean, Word file posted on the working group’s page as the exposure draft. Footers read “DRAFT AI Risk Evaluation Supplement”; the file prints no version number. Cited by section: Background, Intent, and Instructions; General Guidance (Machine Learning Models, Materiality, Risk Assessment, Confidentiality); Exhibit A Parts One and Two; Exhibit B narrative and checklist; Exhibit C; Exhibit D; Definitions and Appendix. The same text is the 23-page draft in Attachment 2 of the August 31 materials packet, where Intent is on packet p.12, the Instructions on p.13, the machine learning guidance on pp.13 to 14, Materiality, Risk Assessment, and Confidentiality on p.14, the Exhibit A grid on pp.17 to 18, Exhibit A Part Two on p.19, and the new definitions on pp.31 to 33. Retrieved September 18, 2026. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵ ↩¹⁶ ↩¹⁷ ↩¹⁸ ↩¹⁹ ↩²⁰ ↩²¹ ↩²² ↩²³ ↩²⁴ ↩²⁵ ↩²⁶ ↩²⁷ ↩²⁸ ↩²⁹ ↩³⁰ ↩³¹ ↩³² ↩³³ ↩³⁴ ↩³⁵ ↩³⁶ ↩³⁷ ↩³⁸ ↩³⁹ ↩⁴⁰ ↩⁴¹ ↩⁴² ↩⁴³ ↩⁴⁴ ↩⁴⁵ ↩⁴⁶ ↩⁴⁷ ↩⁴⁸ ↩⁴⁹ ↩⁵⁰

5. National Association of Insurance Commissioners, Big Data and Artificial Intelligence (H) Working Group, Exposure Drafts section, which reads “we are exposing the AI Risk Evaluation Supplement version 5.0 for written comments and feedback, for a 30-day comment period ending Tuesday, September 29, 2026,” describes the linked summary as covering changes “from the prior version (4.0),” and directs comments to Scott Sobel (ssobel@naic.org) or Miguel Romero (maromero@naic.org), due at close of business on the deadline date. As of October 9, 2026 the same section also links a file titled “Comments Received on version 5.0 exposure” and shows no later version, and the page lists a public Webex meeting on Monday, November 2, 2026 “to continue public discussions on the AI Risk Evaluation Supplement,” the purpose it also gives for the meeting of Thursday, October 8, 2026, whose agenda it links. Retrieved October 9, 2026. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

6. National Association of Insurance Commissioners, AI Systems Evaluation Tool Pilot: Pilot Project Background, undated, p.2: “States will use the Tool from March 2026 to September 2026,” and each jurisdiction “has the authority to modify the Tool in the pilot to meet its needs.” Retrieved September 18, 2026. ↩ ↩²

7. NAIC staff, AI Risk Evaluation Supplement, Summary of Changes, two pages, posted beside version 5.0 and identical to pp.35 to 36 of the August 31 packet. Items cited: Intent; Exhibit A (model-type columns, explicitly listed requests); Exhibit B (AIS Program focus, narrative 3d, 3o, 3p); Exhibit C (field 3 examples, field 4 development, the risks and limitations split); Exhibit D (model list and deleted column, line-of-business instruction); Definitions (eight added, one removed). Retrieved September 18, 2026. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹²

8. National Association of Insurance Commissioners, Events, listing the 2026 Fall National Meeting in Dallas, November 14 through 17, 2026. Retrieved September 18, 2026. ↩