Source: NAIC official document, content.naic.org (opens in a new tab)
The draft minutes of the NAIC Big Data and Artificial Intelligence (H) Working Group’s October 8 meeting sit at the front of the materials file for its November 2 Webex, attached to the agenda item on whether to adopt them. Dated October 23, they record the chair, Commissioner Nathan Houdek of Wisconsin, describing how regulators expect to use the AI Risk Evaluation Supplement after its planned adoption. Their sentence on the examination handbooks reads: “Possible incorporation of elements of the supplement into the financial or market conduct handbooks will be discussed next year, and no decisions have been made.”
In the minutes’ summary of oral comments, LaCosta Wix of AHIP recommended incorporating a prescribed AI review approach into the financial condition and market conduct examination handbooks. Randi Chapman of BCBSA recommended referring the supplement to the Market Regulation and Consumer Affairs (D) Committee and the Financial Condition (E) Committee to guide its use through existing handbooks. Houdek said any referrals to those committees would follow future decisions on handbook incorporation. On our reading, adoption on November 16, if it comes, would not by itself change the handbooks that financial condition and market conduct examinations are run from.
The pilot used the supplement broadly to test every exhibit. After adoption, Houdek said, regulators anticipate “a funneled, risk-focused approach.” Companies using AI would receive Exhibit A requests, and many would also receive Exhibit B inquiries. Based on a risk assessment, some would be asked to complete Exhibits C and D “for a specific set of AI models or data sets rather than for a large number of models.” Those two exhibits will be kept as templates for deeper reviews, though not every company would receive them. Language will be added to reflect this intention, the minutes say, and our report on version 6.0 covers the instructions that draft added.
Houdek also told the October 8 meeting that the working group would clarify that existing company documents may be submitted in lieu of responses if they provide the information regulators need. It would likely provide an anonymized example of a company submission once the supplement is adopted. The working group expected to add questions on agentic AI, though not all suggestions could be included at that point, he said, and the supplement will keep evolving, with the adopted version as its foundation. Version 6.0, exposed later in October, added one.
One handbook already shapes the supplement’s materiality wording, by the account of Miguel Romero of NAIC staff. That language, he said, lets regulators either accept a company’s explained materiality or rely on the materiality already set for scoping examinations. It is intentionally not prescriptive, to avoid conflicting with the Financial Condition Examiners Handbook, which is tied to accreditation requirements.
Coral Manning of Wisconsin reported that 18 pilot companies had given feedback through a survey of more than 40 questions, followed by one-on-one discussions with states. That feedback generally tracks the comment letters on where the supplement was unclear, she said, and a longer pilot update with anonymized pilot information and survey data is planned for the Fall National Meeting.
Some companies reported in the pilot survey using GLMs outside ratemaking, Romero noted. He had asked whether excluding GLMs used in ratemaking would address commenters’ concerns, and Lindsey Stephani of NAMIC had said GLMs are used only in ratemaking, so they need not be in scope at all.
The file’s timeline slide puts “Consider v. 7.0 for adoption at Fall NM” last in November, and the working group’s Fall National Meeting session is on November 16. As of November 3, its page has posted no version 7.0 and no account of what happened on the November 2 call.