How Insurers Assess AI Vendor Risk

Among health insurers already using AI or machine learning, 55% build with third-party components and 15% rely entirely on a third-party AI solution, the NAIC found in its 2025 survey of the segment.¹ The number is not surprising. Insurers buy fraud-detection models, underwriting platforms, claims triage tools, and generative AI copilots the same way they buy other software. What is surprising is how many carriers still believe that outsourcing the AI also outsources the compliance obligation.

It does not. The NAIC Model Bulletin is explicit: insurers remain responsible for compliance with insurance laws even when they use AI developed or operated by a third party.² When a regulator walks in with an examination notice, the carrier is the one that must produce the documentation, explain the model, and demonstrate that the system did not produce unfairly discriminatory or inaccurate outcomes. The vendor’s office is not where the examiner will sit. Vendor oversight is one pillar of insurance AI governance; the other three do not cover for it.

This article is a practical checklist for vendor risk assessment in insurance. It translates the NAIC Model Bulletin and related state guidance into questions you can ask before signing a contract, clauses you should insist on, and monitoring you should keep running after the deal is done.

What the NAIC Model Bulletin says about vendors

The NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers, adopted in December 2023,² has since been adopted by roughly half the states, jurisdiction by jurisdiction in our state adoption tracker. It does not create new law. It restates that existing insurance laws, including unfair trade practice and unfair discrimination prohibitions, apply fully to AI-supported decisions.²

Read the third-party guidance closely: its register matters as much as its content. The bulletin’s guidelines run 1.0 through 4.0, and 4.0 is the third-party block. It says each AIS Program “should address” how the insurer acquires, uses, or relies on third-party data and third-party AI systems. What that process contains is left open: standards, policies, procedures, and protocols, “which may include, as appropriate,” three considerations.²

1. Due diligence. The methods the insurer uses to assess the third party and its data or AI systems, so that decisions which could lead to adverse consumer outcomes still meet the legal standards imposed on the insurer itself (4.1).
2. Contract terms, where appropriate and available. Terms that provide audit rights and/or entitle the insurer to receive audit reports from qualified auditing entities, and that require the vendor to cooperate on regulatory inquiries and investigations related to the insurer’s use of its product or services (4.2).
3. Actually exercising those rights. Performing the contractual audit rights, or other activities that confirm the vendor is complying with its contractual and, where applicable, regulatory obligations (4.3).

A different part of the document, Section 4 on regulatory oversight and examination considerations, sets out what a regulator may ask for about third-party systems: the due-diligence work, the contracts, the audits or confirmation processes, and the validation, testing, and auditing documentation including model drift. Section 4 opens by saying an insurer can expect to be asked about its AI systems “regardless of the existence or scope of a written AIS Program.”²

Two things follow from that wording. The hedges are real. “Should address,” “may include, as appropriate,” “where appropriate and available,” audit rights “and/or” audit reports: none of that is a rulebook. The bulletin says so itself in its closing paragraph, where the stated goal “is not to prescribe specific practices or to prescribe specific documentation requirements.”²

Read the rest of that paragraph, though, because it is routinely quoted at half length. The sentence before it says insurers may demonstrate compliance “through alternative means,” including practices that differ from the ones the bulletin describes. The sentence after it gives the goal the bulletin does claim: making insurers aware of the department’s expectations, and of the documents “that the department expects an Insurer to produce when requested.” The method is flexible; accountability remains. The laws underneath the bulletin, unfair trade practices and unfair discrimination, attach to the decision your vendor’s model produced. The checklist below is ours. It turns that standard into twenty questions; the NAIC stopped at the standard.

The New York State Department of Financial Services (NY DFS) took the same principle into a narrower context. Insurance Circular Letter No. 7, issued in July 2024,³ sets its gate on the data itself. An insurer should not use external consumer data or AI systems for underwriting or pricing unless it can establish that the data source or model does not use, and is not based in any way on, a class protected under Insurance Law Article 26. The same letter closes the outsourcing question directly. An insurer may not rely solely on a third party’s claim of non-discrimination or on a proprietary third-party process, and the duty to comply with anti-discrimination laws “remains with the insurer at all times.”³ Colorado’s amended Regulation 10-1-1 then took effect on October 15, 2025, extending the state’s existing governance and risk management framework from life insurers to health benefit plan insurers and private passenger auto insurers.⁴

The pre-signing vendor assessment checklist

A good vendor assessment is risk-based. A vendor whose AI merely drafts marketing copy needs less scrutiny than one whose model recommends coverage denials or sets prices. Sort your vendors by risk first, then apply the following questions. For high-risk vendors, every question below should be answered in writing before contract signature.

The download turns them into a working questionnaire: twenty questions grouped by category with answer and gap columns, a risk-tiering reference tab, and the decision-record template from the section below with its example column pre-filled.

How to classify risk. A simple three-tier model works for most insurers:

• High risk: The AI can directly influence an adverse consumer outcome, such as a coverage denial, a premium increase, a claims payment decision, or a life insurance underwriting declination. These vendors require the full checklist, legal review, and board-level reporting.
• Medium risk: The AI supports operational decisions that affect consumers indirectly, such as fraud detection triage or customer service routing. These vendors require most of the checklist and periodic re-assessment.
• Low risk: The AI is used for internal productivity, such as drafting emails or summarizing documents, with no direct consumer impact. These vendors require basic security and privacy due diligence but not model-level validation.

This classification should be documented and revisited at least annually, or whenever the vendor’s system is expanded to a new use case.

Model and data

1. What is the model’s intended use case, and has it been validated for that specific use case in insurance?
2. What data was used to train or fine-tune the model? Is it internal data, third-party data, or a combination?
3. How does the vendor test for bias, fairness, and accuracy? Can they provide the testing methodology and results?
4. Does the vendor monitor for model drift? If so, how often, and what triggers a re-validation?
5. What is the model’s error rate, and how is that defined and measured?
6. Can the vendor provide model documentation, including a description of the model architecture, inputs, and outputs?
7. Are protected-class proxies, such as zip code or credit data, used as inputs? If so, how are they managed under state insurance law?

Security and privacy

8. Where is insurer data processed and stored? Is it in the United States, and does it cross borders?
9. Will the insurer’s data be used to train the vendor’s models or improve products for other customers?
10. What access controls and encryption standards are in place?
11. Does the vendor have a documented incident response plan? Has it been tested?
12. Does the vendor comply with relevant insurance data privacy requirements, including state-specific laws?

Governance and compliance

13. Does the vendor have a written AI governance framework? Who is accountable for it?
14. Is the vendor familiar with the NAIC Model Bulletin and relevant state insurance AI requirements?
15. Can the vendor provide evidence of its own testing, validation, and monitoring practices?
16. Has the vendor been subject to any regulatory enforcement, litigation, or public complaints related to its AI systems?

Operations and business continuity

17. What is the service-level agreement for uptime, latency, and support?
18. How much notice will the vendor give before model updates, retraining, or architecture changes?
19. What happens to the insurer’s data if the contract is terminated? Is there a defined exit process?
20. Does the vendor have a business continuity and disaster recovery plan that covers AI operations?

If a vendor cannot answer a question, that is not automatically a disqualifier. But it is a gap that must be documented, remediated, or accepted at the right level of risk oversight.

How to turn twenty answers into a decision

A completed questionnaire does not decide anything by itself. The assessment should end in one of three documented outcomes, and the sign-off level should match the risk tier you assigned at the start.

• Approve. All material questions answered, no open red flags. The record notes who reviewed it and when the first re-assessment is due.
• Approve with conditions. Gaps exist but are tolerable for a defined period. Every condition needs three things in writing: the specific gap, the owner and due date for closing it, and the interim control that covers the exposure until then. A condition without a due date is an approval with extra paperwork.
• Reject or defer. A red flag from the list below stands unresolved, or the vendor declines a required contract clause. For high-risk vendors, an unresolved red flag should not be converted into a “condition”; that is what the escalation path is for.

Who signs depends on the tier. For high-risk vendors, the decision belongs to the senior risk owner (CRO or equivalent), with the outcome reported to the board-level committee that oversees the AI program. Medium risk can sit with the compliance lead. Low risk can stay in procurement, with compliance able to see the log. At an exam the question is who accepted this risk and whether they had the authority to.

Record the outcome in a short, standing format so the decisions accumulate into an auditable trail. The example column below is illustrative, not a real company’s assessment, and it deliberately shows a vendor that could not answer everything:

Contract clauses that matter

The vendor questionnaire is only the first half of the control. The contract is where the control becomes enforceable. At minimum, a high-risk AI vendor contract should include the following:

• Audit rights. The insurer must be able to audit the vendor’s AI systems, data practices, and compliance controls, or to receive audit reports from a qualified third party.
• Documentation on request. The vendor must supply model documentation, validation and testing records, and change logs within a stated number of business days when the insurer asks, including when the request originates with a regulator. Audit rights let you go and look; they do not by themselves oblige the vendor to hand anything over on a deadline, and a deadline is what an exam runs on.
• Regulatory cooperation. The vendor must cooperate with regulatory inquiries and examinations related to the AI system and must notify the insurer of any regulatory contact concerning the system.
• Data use restrictions. The contract should clearly prohibit the vendor from using the insurer’s data to train models for other customers or for general product improvement without explicit consent.
• Model change notification. The vendor must notify the insurer before material changes to the model, training data, or intended use case.
• Indemnification and liability. The vendor should indemnify the insurer for claims arising from the vendor’s AI system, subject to negotiation.
• Exit and data portability. Upon termination, the vendor must return or securely destroy the insurer’s data and provide reasonable transition assistance.

A vendor that refuses to accept these clauses is not necessarily malicious. It may simply be a vendor that has never operated in a regulated insurance environment. Either way, the refusal is a data point for your risk assessment.

After the contract: ongoing monitoring

Due diligence at signing is not enough. AI systems change. Models are retrained. New data sources are added. Vendors are acquired. A vendor risk assessment program must include ongoing monitoring.

At least annually, re-assess each high-risk vendor against the same checklist. Track whether the vendor’s error rates, bias testing results, or data practices have changed. Monitor consumer complaints and adverse outcomes for patterns that might trace back to the vendor’s AI system. And keep an internal record of any material model changes or incidents.

A practical monitoring calendar for high-risk AI vendors should include:

• Quarterly: Review of error rates, complaints, and any known model changes.
• Semi-annually: Review of security or privacy incidents, including any involving the vendor’s other customers.
• Annually: Full re-assessment against the checklist and a contract compliance review.
• Ad hoc: Immediate review after any significant model change, data breach, regulatory action, or litigation involving the vendor.

If the vendor’s AI system supports adverse decisions, such as coverage denials or premium increases, the monitoring should be more frequent and should include human review of a sample of decisions. See our framework on AI in health insurance governance for how this works in a high-stakes line of business.

Red flags

Some vendor behaviors should trigger immediate escalation. The most common are:

• The vendor refuses to provide model documentation or validation records.
• The vendor will not explain how training data was selected or cleaned.
• The contract does not include audit rights or regulatory cooperation.
• The vendor claims its model is a “black box” that cannot be tested for bias.
• The vendor updates the model without notifying the insurer.
• The vendor’s data security practices are vague or undocumented, which is also a third-party oversight gap under the Insurance Data Security Model Law where a state has enacted it.
• The vendor is not familiar with insurance-specific AI regulation.

Each of these is a signal that the vendor may not be ready for a regulated insurance environment. The compliance cost of working with such a vendor often exceeds the purchase price.

Rented algorithm, owned outcome

AI vendor risk assessment arrives dressed as a procurement exercise and behaves like a compliance one. The NAIC Model Bulletin, NY DFS guidance, and the regulation under Colorado’s SB 21-169 differ in force and in reach, but not one of them lets “ask our vendor” be the answer to an examiner’s question.

So start with the contracts that are about to reopen. Pull the AI vendor agreements up for renewal in the next two quarters, mark which of the seven clauses above each one already has, and take the gap list into the renewal conversation. Mid-term, a vendor has no particular reason to reopen anything. Expect to lose a few of the clauses: a carrier renewing with a dominant scoring vendor has far less leverage than a clause list implies, and the honest file in that case records what was asked for, what was refused, and what compensating control went in instead. That file is worth considerably more at an exam than a clean contract that quietly never asked.

One assignment carries the rest of this. Each depth-of-review call above hangs on the tier set before the first question is asked, and on that point the guidance offers proportionality and stops: the scope of controls should align with the degree of potential harm to consumers.² So the three tiers here are ours, and a vendor sorted one tier low never gets the twenty questions at all.

FAQ

Is there a standard AI vendor risk assessment questionnaire? Not from an insurance regulator. The NAIC Model Bulletin says its goal “is not to prescribe specific practices or to prescribe specific documentation requirements.”² The twenty questions above are ours, and the download has them with answer and gap columns for a vendor to fill in.

Does the NAIC Model Bulletin reach third-party AI vendors? Yes, but indirectly. The bulletin binds the insurer and leaves the vendor outside the NAIC’s reach entirely, while expecting the insurer’s AI program to address vendor due diligence, contract terms, and confirmation that the vendor is complying. What reaches the vendor is the insurer’s own obligation for the decisions its system produces.²

Can an insurer rely on a vendor’s SOC 2 report instead of doing its own AI due diligence? No. SOC 2 covers information security controls, not model fairness, accuracy, or insurance-specific regulatory compliance. A SOC 2 report may be a useful input, but it does not answer the due-diligence question the bulletin expects your program to address.

What is the most commonly missed vendor risk? Data use. Most contracts are simply silent on whether your claims and policy data can train models sold to the carrier down the street, and silence tends to resolve in the vendor’s favor. It surfaces late, usually when an output looks more familiar than it should.

Should low-risk AI vendors be exempted from all oversight? No. Low-risk vendors should still undergo basic security and privacy review, including data processing location, access controls, and incident response. The risk classification determines the depth of review, not whether there is review at all.

Footnotes

1. National Association of Insurance Commissioners, “Health Insurance Artificial Intelligence/Machine Learning Survey Results,” May 2025: https://content.naic.org/sites/default/files/inline-files/Health%20Survey%20Report%20-%20FINAL%205.9.25.pdf ↩

2. National Association of Insurance Commissioners, “Model Bulletin on the Use of Artificial Intelligence Systems by Insurers,” December 2023: https://content.naic.org/sites/default/files/inline-files/2023-12-4%20Model%20Bulletin_Adopted_0.pdf ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹

3. New York State Department of Financial Services, “Insurance Circular Letter No. 7 (2024): Use of Artificial Intelligence Systems and External Consumer Data and Information Sources in Insurance Underwriting and Pricing,” July 11, 2024: https://www.dfs.ny.gov/industry-guidance/circular-letters/cl2024-07 ↩ ↩²

4. Colorado Division of Insurance, “Notice of Adoption - Amended Regulation 10-1-1 Governance and Risk Management Framework,” 2025: https://doi.colorado.gov/announcements/notice-of-adoption-amended-regulation-10-1-1-governance-and-risk-management-framework ↩