Model Drift
The degradation of a model's performance over time as real-world data or behavior changes away from the data it was trained on.
Model drift is the gradual decline in a model’s accuracy or usefulness as the real world changes away from the data it was originally trained on. Drift can happen because consumer behavior shifts, economic conditions change, new regulations alter inputs, or the model is applied to populations it was not trained on.
The NAIC Model Bulletin and state AI rules require ongoing monitoring for drift, bias, and accuracy degradation. A model that tested clean at launch may produce unfair or inaccurate outcomes a year later. Without revalidation, the carrier cannot show the model remains fit for use.
Monitoring programs should include scheduled revalidation, triggers for review when error rates or outcome distributions shift, and vendor change notifications. See our AI vendor risk assessment checklist for monitoring questions to ask before renewing a high-risk AI vendor contract.