Drift Monitoring - Teradata Vantage

ClearScape Analytics™ ModelOps User Guide

Deployment
VantageCloud
VantageCore
Edition
Enterprise
IntelliFlex
VMware
Product
Teradata Vantage
Release Number
7.1
Published
December 2024
ft:locale
en-US
ft:lastEdition
2024-12-13
dita:mapPath
zdn1704469623418.ditamap
dita:ditavalPath
azq1671041405318.ditaval
dita:id
rgn1654191066978
lifecycle
latest
Product Category
ClearScape

Model drift represents the concept of a model degrading in performance over time due to changes in the underlying dataset or concepts. Monitoring model drift involves the ability to surface this performance information to the end-user and configure alerts so that they can take actions (pre-emptive preferably) to reduce the business impact of this drift. For more details, see Monitoring Alerts.

You can monitor three types of drift using ModelOps: feature, prediction, and performance.

Feature Drift

Also referred to as data drift, feature drift is based on understanding and monitoring changes in the dataset statistics the model was trained on versus the dataset statistics the model is currently predicting. You can monitor this data in an offline process or in real time, as data is fed into the model, to analyze and capture dataset statistics when it has evolved past a certain “divergence” threshold or if it has changed completely. Dataset statistics include only distribution histogram and frequencies to focus column data and importance.

See Feature Drift.

Prediction Drift

Prediction drift uses feature importance to monitor dataset statistics of the model output to look for deviations. Prior to training a model, features and target must be identified. In the case of an imported model that was trained externally, the data for that model must be identified.

After training a model, ModelOps computes and shows the distribution and statistics for each of the model features, and provides a visual overlay of training versus evaluation/scoring statistics.

See Prediction Drift.

Performance Drift

Performance drift involves using a metric to compare predicted and actual data results. If actual data is not available, you can use drift in prediction and feature distributions to identify changes in the model.

See Performance Drift.