# Feature attribution drift for models in production A drift in the distribution of live data for models in production can result in a corresponding drift in the feature attribution values, just as it could cause a drift in bias when monitoring bias metrics. Amazon SageMaker Clarify feature attribution monitoring helps data scientists and ML engineers monitor predictions for feature attribution drift on a regular basis. As the model is monitored, customers can view exportable reports and graphs detailing feature attributions in SageMaker Studio and configure alerts in Amazon CloudWatch to receive notifications if it is detected that the attribution values drift beyond a certain threshold. To illustrate this with a specific situation, consider a hypothetical scenario for college admissions. Assume that we observe the following (aggregated) feature attribution values in the training data and in the live data: College Admission Hypothetical Scenario | Feature | Attribution in training data | Attribution in live data | | --- | --- | --- | | SAT score | 0.70 | 0.10 | | GPA | 0.50 | 0.20 | | Class rank | 0.05 | 0.70 | The change from training data to live data appears significant. The feature ranking has completely reversed. Similar to the bias drift, the feature attribution drifts might be caused by a change in the live data distribution and warrant a closer look into the model behavior on the live data. Again, the first step in these scenarios is to raise an alarm that a drift has happened. We can detect the drift by comparing how the ranking of the individual features changed from training data to live data. In addition to being sensitive to changes in ranking order, we also want to be sensitive to the raw attribution score of the features. For instance, given two features that fall in the ranking by the same number of positions going from training to live data, we want to be more sensitive to the feature that had a higher attribution score in the training data. With these properties in mind, we use the Normalized Discounted Cumulative Gain (NDCG) score for comparing the feature attributions rankings of training and live data. Specifically, assume we have the following: + *F=[f1​,…,fm​] * is the list of features sorted with respect to their attribution scores in the training data where *m* is the total number of features. For instance, in our case, *F*=[SAT Score, GPA, Class Rank]. + *a(f)* is a function that returns the feature attribution score on the training data given a feature *f*. For example, *a*(SAT Score) = 0.70. + *F′=[f′​1​, …, f′​m​] *is the list of features sorted with respect to their attribution scores in the live data. For example, *F*′= [Class Rank, GPA, SAT Score]. Then, we can compute the NDCG as:         NDCG=DCG/iDCG​ with + DCG = ∑1m*a*(*f'i*)/log2​(*i*\$11) + iDCG = ∑1m*a*(*fi*)/log2​(*i*\$11) The quantity DCG measures whether features with high attribution in the training data are also ranked higher in the feature attribution computed on the live data. The quantity iDCG measures the *ideal score* and it's just a normalizing factor to ensure that the final quantity resides in the range [0, 1], with 1 being the best possible value. A NDCG value of 1 means that the feature attribution ranking in the live data is the same as the one in the training data. In this particular example, because the ranking changed by quite a bit, the NDCG value is 0.69. In SageMaker Clarify, if the NDCG value is below 0.90, we automatically raise an alert. ## Model Monitor Example Notebook SageMaker Clarify provides the following example notebook that shows how to capture inference data for a real-time endpoint, create a baseline to monitor evolving bias against, and inspect the results: + [Monitoring bias drift and feature attribution drift Amazon SageMaker Clarify](https://sagemaker-examples.readthedocs.io/en/latest/sagemaker_model_monitor/fairness_and_explainability/SageMaker-Model-Monitor-Fairness-and-Explainability.html) – Use Amazon SageMaker Model Monitor to monitor bias drift and feature attribution drift over time. This notebook has been verified to run in SageMaker Studio only. If you need instructions on how to open a notebook in SageMaker Studio, see [Create or Open an Amazon SageMaker Studio Classic Notebook](notebooks-create-open.md). If you're prompted to choose a kernel, choose **Python 3 (Data Science)**. The following topics contain the highlights from the last two steps, and they contain code examples from the example notebook. **Topics** + [Model Monitor Example Notebook](#clarify-model-monitor-sample-notebooks-feature-drift) + [Create a SHAP Baseline for Models in Production](clarify-model-monitor-shap-baseline.md) + [Model Feature Attribution Drift Violations](clarify-model-monitor-model-attribution-drift-violations.md) + [Parameters to Monitor Attribution Drift](clarify-config-json-monitor-model-explainability-parameters.md) + [Schedule Feature Attribute Drift Monitoring Jobs](clarify-model-monitor-feature-attribute-drift-schedule.md) + [Inspect Reports for Feature Attribute Drift in Production Models](clarify-feature-attribute-drift-report.md) + [CloudWatch Metrics for Feature Drift Analysis](clarify-feature-attribute-drift-cw.md) # Create a SHAP Baseline for Models in Production Explanations are typically contrastive, that is, they account for deviations from a baseline. For information on explainability baselines, see [SHAP Baselines for Explainability](clarify-feature-attribute-shap-baselines.md). In addition to providing explanations for per-instance inferences, SageMaker Clarify also supports global explanation for ML models that helps you understand the behavior of a model as a whole in terms of its features. SageMaker Clarify generates a global explanation of an ML model by aggregating the Shapley values over multiple instances. SageMaker Clarify supports the following different ways of aggregation, which you can use to define baselines: + `mean_abs` – Mean of absolute SHAP values for all instances. + `median` – Median of SHAP values for all instances. + `mean_sq` – Mean of squared SHAP values for all instances. After you have configured your application to capture real-time or batch transform inference data, the first task to monitor for drift in feature attribution is to create a baseline to compare against. This involves configuring the data inputs, which groups are sensitive, how the predictions are captured, and the model and its posttraining bias metrics. Then you need to start the baselining job. Model explainability monitor can explain the predictions of a deployed model that's producing inferences and detect feature attribution drift on a regular basis. ``` model_explainability_monitor = ModelExplainabilityMonitor( role=role, sagemaker_session=sagemaker_session, max_runtime_in_seconds=1800, ) ``` In this example, the explainability baselining job shares the test dataset with the bias baselining job, so it uses the same `DataConfig`, and the only difference is the job output URI. ``` model_explainability_baselining_job_result_uri = f"{baseline_results_uri}/model_explainability" model_explainability_data_config = DataConfig( s3_data_input_path=validation_dataset, s3_output_path=model_explainability_baselining_job_result_uri, label=label_header, headers=all_headers, dataset_type=dataset_type, ) ``` Currently the SageMaker Clarify explainer offers a scalable and efficient implementation of SHAP, so the explainability config is SHAPConfig, including the following: + `baseline` – A list of rows (at least one) or S3 object URI to be used as the baseline dataset in the Kernel SHAP algorithm. The format should be the same as the dataset format. Each row should contain only the feature columns/values and omit the label column/values. + `num_samples` – Number of samples to be used in the Kernel SHAP algorithm. This number determines the size of the generated synthetic dataset to compute the SHAP values. + agg\$1method – Aggregation method for global SHAP values. Following are valid values: + `mean_abs` – Mean of absolute SHAP values for all instances. + `median` – Median of SHAP values for all instances. + `mean_sq` – Mean of squared SHAP values for all instances. + `use_logit` – Indicator of whether the logit function is to be applied to the model predictions. Default is `False`. If `use_logit` is `True`, the SHAP values will have log-odds units. + `save_local_shap_values` (bool) – Indicator of whether to save the local SHAP values in the output location. Default is `False`. ``` # Here use the mean value of test dataset as SHAP baseline test_dataframe = pd.read_csv(test_dataset, header=None) shap_baseline = [list(test_dataframe.mean())] shap_config = SHAPConfig( baseline=shap_baseline, num_samples=100, agg_method="mean_abs", save_local_shap_values=False, ) ``` Start a baselining job. The same `model_config` is required because the explainability baselining job needs to create a shadow endpoint to get predictions for the generated synthetic dataset. ``` model_explainability_monitor.suggest_baseline( data_config=model_explainability_data_config, model_config=model_config, explainability_config=shap_config, ) print(f"ModelExplainabilityMonitor baselining job: {model_explainability_monitor.latest_baselining_job_name}") ``` # Model Feature Attribution Drift Violations Feature attribution drift jobs evaluate the baseline constraints provided by the [baseline configuration](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModelExplainabilityJobDefinition.html#sagemaker-CreateModelExplainabilityJobDefinition-request-ModelExplainabilityBaselineConfig) against the analysis results of current `MonitoringExecution`. If violations are detected, the job lists them to the *constraint\$1violations.json* file in the execution output location, and marks the execution status as [Interpret results](model-monitor-interpreting-results.md). Here is the schema of the feature attribution drift violations file. + `label` – The name of the label, job analysis configuration `label_headers` or a placeholder such as `"label0"`. + `metric_name` – The name of the explainability analysis method. Currently only `shap` is supported. + `constraint_check_type` – The type of violation monitored. Currently only `feature_attribution_drift_check` is supported. + `description` – A descriptive message to explain the violation. ``` { "version": "1.0", "violations": [{ "label": "string", "metric_name": "string", "constraint_check_type": "string", "description": "string" }] } ``` For each label in the `explanations` section, the monitoring jobs calculate the [nDCG score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ndcg_score.html) of its global SHAP values in the baseline constraints file and in the job analysis results file (*analysis.json*). If the score is less than 0.9, then a violation is logged. The combined global SHAP value is evaluated, so there are no `“feature”` fields in the violation entry. The following output provides an example of several logged violations. ``` { "version": "1.0", "violations": [{ "label": "label0", "metric_name": "shap", "constraint_check_type": "feature_attribution_drift_check", "description": "Feature attribution drift 0.7639720923277322 exceeds threshold 0.9" }, { "label": "label1", "metric_name": "shap", "constraint_check_type": "feature_attribution_drift_check", "description": "Feature attribution drift 0.7323763972092327 exceeds threshold 0.9" }] } ``` # Parameters to Monitor Attribution Drift Amazon SageMaker Clarify explainability monitor reuses a subset of the parameters used in the analysis configuration of [Analysis Configuration Files](clarify-processing-job-configure-analysis.md). The following parameters must be provided in a JSON file and the path must be provided in the `ConfigUri` parameter of [https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ModelExplainabilityAppSpecification](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ModelExplainabilityAppSpecification). + `"version"` – (Optional) Schema version of the configuration file. If not provided, the latest supported version is used. + `"headers"` – (Optional) A list of feature names in the dataset. Explainability analysis does not require labels. + `"methods"` – A list of methods and their parameters for the analyses and reports. If any section is omitted, then it is not computed. + `"shap"` – (Optional) Section on SHAP value computation. + `"baseline"` – (Optional) A list of rows (at least one), or an Amazon Simple Storage Service Amazon S3 object URI. To be used as the baseline dataset (also known as a background dataset) in the Kernel SHAP algorithm. The format should be the same as the dataset format. Each row should contain only the feature columns (or values). Before you send each row to the model, omit any column that must be excluded. + `"num_samples"` – Number of samples to be used in the Kernel SHAP algorithm. This number determines the size of the generated synthetic dataset to compute the SHAP values. If not provided, then a SageMaker Clarify job chooses the value based on a count of features. + `"agg_method"` – Aggregation method for global SHAP values. Valid values are as follows: + `"mean_abs"` – Mean of absolute SHAP values for all instances. + `"median"` – Median of SHAP values for all instances. + `"mean_sq"` – Mean of squared SHAP values for all instances. + `"use_logit"` – (Optional) Boolean value to indicate if the logit function is to be applied to the model predictions. If `"use_logit"` is `true`, then the SHAP values have log-odds units. The default value is `false`. + `"save_local_shap_values"` – (Optional) Boolean value to indicate if local SHAP values are to be saved in the output location. Use `true` to save them. Use `false` to not save them. The default is `false`. + `"predictor"` – (Optional for real-time endpoint, required for batch transform) Section on model parameters, required if `"shap"` and `"post_training_bias"` sections are present. + `"model_name"` – Model name created by `CreateModel` API, with container mode as `SingleModel`. + `"instance_type"` – Instance type for the shadow endpoint. + `"initial_instance_count"` – Instance count for the shadow endpoint. + `"content_type"` – (Optional) The model input format to be used for getting inferences with the shadow endpoint. Valid values are `"text/csv"` for CSV, `"application/jsonlines"` for JSON Lines, `application/x-parquet` for Apache Parquet, and `application/x-image` to enable Computer Vision explainability. The default value is the same as the `dataset_type` format. + `"accept_type"` – (Optional) The model *output* format to be used for getting inferences with the shadow endpoint. Valid values are `"text/csv"` for CSV, `"application/jsonlines"` for JSON Lines. If omitted, SageMaker Clarify uses the response data type of the captured data. + `"content_template"` – (Optional) A template string used to construct the model input from dataset instances. It is only used when `"content_type"` is `"application/jsonlines"`. The template should have only one placeholder, `$features`, which is replaced by the features list at runtime. For example, given `"content_template":"{\"myfeatures\":$features}"`, if an instance (no label) is `1,2,3`, then model input becomes JSON Lines `'{"myfeatures":[1,2,3]}'`. + `"label_headers"` – (Optional) A list of values that the `"label"` takes in the dataset. Associates the scores returned by the model endpoint or batch transform job with their corresponding label values. If it is provided, then the analysis report uses the headers instead of placeholders like `“label0”`. The other parameters should be provided in `EndpointInput` (for real-time endpoints) or `BatchTransformInput` (for batch transform jobs) of the [https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ModelExplainabilityJobInput](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ModelExplainabilityJobInput) API. + `FeaturesAttribute` – This parameter is required if endpoint or batch job input data format is `"application/jsonlines"`. It is the JMESPath used to locate the feature columns if the dataset format is JSON Lines. + `ProbabilityAttribute` – Index or JMESPath location in the model output for probabilities. If the model output is JSON Lines with a list of labels and probabilities, for example, then the label that corresponds to the maximum probability is selected for bias computations. ## Example JSON Configuration Files for CSV and JSON Lines Datasets Here are examples of the JSON files used to configure CSV and JSON Lines datasets to monitor them for feature attribution drift. **Topics** + [CSV Datasets](#clarify-config-json-monitor-model-explainability-parameters-example-csv) + [JSON Lines Datasets](#clarify-config-json-monitor-model-explainability-parameters-example-jsonlines) ### CSV Datasets Consider a dataset that has three numerical feature columns, as in the following example. ``` 0.5814568701544718, 0.6651538910132964, 0.3138080342665499 0.6711642728531724, 0.7466687034026017, 0.1215477472819713 0.0453256543003371, 0.6377430803264152, 0.3558625219713576 0.4785191813363956, 0.0265841045263860, 0.0376935084990697 ``` Assume that the model output has two columns, where the first one is the predicted label and the second one is the probability, as in the following example. ``` 1, 0.5385257417814224 ``` The following example JSON configuration file shows how this CSV dataset can be configured. ``` { "headers": [ "feature_1", "feature_2", "feature_3" ], "methods": { "shap": { "baseline": [ [0.4441164946610942, 0.5190374448171748, 0.20722795300473712] ], "num_samples": 100, "agg_method": "mean_abs" } }, "predictor": { "model_name": "my_model", "instance_type": "ml.m5.xlarge", "initial_instance_count": 1 } } ``` The predicted label is selected by the `"ProbabilityAttribute"` parameter. Zero-based numbering is used, so 1 indicates the second column of the model output. ``` "EndpointInput": { ... "ProbabilityAttribute": 1 ... } ``` ### JSON Lines Datasets Consider a dataset that has four feature columns and one label column, where the first feature and the label are binary, as in the following example. ``` {"features":[0, 0.5814568701544718, 0.6651538910132964, 0.3138080342665499], "label":0} {"features":[1, 0.6711642728531724, 0.7466687034026017, 0.1215477472819713], "label":1} {"features":[0, 0.0453256543003371, 0.6377430803264152, 0.3558625219713576], "label":1} {"features":[1, 0.4785191813363956, 0.0265841045263860, 0.0376935084990697], "label":1} ``` The model input is the same as the dataset format, and the model output are JSON Lines, as in the following example. ``` {"predicted_label":1, "probability":0.5385257417814224} ``` In the following example, the JSON configuration file shows how this JSON Lines dataset can be configured. ``` { "headers": [ "feature_1", "feature_2", "feature_3" ], "methods": { "shap": { "baseline": [ {"features":[0.4441164946610942, 0.5190374448171748, 0.20722795300473712]} ], "num_samples": 100, "agg_method": "mean_abs" } }, "predictor": { "model_name": "my_model", "instance_type": "ml.m5.xlarge", "initial_instance_count": 1, "content_template":"{\"features\":$features}" } } ``` Then the `"features"` parameter value in `EndpointInput` (for real-time endpoints) or `BatchTransformInput` (for batch transform jobs) is used to locate the features in the dataset, and the `"probability"` parameter value selects the probability value from model output. ``` "EndpointInput": { ... "FeaturesAttribute": "features", "ProbabilityAttribute": "probability", ... } ``` # Schedule Feature Attribute Drift Monitoring Jobs After you create your SHAP baseline, you can call the `create_monitoring_schedule()` method of your `ModelExplainabilityMonitor` class instance to schedule an hourly model explainability monitor. The following sections show you how to create a model explainability monitor for a model deployed to a real-time endpoint as well as for a batch transform job. **Important** You can specify either a batch transform input or an endpoint input, but not both, when you create your monitoring schedule. If a baselining job has been submitted, the monitor automatically picks up analysis configuration from the baselining job. However, if you skip the baselining step or the capture dataset has a different nature from the training dataset, you have to provide the analysis configuration. `ModelConfig` is required by `ExplainabilityAnalysisConfig` for the same reason that it's required for the baselining job. Note that only features are required for computing feature attribution, so you should exclude Ground Truth labeling. ## Feature attribution drift monitoring for models deployed to real-time endpoint To schedule a model explainability monitor for a real-time endpoint, pass your `EndpointInput` instance to the `endpoint_input` argument of your `ModelExplainabilityMonitor` instance, as shown in the following code sample: ``` from sagemaker.model_monitor import CronExpressionGenerator model_exp_model_monitor = ModelExplainabilityMonitor( role=sagemaker.get_execution_role(), ... ) schedule = model_exp_model_monitor.create_monitoring_schedule( monitor_schedule_name=schedule_name, post_analytics_processor_script=s3_code_postprocessor_uri, output_s3_uri=s3_report_path, statistics=model_exp_model_monitor.baseline_statistics(), constraints=model_exp_model_monitor.suggested_constraints(), schedule_cron_expression=CronExpressionGenerator.hourly(), enable_cloudwatch_metrics=True, endpoint_input=EndpointInput( endpoint_name=endpoint_name, destination="/opt/ml/processing/input/endpoint", ) ) ``` ## Feature attribution drift monitoring for batch transform jobs To schedule a model explainability monitor for a batch transform job, pass your `BatchTransformInput` instance to the `batch_transform_input` argument of your `ModelExplainabilityMonitor` instance, as shown in the following code sample: ``` from sagemaker.model_monitor import CronExpressionGenerator model_exp_model_monitor = ModelExplainabilityMonitor( role=sagemaker.get_execution_role(), ... ) schedule = model_exp_model_monitor.create_monitoring_schedule( monitor_schedule_name=schedule_name, post_analytics_processor_script=s3_code_postprocessor_uri, output_s3_uri=s3_report_path, statistics=model_exp_model_monitor.baseline_statistics(), constraints=model_exp_model_monitor.suggested_constraints(), schedule_cron_expression=CronExpressionGenerator.hourly(), enable_cloudwatch_metrics=True, batch_transform_input=BatchTransformInput( destination="opt/ml/processing/data", model_name="batch-fraud-detection-model", input_manifests_s3_uri="s3://amzn-s3-demo-bucket/batch-fraud-detection/on-schedule-monitoring/in/", excludeFeatures="0", ) ) ``` # Inspect Reports for Feature Attribute Drift in Production Models After the schedule that you set up is started by default, you need to wait for the its first execution to start, and then stop the schedule to avoid incurring charges. To inspect the reports, use the following code: ``` schedule_desc = model_explainability_monitor.describe_schedule() execution_summary = schedule_desc.get("LastMonitoringExecutionSummary") if execution_summary and execution_summary["MonitoringExecutionStatus"] in ["Completed", "CompletedWithViolations"]: last_model_explainability_monitor_execution = model_explainability_monitor.list_executions()[-1] last_model_explainability_monitor_execution_report_uri = last_model_explainability_monitor_execution.output.destination print(f'Report URI: {last_model_explainability_monitor_execution_report_uri}') last_model_explainability_monitor_execution_report_files = sorted(S3Downloader.list(last_model_explainability_monitor_execution_report_uri)) print("Found Report Files:") print("\n ".join(last_model_explainability_monitor_execution_report_files)) else: last_model_explainability_monitor_execution = None print("====STOP==== \n No completed executions to inspect further. Please wait till an execution completes or investigate previously reported failures.") ``` If there are any violations compared to the baseline, they are listed here: ``` if last_model_explainability_monitor_execution: model_explainability_violations = last_model_explainability_monitor_execution.constraint_violations() if model_explainability_violations: print(model_explainability_violations.body_dict) ``` If your model is deployed to a real-time endpoint, you can see visualizations in SageMaker Studio of the analysis results and CloudWatch metrics by choosing the **Endpoints** tab, and then double-clicking the endpoint. # CloudWatch Metrics for Feature Drift Analysis This guide shows CloudWatch metrics and their properties that you can use for feature attribute drift analysis in SageMaker Clarify. Feature attribute drift monitoring jobs compute and publish two types of metrics: + The global SHAP value of each feature. **Note** The name of this metric appends the feature name provided by the job analysis configuration to `feature_`. For example, `feature_X` is the global SHAP value for feature `X`. + The `ExpectedValue` of the metric. These metrics are published to the following CloudWatch namespace: + For real-time endpoints: `aws/sagemaker/Endpoints/explainability-metrics` + For batch transform jobs: `aws/sagemaker/ModelMonitoring/explainability-metrics` Each metric has the following properties: + `Endpoint`: The name of the monitored endpoint, if applicable. + `MonitoringSchedule`: The name of the schedule for the monitoring job. + `ExplainabilityMethod`: The method used to compute Shapley values. Choose `KernelShap`. + `Label`: The name provided by job analysis configuration `label_headers`, or a placeholder like `label0`. + `ValueType`: The type of the value returned by the metric. Choose either `GlobalShapValues` or `ExpectedValue`. To stop the monitoring jobs from publishing metrics, set `publish_cloudwatch_metrics` to `Disabled` in the `Environment` map of [model explainability job](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModelExplainabilityJobDefinition.html) definition.