# Monitor S3 Storage Lens metrics in CloudWatch You can publish S3 Storage Lens metrics to Amazon CloudWatch to create a unified view of your operational health in [CloudWatch dashboards](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Dashboards.html). You can also use CloudWatch features, such as alarms and triggered actions, metric math, and anomaly detection, to monitor and take action on S3 Storage Lens metrics. In addition, CloudWatch API operations enable applications, including third-party providers, to access your S3 Storage Lens metrics. For more information about CloudWatch features, see the [Amazon CloudWatch User Guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html). You can enable the CloudWatch publishing option for new or existing dashboard configurations by using the Amazon S3 console, Amazon S3 REST API, AWS CLI, and AWS SDKs. Dashboards that are upgraded to S3 Storage Lens advanced metrics and recommendations can use the CloudWatch publishing option. For S3 Storage Lens advanced metrics and recommendations pricing, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). No additional CloudWatch metrics publishing charges apply; however, other CloudWatch charges, such as dashboards, alarms, and API calls, do apply. For more information, see [Amazon CloudWatch pricing](https://aws.amazon.com/cloudwatch/pricing/). S3 Storage Lens metrics are published to CloudWatch in the account that owns the S3 Storage Lens configuration. After you enable the CloudWatch publishing option within advanced metrics, you can access account-level and bucket-level metrics by configuration ID, account, bucket (for bucket-level metrics only), Region, and storage class in CloudWatch. Prefix-level metrics are not available in CloudWatch. **Note** S3 Storage Lens metrics are daily metrics and are published to CloudWatch once per day. When you query S3 Storage Lens metrics in CloudWatch, the period for the query must be 1 day (86400 seconds). After your daily S3 Storage Lens metrics appear in your S3 Storage Lens dashboard in the Amazon S3 console, it can take a few hours for these same metrics to appear in CloudWatch. When you enable the CloudWatch publishing option for S3 Storage Lens metrics for the first time, it can take up to 24 hours for your metrics to publish to CloudWatch. After you enable the CloudWatch publishing option, you can use the following CloudWatch features to monitor and analyze your S3 Storage LensStorage Lens data: + [Dashboards](storage-lens-cloudwatch-monitoring-cloudwatch.md#storage-lens-cloudwatch-monitoring-cloudwatch-dashboards) – Use CloudWatch dashboards to create customized S3 Storage Lens dashboards. Share your CloudWatch dashboard with people who don't have direct access to your AWS account, across teams, with stakeholders, and with people external to your organizations. + [Alarms and triggered actions](storage-lens-cloudwatch-monitoring-cloudwatch.md#storage-lens-cloudwatch-monitoring-cloudwatch-alarms) – Configure alarms that watch metrics and take action when a threshold is breached. For example, you can configure an alarm that sends an Amazon SNS notification when the **Incomplete Multipart Upload Bytes** metric exceeds 1 GB for three consecutive days. + [Anomaly detection](storage-lens-cloudwatch-monitoring-cloudwatch.md#storage-lens-cloudwatch-monitoring-cloudwatch-alarms) – Enable anomaly detection to continuously analyze metrics, determine normal baselines, and surface anomalies. You can create an anomaly detection alarm based on the expected value of a metric. For example, you can monitor anomalies for the **Object Lock Enabled Bytes** metric to detect unauthorized removal of Object Lock settings. + [Metric math](storage-lens-cloudwatch-monitoring-cloudwatch.md#storage-lens-cloudwatch-monitoring-cloudwatch-metric-math) – You can also use metric math to query multiple S3 Storage Lens metrics and use math expressions to create new time series based on these metrics. For example, you can create a new metric to get the average object size by dividing `StorageBytes` by `ObjectCount`. For more information about the CloudWatch publishing option for S3 Storage Lens metrics, see the following topics. **Topics** + [S3 Storage Lens metrics and dimensions](storage-lens-cloudwatch-metrics-dimensions.md) + [Enabling CloudWatch publishing for S3 Storage Lens](storage-lens-cloudwatch-enable-publish-option.md) + [Working with S3 Storage Lens metrics in CloudWatch](storage-lens-cloudwatch-monitoring-cloudwatch.md) # S3 Storage Lens metrics and dimensions To send S3 Storage Lens metrics to CloudWatch, you must enable the CloudWatch publishing option within S3 Storage Lens advanced metrics. After advanced metrics are enabled, you can use [CloudWatch dashboards](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Dashboards.html) to monitor S3 Storage Lens metrics alongside other application metrics and create a unified view of your operational health. You can use dimensions to filter your S3 Storage Lens metrics in CloudWatch by organization, account, bucket, storage class, Region, and metrics configuration ID. S3 Storage Lens metrics are published to CloudWatch in the account that owns the S3 Storage Lens configuration. After you enable the CloudWatch publishing option within advanced metrics, you can access account-level and bucket-level metrics by configuration ID, account, bucket (for bucket-level metrics only), Region, and storage class in CloudWatch. Prefix-level metrics are not available in CloudWatch. **Note** S3 Storage Lens metrics are daily metrics and are published to CloudWatch once per day. When you query S3 Storage Lens metrics in CloudWatch, the period for the query must be 1 day (86400 seconds). After your daily S3 Storage Lens metrics appear in your S3 Storage Lens dashboard in the Amazon S3 console, it can take a few hours for these same metrics to appear in CloudWatch. When you enable the CloudWatch publishing option for S3 Storage Lens metrics for the first time, it can take up to 24 hours for your metrics to publish to CloudWatch. For more information about S3 Storage Lens metrics and dimensions in CloudWatch, see the following topics. **Topics** + [Metrics](#storage-lens-cloudwatch-metrics) + [Dimensions](#storage-lens-cloudwatch-dimensions) ## Metrics S3 Storage Lens metrics are available as metrics within CloudWatch. S3 Storage Lens metrics are published to the `AWS/S3/Storage-Lens` namespace. This namespace is only for S3 Storage Lens metrics. Amazon S3 bucket, request, and replication metrics are published to the `AWS/S3` namespace. S3 Storage Lens metrics are published to CloudWatch in the account that owns the S3 Storage Lens configuration. After you enable the CloudWatch publishing option within advanced metrics, you can access account-level and bucket-level metrics by configuration ID, account, bucket (for bucket-level metrics only), Region, and storage class in CloudWatch. Prefix-level metrics are not available in CloudWatch. In S3 Storage Lens, metrics are aggregated and stored only in the designated home Region. S3 Storage Lens metrics are also published to CloudWatch in the home Region that you specify in the S3 Storage Lens configuration. For a complete list of S3 Storage Lens metrics, including a list of those metrics available in CloudWatch, see [Amazon S3 Storage Lens metrics glossary](storage_lens_metrics_glossary.md). **Note** The valid statistic for S3 Storage Lens metrics in CloudWatch is Average. For more information about statistics in CloudWatch, see [ CloudWatch statistics definitions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Statistics-definitions.html) in the *Amazon CloudWatch User Guide*. ### Granularity of S3 Storage Lens metrics in CloudWatch S3 Storage Lens offers metrics at organization, account, bucket, and prefix granularity. S3 Storage Lens publishes organization, account, and bucket-level S3 Storage Lens metrics to CloudWatch. Prefix-level S3 Storage Lens metrics are not available in CloudWatch. For more information about the granularity of S3 Storage Lens metrics available in CloudWatch, see the following list: + **Organization** – Metrics aggregated across the member accounts in your organization. S3 Storage Lens publishes metrics for member accounts to CloudWatch in the management account. + **Organization and account** – Metrics for the member accounts in your organization. + **Organization and bucket** – Metrics for Amazon S3 buckets in the member accounts of your organization. + **Account** (Non-organization level) – Metrics aggregated across the buckets in your account. + **Bucket** (Non-organization level) – Metrics for a specific bucket. In CloudWatch, S3 Storage Lens publishes these metrics to the AWS account that created the S3 Storage Lens configuration. S3 Storage Lens publishes these metrics only for non-organization configurations. ## Dimensions When S3 Storage Lens sends data to CloudWatch, dimensions are attached to each metric. Dimensions are categories that describe the characteristics of metrics. You can use dimensions to filter the results that CloudWatch returns. For example, all S3 Storage Lens metrics in CloudWatch have the `configuration_id` dimension. You can use this dimension to differentiate between metrics associated with a specific S3 Storage Lens configuration. The `organization_id` identifies organization-level metrics. For more information about dimensions in CloudWatch, see [Dimensions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Dimension) in the *CloudWatch User Guide*. Different dimensions are available for S3 Storage Lens metrics depending on the granularity of the metrics. For example, you can use the `organization_id` dimension to filter organization-level metrics by the AWS Organizations ID. However, you can't use this dimension for bucket and account-level metrics. For more information, see [Filtering metrics using dimensions](storage-lens-cloudwatch-monitoring-cloudwatch.md#storage-lens-cloudwatch-monitoring-cloudwatch-dimensions). To see which dimensions are available for your S3 Storage Lens configuration, see the following table. | **Dimension** | **Description** | **Bucket** | **Account** | **Organization** | **Organization and bucket** | **Organization and account** | | --- | --- | --- | --- | --- | --- | --- | | configuration\$1id | The dashboard name for the S3 Storage Lens configuration reported in the metrics | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | | metrics\$1version | The version of the S3 Storage Lens metrics. The metrics version has a fixed value of `1.0`. | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | | organization\$1id | The AWS Organizations ID for the metrics | ![\[No\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-no.png) | ![\[No\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-no.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | | aws\$1account\$1number | The AWS account that's associated with the metrics | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[No\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-no.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | | aws\$1region | The AWS Region for the metrics | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | | bucket\$1name | The name of the S3 bucket that's reported in the metrics | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[No\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-no.png) | ![\[No\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-no.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[No\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-no.png) | | storage\$1class | The storage class for the bucket that's reported in the metrics | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) | | record\$1type | The granularity of the metrics: ORGANIZATION, ACCOUNT, BUCKET | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) BUCKET | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) ACCOUNT | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) BUCKET | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) ACCOUNT | ![\[Yes\]](http://docs.aws.amazon.com/AmazonS3/latest/userguide/images/icon-yes.png) ORGANIZATION | # Enabling CloudWatch publishing for S3 Storage Lens You can publish S3 Storage Lens metrics to Amazon CloudWatch to create a unified view of your operational health in [CloudWatch dashboards](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Dashboards.html). You can also use CloudWatch features, such as alarms and triggered actions, metric math, and anomaly detection, to monitor and take action on S3 Storage Lens metrics. In addition, CloudWatch API operations enable applications, including third-party providers, to access your S3 Storage Lens metrics. For more information about CloudWatch features, see the [Amazon CloudWatch User Guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html). S3 Storage Lens metrics are published to CloudWatch in the account that owns the S3 Storage Lens configuration. After you enable the CloudWatch publishing option within advanced metrics, you can access account-level and bucket-level metrics by configuration ID, account, bucket (for bucket-level metrics only), Region, and storage class in CloudWatch. Prefix-level metrics are not available in CloudWatch. You can enable CloudWatch support for new or existing dashboard configurations by using the S3 console, Amazon S3 REST APIs, AWS CLI, and AWS SDKs. The CloudWatch publishing option is available for dashboards that are upgraded to S3 Storage Lens advanced metrics and recommendations. For S3 Storage Lens advanced metrics and recommendations pricing, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). No additional CloudWatch metrics publishing charges apply; however, other CloudWatch charges, such as dashboards, alarms, and API calls, do apply. To enable the CloudWatch publishing option for S3 Storage Lens metrics, see the following topics. **Note** S3 Storage Lens metrics are daily metrics and are published to CloudWatch once per day. When you query S3 Storage Lens metrics in CloudWatch, the period for the query must be 1 day (86400 seconds). After your daily S3 Storage Lens metrics appear in your S3 Storage Lens dashboard in the Amazon S3 console, it can take a few hours for these same metrics to appear in CloudWatch. When you enable the CloudWatch publishing option for S3 Storage Lens metrics for the first time, it can take up to 24 hours for your metrics to publish to CloudWatch. Currently, S3 Storage Lens metrics cannot be consumed through CloudWatch streams. ## Using the S3 console When you update an S3 Storage Lens dashboard, you can't change the dashboard name or home Region. You also can't change the scope of the default dashboard, which is scoped to your entire account's storage. **To update an S3 Storage Lens dashboard to enable CloudWatch publishing** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **S3 Storage Lens**, **Dashboards**. 1. Choose the dashboard that you want to edit, and then choose **Edit.** 1. Under **Metrics selection**, choose **Advanced metrics and recommendations**. Advanced metrics and recommendations are available for an additional charge. Advanced metrics and recommendations include a 15-month period for data queries, usage metrics aggregated at the prefix level, activity metrics aggregated by bucket, the CloudWatch publishing option, and contextual recommendations that help you optimize storage costs and apply data-protection best practices. For more information, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). 1. Under **Select Advanced metrics and recommendations features**, select **CloudWatch publishing**. **Important** If your configuration enables prefix aggregation for usage metrics, prefix-level metrics will not be published to CloudWatch. Only bucket, account, and organization-level S3 Storage Lens metrics are published to CloudWatch. 1. Choose **Save changes**. **To create a new S3 Storage Lens dashboard that enables CloudWatch support** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **Storage Lens**, **Dashboards**. 1. Choose **Create dashboard**. 1. Under **General**, define the following configuration options: 1. For **Dashboard name**, enter your dashboard name. Dashboard names must be fewer than 65 characters and must not contain special characters or spaces. You can't change the dashboard name after you create your dashboard. 1. Choose the **Home Region ** for your dashboard. Metrics for all Regions included in this dashboard scope are stored centrally in the designated home Region. In CloudWatch, S3 Storage Lens metrics are also available in the home Region. You can't change the home Region after you create your dashboard. 1. (Optional) To add tags, choose **Add tag** and enter the tag **Key** and **Value**. **Note** You can add up to 50 tags to your dashboard configuration. 1. Define the scope for your configuration: 1. If you're creating an organization-level configuration, choose the accounts to include in the configuration: **Include all accounts in your configuration** or **Limit the scope to your signed-in account**. **Note** When you create an organization-level configuration that includes all accounts, you can include or exclude only Regions, not buckets. 1. Choose the Regions and buckets that you want S3 Storage Lens to include in the dashboard configuration by doing the following: + To include all Regions, choose **Include Regions and buckets**. + To include specific Regions, clear **Include all Regions**. Under **Choose Regions to include**, choose the Regions that you want S3 Storage Lens to include in the dashboard. + To include specific buckets, clear **Include all buckets**. Under **Choose buckets to include**, choose the buckets that you want S3 Storage Lens to include in the dashboard. **Note** You can choose up to 50 buckets. 1. For **Metrics selection**, choose **Advanced metrics and recommendations**. For more information about advanced metrics and recommendations pricing, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). 1. Under **Advanced metrics and recommendations features**, select the options that you want to enable: + **Advanced metrics** + **CloudWatch publishing** **Important** If you enable prefix aggregation for your S3 Storage Lens configuration, prefix-level metrics will not be published to CloudWatch. Only bucket, account, and organization-level S3 Storage Lens metrics are published to CloudWatch. + **Prefix aggregation** **Note** For more information about advanced metrics and recommendations features, see [Metrics selection](storage_lens_basics_metrics_recommendations.md#storage_lens_basics_metrics_selection). 1. If you enabled **Advanced metrics**, select the **Advanced metrics categories** that you want to display in your S3 Storage Lens dashboard: + **Activity metrics** + **Detailed status code metrics** + **Advanced cost optimization metrics** + **Advanced data protection metrics** For more information about metrics categories, see [Metrics categories](storage_lens_basics_metrics_recommendations.md#storage_lens_basics_metrics_types). For a complete list of metrics, see [Amazon S3 Storage Lens metrics glossary](storage_lens_metrics_glossary.md). 1. (Optional) Configure your metrics export. For more information about how to configure a metrics export, see step [Using the S3 console](storage_lens_creating_dashboard.md#storage_lens_console_creating). 1. Choose **Create dashboard**. ## Using the AWS CLI The following AWS CLI example enables the CloudWatch publishing option by using a S3 Storage Lens organization-level advanced metrics and recommendations configuration. To use this example, replace the `user input placeholders` with your own information. ``` aws s3control put-storage-lens-configuration --account-id=555555555555 --config-id=your-configuration-id --region=us-east-1 --storage-lens-configuration=file://./config.json config.json { "Id": "SampleS3StorageLensConfiguration", "AwsOrg": { "Arn": "arn:aws:organizations::123456789012:organization/o-abcdefgh" }, "AccountLevel": { "ActivityMetrics": { "IsEnabled":true }, "AdvancedCostOptimizationMetrics": { "IsEnabled":true }, "AdvancedDataProtectionMetrics": { "IsEnabled":true }, "DetailedStatusCodesMetrics": { "IsEnabled":true }, "BucketLevel": { "ActivityMetrics": { "IsEnabled":true }, "AdvancedCostOptimizationMetrics": { "IsEnabled":true }, "DetailedStatusCodesMetrics": { "IsEnabled":true }, "PrefixLevel":{ "StorageMetrics":{ "IsEnabled":true, "SelectionCriteria":{ "MaxDepth":5, "MinStorageBytesPercentage":1.25, "Delimiter":"/" } } } } }, "Exclude": { "Regions": [ "eu-west-1" ], "Buckets": [ "arn:aws:s3:::amzn-s3-demo-source-bucket " ] }, "IsEnabled": true, "DataExport": { "S3BucketDestination": { "OutputSchemaVersion": "V_1", "Format": "CSV", "AccountId": "111122223333", "Arn": "arn:aws:s3:::amzn-s3-demo-destination-bucket", "Prefix": "prefix-for-your-export-destination", "Encryption": { "SSES3": {} } }, "CloudWatchMetrics": { "IsEnabled": true } } } ``` ## Using the AWS SDK for Java ``` package aws.example.s3control; import com.amazonaws.AmazonServiceException; import com.amazonaws.SdkClientException; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.services.s3control.AWSS3Control; import com.amazonaws.services.s3control.AWSS3ControlClient; import com.amazonaws.services.s3control.model.AccountLevel; import com.amazonaws.services.s3control.model.ActivityMetrics; import com.amazonaws.services.s3control.model.BucketLevel; import com.amazonaws.services.s3control.model.CloudWatchMetrics; import com.amazonaws.services.s3control.model.Format; import com.amazonaws.services.s3control.model.Include; import com.amazonaws.services.s3control.model.OutputSchemaVersion; import com.amazonaws.services.s3control.model.PrefixLevel; import com.amazonaws.services.s3control.model.PrefixLevelStorageMetrics; import com.amazonaws.services.s3control.model.PutStorageLensConfigurationRequest; import com.amazonaws.services.s3control.model.S3BucketDestination; import com.amazonaws.services.s3control.model.SSES3; import com.amazonaws.services.s3control.model.SelectionCriteria; import com.amazonaws.services.s3control.model.StorageLensAwsOrg; import com.amazonaws.services.s3control.model.StorageLensConfiguration; import com.amazonaws.services.s3control.model.StorageLensDataExport; import com.amazonaws.services.s3control.model.StorageLensDataExportEncryption; import com.amazonaws.services.s3control.model.StorageLensTag; import java.util.Arrays; import java.util.List; import static com.amazonaws.regions.Regions.US_WEST_2; public class CreateAndUpdateDashboard { public static void main(String[] args) { String configurationId = "ConfigurationId"; String sourceAccountId = "Source Account ID"; String exportAccountId = "Destination Account ID"; String exportBucketArn = "arn:aws:s3:::amzn-s3-demo-destination-bucket"; // The destination bucket for your metrics export must be in the same Region as your S3 Storage Lens configuration. String awsOrgARN = "arn:aws:organizations::123456789012:organization/o-abcdefgh"; Format exportFormat = Format.CSV; try { SelectionCriteria selectionCriteria = new SelectionCriteria() .withDelimiter("/") .withMaxDepth(5) .withMinStorageBytesPercentage(10.0); PrefixLevelStorageMetrics prefixStorageMetrics = new PrefixLevelStorageMetrics() .withIsEnabled(true) .withSelectionCriteria(selectionCriteria); BucketLevel bucketLevel = new BucketLevel() .withActivityMetrics(new ActivityMetrics().withIsEnabled(true)) .withAdvancedCostOptimizationMetrics(new AdvancedCostOptimizationMetrics().withIsEnabled(true)) .withAdvancedDataProtectionMetrics(new AdvancedDataProtectionMetrics().withIsEnabled(true)) .withDetailedStatusCodesMetrics(new DetailedStatusCodesMetrics().withIsEnabled(true)) .withPrefixLevel(new PrefixLevel().withStorageMetrics(prefixStorageMetrics)); AccountLevel accountLevel = new AccountLevel() .withActivityMetrics(new ActivityMetrics().withIsEnabled(true)) .withAdvancedCostOptimizationMetrics(new AdvancedCostOptimizationMetrics().withIsEnabled(true)) .withAdvancedDataProtectionMetrics(new AdvancedDataProtectionMetrics().withIsEnabled(true)) .withDetailedStatusCodesMetrics(new DetailedStatusCodesMetrics().withIsEnabled(true)) .withBucketLevel(bucketLevel); Include include = new Include() .withBuckets(Arrays.asList("arn:aws:s3:::amzn-s3-demo-bucket")) .withRegions(Arrays.asList("us-west-2")); StorageLensDataExportEncryption exportEncryption = new StorageLensDataExportEncryption() .withSSES3(new SSES3()); S3BucketDestination s3BucketDestination = new S3BucketDestination() .withAccountId(exportAccountId) .withArn(exportBucketArn) .withEncryption(exportEncryption) .withFormat(exportFormat) .withOutputSchemaVersion(OutputSchemaVersion.V_1) .withPrefix("Prefix"); CloudWatchMetrics cloudWatchMetrics = new CloudWatchMetrics() .withIsEnabled(true); StorageLensDataExport dataExport = new StorageLensDataExport() .withCloudWatchMetrics(cloudWatchMetrics) .withS3BucketDestination(s3BucketDestination); StorageLensAwsOrg awsOrg = new StorageLensAwsOrg() .withArn(awsOrgARN); StorageLensConfiguration configuration = new StorageLensConfiguration() .withId(configurationId) .withAccountLevel(accountLevel) .withInclude(include) .withDataExport(dataExport) .withAwsOrg(awsOrg) .withIsEnabled(true); List tags = Arrays.asList( new StorageLensTag().withKey("key-1").withValue("value-1"), new StorageLensTag().withKey("key-2").withValue("value-2") ); AWSS3Control s3ControlClient = AWSS3ControlClient.builder() .withCredentials(new ProfileCredentialsProvider()) .withRegion(US_WEST_2) .build(); s3ControlClient.putStorageLensConfiguration(new PutStorageLensConfigurationRequest() .withAccountId(sourceAccountId) .withConfigId(configurationId) .withStorageLensConfiguration(configuration) .withTags(tags) ); } catch (AmazonServiceException e) { // The call was transmitted successfully, but Amazon S3 couldn't process // it and returned an error response. e.printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon S3. e.printStackTrace(); } } } ``` ## Using the REST API To enable the CloudWatch publishing option by using the Amazon S3 REST API, you can use [https://docs.aws.amazon.com/AmazonS3/latest/API/API_control_PutStorageLensConfiguration.html](https://docs.aws.amazon.com/AmazonS3/latest/API/API_control_PutStorageLensConfiguration.html). **Next steps** After you enable the CloudWatch publishing option, you can access your S3 Storage Lens metrics in CloudWatch. You also can leverage CloudWatch features to monitor and analyze your S3 Storage Lens data in CloudWatch. For more information, see the following topics: + [S3 Storage Lens metrics and dimensions](storage-lens-cloudwatch-metrics-dimensions.md) + [Working with S3 Storage Lens metrics in CloudWatch](storage-lens-cloudwatch-monitoring-cloudwatch.md) # Working with S3 Storage Lens metrics in CloudWatch You can publish S3 Storage Lens metrics to Amazon CloudWatch to create a unified view of your operational health in [CloudWatch dashboards](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Dashboards.html). You can also use CloudWatch features, such as alarms and triggered actions, metric math, and anomaly detection, to monitor and take action on S3 Storage Lens metrics. In addition, CloudWatch API operations enable applications, including third-party providers, to access your S3 Storage Lens metrics. For more information about CloudWatch features, see the [Amazon CloudWatch User Guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html). You can enable the CloudWatch publishing option for new or existing dashboard configurations by using the Amazon S3 console, Amazon S3 REST APIs, AWS CLI, and AWS SDKs. The CloudWatch publishing option is available for dashboards that are upgraded to S3 Storage Lens advanced metrics and recommendations. For S3 Storage Lens advanced metrics and recommendations pricing, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). No additional CloudWatch metrics publishing charges apply; however, other CloudWatch charges, such as dashboards, alarms, and API calls, do apply. For more information, see [Amazon CloudWatch pricing](https://aws.amazon.com/cloudwatch/pricing/). S3 Storage Lens metrics are published to CloudWatch in the account that owns the S3 Storage Lens configuration. After you enable the CloudWatch publishing option within advanced metrics, you can access account-level and bucket-level metrics by configuration ID, account, bucket (for bucket-level metrics only), Region, and storage class in CloudWatch. Prefix-level metrics are not available in CloudWatch. **Note** S3 Storage Lens metrics are daily metrics and are published to CloudWatch once per day. When you query S3 Storage Lens metrics in CloudWatch, the period for the query must be 1 day (86400 seconds). After your daily S3 Storage Lens metrics appear in your S3 Storage Lens dashboard in the Amazon S3 console, it can take a few hours for these same metrics to appear in CloudWatch. When you enable the CloudWatch publishing option for S3 Storage Lens metrics for the first time, it can take up to 24 hours for your metrics to publish to CloudWatch. Currently, S3 Storage Lens metrics cannot be consumed through CloudWatch streams. For more information about working with S3 Storage Lens metrics in CloudWatch, see the following topics. **Topics** + [Working with CloudWatch dashboards](#storage-lens-cloudwatch-monitoring-cloudwatch-dashboards) + [Setting alarms, triggering actions, and using anomaly detection](#storage-lens-cloudwatch-monitoring-cloudwatch-alarms) + [Filtering metrics using dimensions](#storage-lens-cloudwatch-monitoring-cloudwatch-dimensions) + [Calculating new metrics with metric math](#storage-lens-cloudwatch-monitoring-cloudwatch-metric-math) + [Using search expressions in graphs](#storage-lens-cloudwatch-monitoring-cloudwatch-search-expressions) ## Working with CloudWatch dashboards You can use CloudWatch dashboards to monitor S3 Storage Lens metrics alongside other application metrics and create a unified view of your operational health. Dashboards are customizable home pages in the CloudWatch console that you can use to monitor your resources in a single view. CloudWatch has broad permissions control that doesn't support limiting access to a specific set of metrics or dimensions. Users in your account or organization who have access to CloudWatch will have access to metrics for all S3 Storage Lens configurations where the CloudWatch support option is enabled. You can't manage permissions for specific dashboards as you can in S3 Storage Lens. For more information about CloudWatch permissions, see [Managing access permissions to your CloudWatch resources](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/iam-access-control-overview-cw.html) in the *Amazon CloudWatch User Guide*. For more information about using CloudWatch dashboards and configuring permissions, see [Using Amazon CloudWatch dashboards](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Dashboards.html) and [Sharing CloudWatch dashboards](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-dashboard-sharing.html) in the *Amazon CloudWatch User Guide*. ## Setting alarms, triggering actions, and using anomaly detection You can configure CloudWatch alarms that watch S3 Storage Lens metrics in CloudWatch and take action when a threshold is breached. For example, you can configure an alarm that sends an Amazon SNS notification when the **Incomplete Multipart Upload Bytes** metric exceeds 1 GB for three consecutive days. You can also enable anomaly detection to continuously analyze your S3 Storage Lens metrics, determine normal baselines, and surface anomalies. You can create an anomaly detection alarm based on a metric's expected value. For example, you can monitor anomalies for the **Object Lock Enabled Bytes** metric to detect unauthorized removal of Object Lock settings. For more information and examples, see [Using Amazon CloudWatch alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html) and [Creating an alarm from a metric on a graph](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create_alarm_metric_graph.html) in the *Amazon CloudWatch User Guide*. ## Filtering metrics using dimensions You can use dimensions to filter S3 Storage Lens metrics in the CloudWatch console. For example, you can filter by `configuration_id`, `aws_account_number`, `aws_region`, `bucket_name`, and more. S3 Storage Lens supports multiple dashboard configurations per account. This means that different configurations can include the same bucket. When these metrics are published to CloudWatch, the bucket will have duplicate metrics within CloudWatch. To view metrics only for a specific S3 Storage Lens configuration in CloudWatch, you can use the `configuration_id` dimension. When you filter by `configuration_id`, you see only the metrics that are associated with the configuration that you identify. For more information about filtering by configuration ID, see [Searching for available metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/finding_metrics_with_cloudwatch.html) in the *Amazon CloudWatch User Guide*. ## Calculating new metrics with metric math You can use metric math to query multiple S3 Storage Lens metrics and use math expressions to create new time series based on these metrics. For example, you can create a new metric for unencrypted objects by subtracting Encrypted Objects from Object Count. You can also create a metric to get the average object size by dividing `StorageBytes` by `ObjectCount` or the number bytes accessed on one day by dividing `BytesDownloaded` by `StorageBytes`. For more information, see [Using metric math](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html) in the *Amazon CloudWatch User Guide*. ## Using search expressions in graphs With S3 Storage Lens metrics, you can create a search expression. For example, you can create a search expression for all metrics that are named **IncompleteMultipartUploadStorageBytes** and add `SUM` to the expression. With this search expression, you can see your total incomplete multipart upload bytes across all dimensions of your storage in a single metric. This example shows the syntax that you would use to create a search expression for all metrics named **IncompleteMultipartUploadStorageBytes**. ``` SUM(SEARCH('{AWS/S3/Storage-Lens,aws_account_number,aws_region,configuration_id,metrics_version,record_type,storage_class} MetricName="IncompleteMultipartUploadStorageBytes"', 'Average',86400)) ``` For more information about this syntax, see [CloudWatch search expression syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html) in the *Amazon CloudWatch User Guide*. To create a CloudWatch graph with a search expression, see [Creating a CloudWatch graph with a search expression](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-search-expression.html)in the *Amazon CloudWatch User Guide*.