# Acting on alarm changes
CloudWatch can notify users on two types of alarm changes: when an alarm changes state, and when the configuration of an alarm gets updated.
When an alarm evaluates, it might change from one state to another, such as ALARM or OK. For Metrics Insights alarms that monitor multiple time series, each time series (contributor) can only be in ALARM or OK state, never in INSUFFICIENT\$1DATA state. This is because a time series only exists when data is present.
Additionally, CloudWatch sends events to Amazon EventBridge whenever alarms change state, and when alarms are created, deleted, or updated. You can write EventBridge rules to take actions or be notified when EventBridge receives these events.
For more information about alarm actions, see [Alarm actions](alarm-actions.md).
**Topics**
+ [Notifying users on alarm changes](Notify_Users_Alarm_Changes.md)
+ [Invoke a Lambda function from an alarm](alarms-and-actions-Lambda.md)
+ [Start a CloudWatch investigations from an alarm](Start-Investigation-Alarm.md)
+ [Stop, terminate, reboot, or recover an EC2 instance](UsingAlarmActions.md)
+ [Alarm events and EventBridge](cloudwatch-and-eventbridge.md)
# Notifying users on alarm changes
This section explains how you can use AWS User Notifications or Amazon Simple Notification Service to have users be notified of alarm changes.
## Setting up AWS User Notifications
You can use [AWS User Notifications](https://docs.aws.amazon.com/notifications/latest/userguide/what-is-service.html) to set up delivery channels to get notified about CloudWatch alarm state change and configuration change events. You receive a notification when an event matches a rule that you specify. You can receive notifications for events through multiple channels, including email, [AWS Chatbot](https://docs.aws.amazon.com/chatbot/latest/adminguide/what-is.html) chat notifications, or [AWS Console Mobile Application push notifications](https://docs.aws.amazon.com/consolemobileapp/latest/userguide/managing-notifications.html). You can also see notifications in the at [Console Notifications Center](https://console.aws.amazon.com/notifications). User Notifications supports aggregation, which can reduce the number of notifications you receive during specific events.
The notification configurations you create with AWS User Notifications do not count towards the limit on the number of actions you can configure per target alarm state. As AWS User Notifications matches the events emitted to Amazon EventBridge, it sends notifications for all the alarms in your account and selected Regions, unless you specify an advanced filter to allowlist or denylist specific alarms or patterns.
The following example of an advanced filter matches an alarm state change from OK to ALARM on the alarm named `ServerCpuTooHigh`.
```
{
"detail": {
"alarmName": ["ServerCpuTooHigh"],
"previousState": { "value": ["OK"] },
"state": { "value": ["ALARM"] }
}
}
```
You can use any of the properties published by an alarm in EventBridge events to create a filter. For more information, see [Alarm events and EventBridge](cloudwatch-and-eventbridge.md).
## Setting up Amazon SNS notifications
You can use Amazon Simple Notification Service to send both application-to-application (A2A) messaging and application-to-person (A2P) messaging, including mobile text messaging (SMS) and email messages. For more information, see [ Amazon SNS event destinations](https://docs.aws.amazon.com/sns/latest/dg/sns-event-destinations.html).
For every state that an alarm can take, you can configure the alarm to send a message to an SNS topic. Every Amazon SNS topic you configure for a state on a given alarm counts towards the limit on the number of actions you can configure for that alarm and state. You can send messages to the same Amazon SNS topic from any alarms in your account, and use the same Amazon SNS topic for both application (A2A) and person (A2P) consumers. Because this configuration is done at the alarm level, only the alarms you have configured send messages to the selected Amazon SNS topic.
First, create a topic, then subscribe to it. You can optionally publish a test message to the topic. For an example, see [Setting up an Amazon SNS topic using the AWS Management Console](#set-up-sns-topic-console). Or for more information, see [ Getting started with Amazon SNS](https://docs.aws.amazon.com/sns/latest/dg/sns-getting-started.html).
Alternatively, if you plan to use the AWS Management Console to create your CloudWatch alarm, you can skip this procedure because you can create the topic when you create the alarm.
When you create a CloudWatch alarm, you can add actions for any target state the alarm enters. Add an Amazon SNS notification for the state you want to be notified about, and select the Amazon SNS topic you created in the previous step to send an email notification when the alarm enters the selected state.
**Note**
When you create an Amazon SNS topic, you choose to make it a *standard topic* or a *FIFO topic*. CloudWatch guarantees the publication of all alarm notifications to both types of topics. However, even if you use a FIFO topic, in rare cases CloudWatch sends the notifications to the topic out of order. If you use a FIFO topic, the alarm sets the message group ID of the alarm notifications to be a hash of the ARN of the alarm.
**Topics**
+ [Preventing confused deputy security issues](#SNS_Confused_Deputy)
+ [Setting up an Amazon SNS topic using the AWS Management Console](#set-up-sns-topic-console)
+ [Setting up an SNS topic using the AWS CLI](#set-up-sns-topic-cli)
### Preventing confused deputy security issues
The confused deputy problem is a security issue where an entity that doesn't have permission to perform an action can coerce a more-privileged entity to perform the action. In AWS, cross-service impersonation can result in the confused deputy problem. Cross-service impersonation can occur when one service (the *calling service*) calls another service (the *called service*). The calling service can be manipulated to use its permissions to act on another customer's resources in a way it should not otherwise have permission to access. To prevent this, AWS provides tools that help you protect your data for all services with service principals that have been given access to resources in your account.
We recommend using the [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn), [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount), [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceorgid](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceorgid), and [https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceorgpaths](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceorgpaths) global condition context keys in resource policies to limit the permissions that Amazon SNS gives another service to the resource. Use `aws:SourceArn` to associate only one resource with cross-service access. Use `aws:SourceAccount` to let any resource in that account be associated with the cross-service use. Use `aws:SourceOrgID` to allow any resource from any account within an organization be associated with the cross-service use. Use `aws:SourceOrgPaths` to associate any resource from accounts within an AWS Organizations path with the cross-service use. For more information about using and understanding paths, see [aws:SourceOrgPaths](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceorgpaths) in the IAM User Guide.
The most effective way to protect against the confused deputy problem is to use the `aws:SourceArn` global condition context key with the full ARN of the resource. If you don't know the full ARN of the resource or if you are specifying multiple resources, use the `aws:SourceArn` global context condition key with wildcard characters (`*`) for the unknown portions of the ARN. For example, `arn:aws:servicename:*:123456789012:*`.
If the `aws:SourceArn` value does not contain the account ID, such as an Amazon S3 bucket ARN, you must use both `aws:SourceAccount` and `aws:SourceArn` to limit permissions.
To protect against the confused deputy problem at scale, use the `aws:SourceOrgID` or `aws:SourceOrgPaths` global condition context key with the organization ID or organization path of the resource in your resource-based policies. Policies that include the `aws:SourceOrgID` or `aws:SourceOrgPaths` key will automatically include the correct accounts and you don't have to manually update the policies when you add, remove, or move accounts in your organization.
The value of `aws:SourceArn` must be the ARN of the alarm that is sending notifications.
The following example shows how you can use the `aws:SourceArn` and `aws:SourceAccount` global condition context keys in CloudWatch to prevent the confused deputy problem.
```
{
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": "cloudwatch.amazonaws.com"
},
"Action": "SNS:Publish",
"Resource": "arn:aws:sns:us-east-2:444455556666:MyTopic",
"Condition": {
"ArnLike": {
"aws:SourceArn": "arn:aws:cloudwatch:us-east-2:111122223333:alarm:*"
},
"StringEquals": {
"aws:SourceAccount": "111122223333"
}
}
}]
}
```
If an alarm ARN includes any non-ASCII characters, use only the `aws:SourceAccount` global condition key to limit the permissions.
### Setting up an Amazon SNS topic using the AWS Management Console
First, create a topic, then subscribe to it. You can optionally publish a test message to the topic.
**To create an SNS topic**
1. Open the Amazon SNS console at [https://console.aws.amazon.com/sns/v3/home](https://console.aws.amazon.com/sns/v3/home).
1. On the Amazon SNS dashboard, under **Common actions**, choose **Create Topic**.
1. In the **Create new topic** dialog box, for **Topic name**, enter a name for the topic (for example, **my-topic**).
1. Choose **Create topic**.
1. Copy the **Topic ARN** for the next task (for example, arn:aws:sns:us-east-1:111122223333:my-topic).
**To subscribe to an SNS topic**
1. Open the Amazon SNS console at [https://console.aws.amazon.com/sns/v3/home](https://console.aws.amazon.com/sns/v3/home).
1. In the navigation pane, choose **Subscriptions**, **Create subscription**.
1. In the **Create subscription** dialog box, for **Topic ARN**, paste the topic ARN that you created in the previous task.
1. For **Protocol**, choose **Email**.
1. For **Endpoint**, enter an email address that you can use to receive the notification, and then choose **Create subscription**.
1. From your email application, open the message from AWS Notifications and confirm your subscription.
Your web browser displays a confirmation response from Amazon SNS.
**To publish a test message to an SNS topic**
1. Open the Amazon SNS console at [https://console.aws.amazon.com/sns/v3/home](https://console.aws.amazon.com/sns/v3/home).
1. In the navigation pane, choose **Topics**.
1. On the **Topics** page, select a topic and choose **Publish to topic**.
1. In the **Publish a message** page, for **Subject**, enter a subject line for your message, and for **Message**, enter a brief message.
1. Choose **Publish Message**.
1. Check your email to confirm that you received the message.
### Setting up an SNS topic using the AWS CLI
First, you create an SNS topic, and then you publish a message directly to the topic to test that you have properly configured it.
**To set up an SNS topic**
1. Create the topic using the [create-topic](https://docs.aws.amazon.com/cli/latest/reference/sns/create-topic.html) command as follows.
```
1. aws sns create-topic --name my-topic
```
Amazon SNS returns a topic ARN with the following format:
```
1. {
2. "TopicArn": "arn:aws:sns:us-east-1:111122223333:my-topic"
3. }
```
1. Subscribe your email address to the topic using the [subscribe](https://docs.aws.amazon.com/cli/latest/reference/sns/subscribe.html) command. If the subscription request succeeds, you receive a confirmation email message.
```
1. aws sns subscribe --topic-arn arn:aws:sns:us-east-1:111122223333:my-topic --protocol email --notification-endpoint my-email-address
```
Amazon SNS returns the following:
```
1. {
2. "SubscriptionArn": "pending confirmation"
3. }
```
1. From your email application, open the message from AWS Notifications and confirm your subscription.
Your web browser displays a confirmation response from Amazon Simple Notification Service.
1. Check the subscription using the [list-subscriptions-by-topic](https://docs.aws.amazon.com/cli/latest/reference/sns/list-subscriptions-by-topic.html) command.
```
1. aws sns list-subscriptions-by-topic --topic-arn arn:aws:sns:us-east-1:111122223333:my-topic
```
Amazon SNS returns the following:
```
1. {
2. "Subscriptions": [
3. {
4. "Owner": "111122223333",
5. "Endpoint": "me@mycompany.com",
6. "Protocol": "email",
7. "TopicArn": "arn:aws:sns:us-east-1:111122223333:my-topic",
8. "SubscriptionArn": "arn:aws:sns:us-east-1:111122223333:my-topic:64886986-bf10-48fb-a2f1-dab033aa67a3"
9. }
10. ]
11. }
```
1. (Optional) Publish a test message to the topic using the [publish](https://docs.aws.amazon.com/cli/latest/reference/sns/publish.html) command.
```
1. aws sns publish --message "Verification" --topic arn:aws:sns:us-east-1:111122223333:my-topic
```
Amazon SNS returns the following.
```
1. {
2. "MessageId": "42f189a0-3094-5cf6-8fd7-c2dde61a4d7d"
3. }
```
1. Check your email to confirm that you received the message.
## Schema of Amazon SNS notifications when alarms change state
This section lists the schemas of the notifications sent to Amazon SNS topics when alarms change their state.
**Schema when a metric alarm changes state**
```
{
"AlarmName": "string",
"AlarmDescription": "string",
"AWSAccountId": "string",
"AlarmConfigurationUpdatedTimestamp": "string",
"NewStateValue": "string",
"NewStateReason": "string",
"StateChangeTime": "string",
"Region": "string",
"AlarmArn": "string",
"OldStateValue": "string",
"OKActions": ["string"],
"AlarmActions": ["string"],
"InsufficientDataActions": ["string"],
"Trigger": {
"MetricName": "string",
"Namespace": "string",
"StatisticType": "string",
"Statistic": "string",
"Unit": "string or null",
"Dimensions": [
{
"value": "string",
"name": "string"
}
],
"Period": "integer",
"EvaluationPeriods": "integer",
"DatapointsToAlarm": "integer",
"ComparisonOperator": "string",
"Threshold": "number",
"TreatMissingData": "string",
"EvaluateLowSampleCountPercentile": "string or null"
}
}
```
**Schema when a composite alarm changes state**
```
{
"AlarmName": "string",
"AlarmDescription": "string",
"AWSAccountId": "string",
"NewStateValue": "string",
"NewStateReason": "string",
"StateChangeTime": "string",
"Region": "string",
"AlarmArn": "string",
"OKActions": [String],
"AlarmActions": [String],
"InsufficientDataActions": [String],
"OldStateValue": "string",
"AlarmRule": "string",
"TriggeringChildren": [String]
}
```
# Invoke a Lambda function from an alarm
CloudWatch alarms guarantees an asynchronous invocation of the Lambda function for a given state change, except in the following cases:
+ When the function doesn't exist.
+ When CloudWatch is not authorized to invoke the Lambda function.
If CloudWatch can't reach the Lambda service or the message is rejected for another reason, CloudWatch retries until the invocation is successful. Lambda queues the message and handles execution retries. For more information about this execution model, including information about how Lambda handles errors, see [ Asynchronous invocation](https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html) in the AWS Lambda Developer Guide.
You can invoke a Lambda function in the same account, or in other AWS accounts.
When you specify an alarm to invoke a Lambda function as an alarm action, you can choose to specify the function name, function alias, or a specific version of a function.
When you specify a Lambda function as an alarm action, you must create a resource policy for the function to allow the CloudWatch service principal to invoke the function.
One way to do this is by using the AWS CLI, as in the following example:
```
aws lambda add-permission \
--function-name my-function-name \
--statement-id AlarmAction \
--action 'lambda:InvokeFunction' \
--principal lambda.alarms.cloudwatch.amazonaws.com \
--source-account 111122223333 \
--source-arn arn:aws:cloudwatch:us-east-1:111122223333:alarm:alarm-name
```
Alternatively, you can create a policy similar to one of the following examples and then assign it to the function.
The following example specifies the account where the alarm is located, so that only alarms in that account (111122223333) can invoke the function.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Id": "default",
"Statement": [{
"Sid": "AlarmAction",
"Effect": "Allow",
"Principal": {
"Service": "lambda.alarms.cloudwatch.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:444455556666:function:function-name",
"Condition": {
"StringEquals": {
"AWS:SourceAccount": "111122223333"
}
}
}]
}
```
------
The following example has a narrower scope, allowing only the specified alarm in the specified account to invoke the function.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "AlarmAction",
"Effect": "Allow",
"Principal": {
"Service": "lambda.alarms.cloudwatch.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:444455556666:function:function-name",
"Condition": {
"StringEquals": {
"AWS:SourceAccount": "111122223333",
"AWS:SourceArn": "arn:aws:cloudwatch:us-east-1:111122223333:alarm:alarm-name"
}
}
}]
}
```
------
We don't recommend creating a policy that doesn't specify a source account, because such policies are vulnerable to confused deputy issues.
## Add Lambda metrics to CloudWatch investigations
You can add Lambda metrics to your active CloudWatch investigations. When investigating an issue, Lambda metrics can provide valuable insights about function performance and behavior. For example, if you're investigating an application performance issue, Lambda metrics such as duration, error rates, or throttles might help identify the root cause.
To add Lambda metrics to CloudWatch investigations:
1. Open the AWS Lambda console at [https://console.aws.amazon.com/lambda/](https://console.aws.amazon.com/lambda/).
1. In the **Monitor** section, find the metric.
1. Open the context menu for the metric, choose **Investigate**, **Add to investigation**. Then, in the **Investigate** pane, select the name of the investigation.
## Event object sent from CloudWatch to Lambda
When you configure a Lambda function as an alarm action, CloudWatch delivers a JSON payload to the Lambda function when it invokes the function. This JSON payload serves as the event object for the function. You can extract data from this JSON object and use it in your function. The following is an example of an event object from a metric alarm.
```
{
'source': 'aws.cloudwatch',
'alarmArn': 'arn:aws:cloudwatch:us-east-1:444455556666:alarm:lambda-demo-metric-alarm',
'accountId': '444455556666',
'time': '2023-08-04T12:36:15.490+0000',
'region': 'us-east-1',
'alarmData': {
'alarmName': 'lambda-demo-metric-alarm',
'state': {
'value': 'ALARM',
'reason': 'test',
'timestamp': '2023-08-04T12:36:15.490+0000'
},
'previousState': {
'value': 'INSUFFICIENT_DATA',
'reason': 'Insufficient Data: 5 datapoints were unknown.',
'reasonData': '{"version":"1.0","queryDate":"2023-08-04T12:31:29.591+0000","statistic":"Average","period":60,"recentDatapoints":[],"threshold":5.0,"evaluatedDatapoints":[{"timestamp":"2023-08-04T12:30:00.000+0000"},{"timestamp":"2023-08-04T12:29:00.000+0000"},{"timestamp":"2023-08-04T12:28:00.000+0000"},{"timestamp":"2023-08-04T12:27:00.000+0000"},{"timestamp":"2023-08-04T12:26:00.000+0000"}]}',
'timestamp': '2023-08-04T12:31:29.595+0000'
},
'configuration': {
'description': 'Metric Alarm to test Lambda actions',
'metrics': [
{
'id': '1234e046-06f0-a3da-9534-EXAMPLEe4c',
'metricStat': {
'metric': {
'namespace': 'AWS/Logs',
'name': 'CallCount',
'dimensions': {
'InstanceId': 'i-12345678'
}
},
'period': 60,
'stat': 'Average',
'unit': 'Percent'
},
'returnData': True
}
]
}
}
}
```
The following is an example of an event object from a composite alarm.
```
{
'source': 'aws.cloudwatch',
'alarmArn': 'arn:aws:cloudwatch:us-east-1:111122223333:alarm:SuppressionDemo.Main',
'accountId': '111122223333',
'time': '2023-08-04T12:56:46.138+0000',
'region': 'us-east-1',
'alarmData': {
'alarmName': 'CompositeDemo.Main',
'state': {
'value': 'ALARM',
'reason': 'arn:aws:cloudwatch:us-east-1:111122223333:alarm:CompositeDemo.FirstChild transitioned to ALARM at Friday 04 August, 2023 12:54:46 UTC',
'reasonData': '{"triggeringAlarms":[{"arn":"arn:aws:cloudwatch:us-east-1:111122223333:alarm:CompositeDemo.FirstChild","state":{"value":"ALARM","timestamp":"2023-08-04T12:54:46.138+0000"}}]}',
'timestamp': '2023-08-04T12:56:46.138+0000'
},
'previousState': {
'value': 'ALARM',
'reason': 'arn:aws:cloudwatch:us-east-1:111122223333:alarm:CompositeDemo.FirstChild transitioned to ALARM at Friday 04 August, 2023 12:54:46 UTC',
'reasonData': '{"triggeringAlarms":[{"arn":"arn:aws:cloudwatch:us-east-1:111122223333:alarm:CompositeDemo.FirstChild","state":{"value":"ALARM","timestamp":"2023-08-04T12:54:46.138+0000"}}]}',
'timestamp': '2023-08-04T12:54:46.138+0000',
'actionsSuppressedBy': 'WaitPeriod',
'actionsSuppressedReason': 'Actions suppressed by WaitPeriod'
},
'configuration': {
'alarmRule': 'ALARM(CompositeDemo.FirstChild) OR ALARM(CompositeDemo.SecondChild)',
'actionsSuppressor': 'CompositeDemo.ActionsSuppressor',
'actionsSuppressorWaitPeriod': 120,
'actionsSuppressorExtensionPeriod': 180
}
}
}
```
# Start a CloudWatch investigations from an alarm
Start a CloudWatch investigations from an alarm, or from any point in the last two weeks of a CloudWatch alarm's history.
For more information about CloudWatch investigations, see [CloudWatch investigations](Investigations.md).
## Prerequisites
Before you can start a CloudWatch investigations from a CloudWatch alarm, you must create a resource policy for the function to allow the CloudWatch service principal to start the investigation. To do this using the AWS CLI, use a command similar to the following example:
```
aws aiops put-investigation-group-policy \
--identifier arn:aws:aiops:us-east-1:111122223333:investigation-group/investigation_group_id \
--policy "{\"Version\":\"2008-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"aiops.alarms.cloudwatch.amazonaws.com\"},\"Action\":[\"aiops:CreateInvestigation\",\"aiops:CreateInvestigationEvent\"],\"Resource\":\"*\",\"Condition\":{\"StringEquals\":{\"aws:SourceAccount\":\"111122223333\"},\"ArnLike\":{\"aws:SourceArn\":\"arn:aws:cloudwatch:us-east-1:111122223333:alarm:*\"}}}]}" \
--region eu-north-1
```
Replace the example values with your own AWS account ID, region, and investigation group ID.
**Start an investigation from a CloudWatch alarm**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the left navigation pane, choose **Alarms**, **All alarms**.
1. Choose the name of the alarm.
1. Choose the time period in the alarm history that you want to investigate.
1. Choose **Investigate**, **Start new investigation**.
1. For **New investigation title**, enter a name for the investigation. Then choose **Start investigation**.
The CloudWatch investigations assistant starts and scans your telemetry data to find data that might be associated with this situation.
1. In the CloudWatch console's navigation pane, choose **Investigations**, then choose the name of the investigation that you just started.
The **Findings** section displays a natural-language summary of the alarm's status and the reason that it was triggered.
1. (Optional) In the graph of the alarm, right-click and choose to deep-dive into the alarm or the metric that it watches.
1. On the right side of the screen, choose the **Suggestions** tab.
A list of other telemetry that CloudWatch investigations has discovered and that might be relevant to the investigation appears. These findings can include other metrics and CloudWatch Logs Insights query results. CloudWatch investigations ran these queries based on the alarm.
+ For each finding, choose **Add to findings** or **Discard**.
When you choose **Add to findings**, the telemetry is added to the **Findings** section, and CloudWatch investigations uses this information to direct its further scanning and suggestions.
+ For a CloudWatch Logs Insights query result, to change or edit the query and re-run it, open the context (right-click) menu for the results, and then choose **Open in Logs Insights**. For more information, see [Analyzing log data with CloudWatch Logs Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html).
To run a different query, when you get to the Logs Insights page, choose to use query assist to form a query using natural language. For more information, see [Use natural language to generate and update CloudWatch Logs Insights queries](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CloudWatchLogs-Insights-Query-Assist.html).
+ (Optional) If you know of telemetry in another AWS service that might apply to this investigation, go to that service's console and add the telemetry to the investigation.
1. CloudWatch investigations might also add hypotheses to the list in the **Suggestions** tab. These hypotheses are generated by the investigation in natural language.
For each hypothesis, choose **Add to findings** or **Discard**.
1. When you think you have completed the investigation and found the root cause of the issue, choose the **Overview** tab and then choose **Investigation summary**. CloudWatch investigations then creates a natural-language summary of the important findings and hypotheses from the investigation.
# Stop, terminate, reboot, or recover an EC2 instance
Using Amazon CloudWatch alarm actions, you can create alarms that automatically stop, terminate, reboot, or recover your EC2 instances. You can use the stop or terminate actions to help you save money when you no longer need an instance to be running. You can use the reboot and recover actions to automatically reboot those instances or recover them onto new hardware if a system impairment occurs.
There are a number of scenarios in which you might want to automatically stop or terminate your instance. For example, you might have instances dedicated to batch payroll processing jobs or scientific computing tasks that run for a period of time and then complete their work. Rather than letting those instances sit idle (and accrue charges), you can stop or terminate them, which helps you to save money. The main difference between using the stop and the terminate alarm actions is that you can easily restart a stopped instance if you need to run it again later. You can also keep the same instance ID and root volume. However, you cannot restart a terminated instance. Instead, you must launch a new instance.
You can add the stop, terminate, or reboot, actions to any alarm that is set on an Amazon EC2 per-instance metric, including basic and detailed monitoring metrics provided by Amazon CloudWatch (in the AWS/EC2 namespace), in addition to any custom metrics that include the "InstanceId=" dimension, as long as the InstanceId value refers to a valid running Amazon EC2 instance. You can also add the recover action to alarms that is set on any Amazon EC2 per-instance metric except for `StatusCheckFailed_Instance`.
**Important**
Alarms configured on Amazon EC2 metrics can temporarily enter the INSUFFICIENT\$1DATA state if there are missing metric data points. This is rare, but can happen when the metric reporting is interrupted, even when the Amazon EC2 instance is healthy. For alarms on Amazon EC2 metrics that are configured to take stop, terminate, reboot, or recover actions, we recommend that you configure those alarms to treat missing data as `missing`, and to have these alarms trigger only when in the ALARM state.
For more information about how you can configure CloudWatch to act on missing metrics that have alarms set on them, see [Configuring how CloudWatch alarms treat missing data](alarms-and-missing-data.md).
To set up a CloudWatch alarm action that can reboot, stop, or terminate an instance, you must use a service-linked IAM role, *AWSServiceRoleForCloudWatchEvents*. The AWSServiceRoleForCloudWatchEvents IAM role enables AWS to perform alarm actions on your behalf.
To create the service-linked role for CloudWatch Events, use the following command:
```
aws iam create-service-linked-role --aws-service-name events.amazonaws.com
```
**Console support**
You can create alarms using the CloudWatch console or the Amazon EC2 console. The procedures in this documentation use the CloudWatch console. For procedures that use the Amazon EC2 console, see [Create alarms that stop, terminate, reboot, or rcover an instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/UsingAlarmActions.html) in the *Amazon EC2 User Guide*.
**Permissions**
If you are using an AWS Identity and Access Management (IAM) account to create or modify an alarm that performs EC2 actions or Systems Manager OpsItem actions, you must have the `iam:CreateServiceLinkedRole` permission.
**Topics**
+ [Adding stop actions to Amazon CloudWatch alarms](#AddingStopActions)
+ [Adding terminate actions to Amazon CloudWatch alarms](#AddingTerminateActions)
+ [Adding reboot actions to Amazon CloudWatch alarms](#AddingRebootActions)
+ [Adding recover actions to Amazon CloudWatch alarms](#AddingRecoverActions)
+ [Viewing the history of triggered alarms and actions](#ViewAlarmHistory)
## Adding stop actions to Amazon CloudWatch alarms
You can create an alarm that stops an Amazon EC2 instance when a certain threshold has been met. For example, you may run development or test instances and occasionally forget to shut them off. You can create an alarm that is triggered when the average CPU utilization percentage has been lower than 10 percent for 24 hours, signaling that it is idle and no longer in use. You can adjust the threshold, duration, and period to suit your needs, plus you can add an SNS notification, so that you will receive an email when the alarm is triggered.
Amazon EC2 instances that use an Amazon Elastic Block Store volume as the root device can be stopped or terminated, whereas instances that use the instance store as the root device can only be terminated.
**To create an alarm to stop an idle instance using the Amazon CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Alarms**, **All alarms**.
1. Choose **Create alarm**.
1. Choose **Select Metric**.
1. For **AWS namespaces**, choose **EC2**.
1. Do the following:
1. Choose **Per-Instance Metrics**.
1. Select the check box in the row with the correct instance and the **CPUUtilization** metric.
1. Choose the **Graphed metrics** tab.
1. For the statistic, choose **Average**.
1. Choose a period (for example, **1 Hour**).
1. Choose **Select metric**.
1. Under **Conditions**, do the following:
1. Choose **Static**.
1. Under **Whenever CPUUtilization is**, choose **Lower**.
1. For **than**, type **10**.
1. Choose **Next**.
1. Under **Notification**, for **Send notification to**, choose an existing SNS topic or create a new one.
To create an SNS topic, choose **New list**. For **Send notification to**, type a name for the SNS topic (for example, Stop\$1EC2\$1Instance). For **Email list**, type a comma-separated list of email addresses to be notified when the alarm changes to the `ALARM` state. Each email address is sent a topic subscription confirmation email. You must confirm the subscription before notifications can be sent to an email address.
1. Choose **Add EC2 Action**.
1. For **Alarm state trigger**, choose **In alarm**. For **Take the following action**, choose **Stop this instance**.
1. Choose **Next**.
1. Enter a name and description for the alarm. The name must contain only ASCII characters. Then choose **Next**.
1. Under **Preview and create**, confirm that the information and conditions are what you want, then choose **Create alarm**.
## Adding terminate actions to Amazon CloudWatch alarms
You can create an alarm that terminates an EC2 instance automatically when a certain threshold has been met (as long as termination protection is not enabled for the instance). For example, you might want to terminate an instance when it has completed its work, and you don't need the instance again. If you might want to use the instance later, you should stop the instance instead of terminating it. For information about enabling and disabling termination protection for an instance, see [Enabling Termination Protection for an Instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_ChangingDisableAPITermination.html) in the *Amazon EC2 User Guide*.
**To create an alarm to terminate an idle instance using the Amazon CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Alarms**, **Create Alarm**.
1. For the **Select Metric** step, do the following:
1. Under **EC2 Metrics**, choose **Per-Instance Metrics**.
1. Select the row with the instance and the **CPUUtilization** metric.
1. For the statistic, choose **Average**.
1. Choose a period (for example, **1 Hour**).
1. Choose **Next**.
1. For the **Define Alarm** step, do the following:
1. Under **Alarm Threshold**, type a unique name for the alarm (for example, Terminate EC2 instance) and a description of the alarm (for example, Terminate EC2 instance when CPU is idle for too long). Alarm names must contain only ASCII characters.
1. Under **Whenever**, for **is**, choose **<** and type **10**. For **for**, type **24** consecutive periods.
A graphical representation of the threshold is shown under **Alarm Preview**.
1. Under **Notification**, for **Send notification to**, choose an existing SNS topic or create a new one.
To create an SNS topic, choose **New list**. For **Send notification to**, type a name for the SNS topic (for example, Terminate\$1EC2\$1Instance). For **Email list**, type a comma-separated list of email addresses to be notified when the alarm changes to the `ALARM` state. Each email address is sent a topic subscription confirmation email. You must confirm the subscription before notifications can be sent to an email address.
1. Choose **EC2 Action**.
1. For **Whenever this alarm**, choose **State is ALARM**. For **Take this action**, choose **Terminate this instance**.
1. Choose **Create Alarm**.
## Adding reboot actions to Amazon CloudWatch alarms
You can create an Amazon CloudWatch alarm that monitors an Amazon EC2 instance and automatically reboots the instance. The reboot alarm action is recommended for Instance Health Check failures (as opposed to the recover alarm action, which is suited for System Health Check failures). An instance reboot is equivalent to an operating system reboot. In most cases, it takes only a few minutes to reboot your instance. When you reboot an instance, it remains on the same physical host, so your instance keeps its public DNS name, private IP address, and any data on its instance store volumes.
Rebooting an instance doesn't start a new instance billing hour, unlike stopping and restarting your instance. For more information about rebooting an instance, see [Reboot Your Instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-reboot.html) in the *Amazon EC2 User Guide*.
**Important**
To avoid a race condition between the reboot and recover actions, avoid setting the same evaluation period for both a reboot alarm and a recover alarm. We recommend that you set reboot alarms to three evaluation periods of one minute each.
**To create an alarm to reboot an instance using the Amazon CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Alarms**, **Create Alarm**.
1. For the **Select Metric** step, do the following:
1. Under **EC2 Metrics**, choose **Per-Instance Metrics**.
1. Select the row with the instance and the **StatusCheckFailed\$1Instance** metric.
1. For the statistic, choose **Minimum**.
1. Choose a period (for example, **1 Minute**).
1. Choose **Next**.
1. For the **Define Alarm** step, do the following:
1. Under **Alarm Threshold**, type a unique name for the alarm (for example, Reboot EC2 instance) and a description of the alarm (for example, Reboot EC2 instance when health checks fail). Alarm names must contain only ASCII characters.
1. Under **Whenever**, for **is**, choose **>** and type **0**. For **for**, type **3** consecutive periods.
A graphical representation of the threshold is shown under **Alarm Preview**.
1. Under **Notification**, for **Send notification to**, choose an existing SNS topic or create a new one.
To create an SNS topic, choose **New list**. For **Send notification to**, type a name for the SNS topic (for example, Reboot\$1EC2\$1Instance). For **Email list**, type a comma-separated list of email addresses to be notified when the alarm changes to the `ALARM` state. Each email address is sent a topic subscription confirmation email. You must confirm the subscription before notifications can be sent to an email address.
1. Choose **EC2 Action**.
1. For **Whenever this alarm**, choose **State is ALARM**. For **Take this action**, choose **Reboot this instance**.
1. Choose **Create Alarm**.
## Adding recover actions to Amazon CloudWatch alarms
You can create an Amazon CloudWatch alarm that monitors an Amazon EC2 instance and automatically recovers the instance if it becomes impaired due to an underlying hardware failure or a problem that requires AWS involvement to repair. Terminated instances cannot be recovered. A recovered instance is identical to the original instance, including the instance ID, private IP addresses, Elastic IP addresses, and all instance metadata.
When the `StatusCheckFailed_System` alarm is triggered, and the recover action is initiated, you will be notified by the Amazon SNS topic that you chose when you created the alarm and associated the recover action. During instance recovery, the instance is migrated during an instance reboot, and any data that is in-memory is lost. When the process is complete, information is published to the SNS topic you've configured for the alarm. Anyone who is subscribed to this SNS topic will receive an email notification that includes the status of the recovery attempt and any further instructions. You will notice an instance reboot on the recovered instance.
The recover action can be used only with `StatusCheckFailed_System`, not with `StatusCheckFailed_Instance`.
Examples of problems that cause system status checks to fail include:
+ Loss of network connectivity
+ Loss of system power
+ Software issues on the physical host
+ Hardware issues on the physical host that impact network reachability
The recover action is supported only on some instance types. For more information about supported instance types and other requirements, see [ Recover your instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-recover.html) and [ Requirements](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-recover.html#requirements-for-recovery).
**Important**
To avoid a race condition between the reboot and recover actions, avoid setting the same evaluation period for both a reboot alarm and a recover alarm. We recommend that you set recover alarms to two evaluation periods of one minute each and reboot alarms to three evaluation periods of one minute each.
**To create an alarm to recover an instance using the Amazon CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Alarms**, **All alarms**.
1. Choose **Create Alarm**.
1. Choose **Select Metric** and then do the following:
1. Choose **EC2 Metrics**, **Per-Instance Metrics**.
1. Select the row with the instance and the **StatusCheckFailed\$1System** metric, and then choose **Select metric**.
1. For the statistic, choose **Minimum**.
1. Choose a period (for example, **1 Minute**).
**Important**
To avoid a race condition between the reboot and recover actions, avoid setting the same evaluation period for both a reboot alarm and a recover alarm. We recommend that you set recover alarms to two evaluation periods of one minute each.
1. For **Conditions**, do the following:
1. Under **Threshold type**, choose **Static**.
1. Under **Whenever**, choose **Greater** and enter **0** for **than...**.
1. Choose **Additional configuration**, then for **Datapoints to alarm** specify 2 **out of** 2.
1. Choose **Next**.
1. Under **Notification**, do the following:
1. For **Alarm state trigger**, choose **In alarm**.
1. For **Send notification to the following SNS topic**, choose an existing SNS topic or create a new one.
1. Choose **Add EC2 Action**.
1. For **Alarm state trigger**, choose **In alarm**.
1. For **Take the following action**, choose **Recover this instance**.
1. Choose **Next**.
1. For **Alarm name**, type a unique name for the alarm (for example, **Recover EC2 instance**) and a description of the alarm (for example, **Recover EC2 instance when health checks fail**). Alarm names must contain only ASCII characters.
1. Choose **Next**.
1. Choose **Create Alarm**.
## Viewing the history of triggered alarms and actions
You can view alarm and action history in the Amazon CloudWatch console. Amazon CloudWatch keeps the last 30 days of alarm and action history.
**To view the history of triggered alarms and actions**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Alarms** and select an alarm.
1. To view the most recent state transition along with the time and metric values, choose **Details**.
1. To view the most recent history entries, choose **History**.
# Alarm events and EventBridge
CloudWatch sends events to Amazon EventBridge whenever a CloudWatch alarm is created, updated, deleted, or changes alarm state. You can use EventBridge and these events to write rules that take actions, such as notifying you, when an alarm changes state. For more information, see [What is Amazon EventBridge?](https://docs.aws.amazon.com/eventbridge/latest/userguide/what-is-amazon-eventbridge.html)
CloudWatch guarantees the delivery of alarm state change events to EventBridge.
## Alarm State Change Events
This section shows example events sent to EventBridge when an alarm's state changes. Select a tab to view different types of alarm state change events.
------
#### [ Single Metric Alarm ]
Events generated when a single metric alarm changes state. These events include `state` and `previousState` fields with the alarm's evaluation results.
```
{
"version": "0",
"id": "c4c1c1c9-6542-e61b-6ef0-8c4d36933a92",
"detail-type": "CloudWatch Alarm State Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2019-10-02T17:04:40Z",
"region": "us-east-1",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:ServerCpuTooHigh"
],
"detail": {
"alarmName": "ServerCpuTooHigh",
"configuration": {
"description": "Goes into alarm when server CPU utilization is too high!",
"metrics": [
{
"id": "30b6c6b2-a864-43a2-4877-c09a1afc3b87",
"metricStat": {
"metric": {
"dimensions": {
"InstanceId": "i-12345678901234567"
},
"name": "CPUUtilization",
"namespace": "AWS/EC2"
},
"period": 300,
"stat": "Average"
},
"returnData": true
}
]
},
"previousState": {
"reason": "Threshold Crossed: 1 out of the last 1 datapoints [0.0666851903306472 (01/10/19 13:46:00)] was not greater than the threshold (50.0) (minimum 1 datapoint for ALARM -> OK transition).",
"reasonData": "{\"version\":\"1.0\",\"queryDate\":\"2019-10-01T13:56:40.985+0000\",\"startDate\":\"2019-10-01T13:46:00.000+0000\",\"statistic\":\"Average\",\"period\":300,\"recentDatapoints\":[0.0666851903306472],\"threshold\":50.0}",
"timestamp": "2019-10-01T13:56:40.987+0000",
"value": "OK"
},
"state": {
"reason": "Threshold Crossed: 1 out of the last 1 datapoints [99.50160229693434 (02/10/19 16:59:00)] was greater than the threshold (50.0) (minimum 1 datapoint for OK -> ALARM transition).",
"reasonData": "{\"version\":\"1.0\",\"queryDate\":\"2019-10-02T17:04:40.985+0000\",\"startDate\":\"2019-10-02T16:59:00.000+0000\",\"statistic\":\"Average\",\"period\":300,\"recentDatapoints\":[99.50160229693434],\"threshold\":50.0}",
"timestamp": "2019-10-02T17:04:40.989+0000",
"value": "ALARM"
},
"muteDetail": {
"mutedByArn": "arn:aws:cloudwatch:us-east-1:1234567890:alarm-mute-rule:testMute",
"muteWindowStart": "2026-01-01T10:00:00.000+0000",
"muteWindowEnd": "2026-01-01T12:00:00.000+0000"
}
}
}
```
------
#### [ Metric Math Alarm ]
Events generated when a metric math alarm changes state. These events include the math expression details in the `configuration` field.
```
{
"version": "0",
"id": "2dde0eb1-528b-d2d5-9ca6-6d590caf2329",
"detail-type": "CloudWatch Alarm State Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2019-10-02T17:20:48Z",
"region": "us-east-1",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:TotalNetworkTrafficTooHigh"
],
"detail": {
"alarmName": "TotalNetworkTrafficTooHigh",
"configuration": {
"description": "Goes into alarm if total network traffic exceeds 10Kb",
"metrics": [
{
"expression": "SUM(METRICS())",
"id": "e1",
"label": "Total Network Traffic",
"returnData": true
},
{
"id": "m1",
"metricStat": {
"metric": {
"dimensions": {
"InstanceId": "i-12345678901234567"
},
"name": "NetworkIn",
"namespace": "AWS/EC2"
},
"period": 300,
"stat": "Maximum"
},
"returnData": false
},
{
"id": "m2",
"metricStat": {
"metric": {
"dimensions": {
"InstanceId": "i-12345678901234567"
},
"name": "NetworkOut",
"namespace": "AWS/EC2"
},
"period": 300,
"stat": "Maximum"
},
"returnData": false
}
]
},
"previousState": {
"reason": "Unchecked: Initial alarm creation",
"timestamp": "2019-10-02T17:20:03.642+0000",
"value": "INSUFFICIENT_DATA"
},
"state": {
"reason": "Threshold Crossed: 1 out of the last 1 datapoints [45628.0 (02/10/19 17:10:00)] was greater than the threshold (10000.0) (minimum 1 datapoint for OK -> ALARM transition).",
"reasonData": "{\"version\":\"1.0\",\"queryDate\":\"2019-10-02T17:20:48.551+0000\",\"startDate\":\"2019-10-02T17:10:00.000+0000\",\"period\":300,\"recentDatapoints\":[45628.0],\"threshold\":10000.0}",
"timestamp": "2019-10-02T17:20:48.554+0000",
"value": "ALARM"
},
"muteDetail": {
"mutedByArn": "arn:aws:cloudwatch:us-east-1:1234567890:alarm-mute-rule:testMute",
"muteWindowStart": "2026-01-01T10:00:00.000+0000",
"muteWindowEnd": "2026-01-01T12:00:00.000+0000"
}
}
}
```
------
#### [ Anomaly Detection Alarm ]
Events generated when an anomaly detection alarm changes state. These events include upper and lower threshold bounds in the `reasonData` field.
```
{
"version": "0",
"id": "daafc9f1-bddd-c6c9-83af-74971fcfc4ef",
"detail-type": "CloudWatch Alarm State Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2019-10-03T16:00:04Z",
"region": "us-east-1",
"resources": ["arn:aws:cloudwatch:us-east-1:123456789012:alarm:EC2 CPU Utilization Anomaly"],
"detail": {
"alarmName": "EC2 CPU Utilization Anomaly",
"state": {
"value": "ALARM",
"reason": "Thresholds Crossed: 1 out of the last 1 datapoints [0.0 (03/10/19 15:58:00)] was less than the lower thresholds [0.020599444741798756] or greater than the upper thresholds [0.3006915352732461] (minimum 1 datapoint for OK -> ALARM transition).",
"reasonData": "{\"version\":\"1.0\",\"queryDate\":\"2019-10-03T16:00:04.650+0000\",\"startDate\":\"2019-10-03T15:58:00.000+0000\",\"period\":60,\"recentDatapoints\":[0.0],\"recentLowerThresholds\":[0.020599444741798756],\"recentUpperThresholds\":[0.3006915352732461]}",
"timestamp": "2019-10-03T16:00:04.653+0000"
},
"previousState": {
"value": "OK",
"reason": "Thresholds Crossed: 1 out of the last 1 datapoints [0.166666666664241 (03/10/19 15:57:00)] was not less than the lower thresholds [0.0206719426210418] or not greater than the upper thresholds [0.30076870222143803] (minimum 1 datapoint for ALARM -> OK transition).",
"reasonData": "{\"version\":\"1.0\",\"queryDate\":\"2019-10-03T15:59:04.670+0000\",\"startDate\":\"2019-10-03T15:57:00.000+0000\",\"period\":60,\"recentDatapoints\":[0.166666666664241],\"recentLowerThresholds\":[0.0206719426210418],\"recentUpperThresholds\":[0.30076870222143803]}",
"timestamp": "2019-10-03T15:59:04.672+0000"
},
"muteDetail": {
"mutedByArn": "arn:aws:cloudwatch:us-east-1:1234567890:alarm-mute-rule:testMute",
"muteWindowStart": "2026-01-01T10:00:00.000+0000",
"muteWindowEnd": "2026-01-01T12:00:00.000+0000"
},
"configuration": {
"description": "Goes into alarm if CPU Utilization is out of band",
"metrics": [{
"id": "m1",
"metricStat": {
"metric": {
"namespace": "AWS/EC2",
"name": "CPUUtilization",
"dimensions": {
"InstanceId": "i-12345678901234567"
}
},
"period": 60,
"stat": "Average"
},
"returnData": true
}, {
"id": "ad1",
"expression": "ANOMALY_DETECTION_BAND(m1, 0.8)",
"label": "CPUUtilization (expected)",
"returnData": true
}]
}
}
}
```
------
#### [ Composite Alarm ]
Events generated when a composite alarm changes state. These events include suppression information in the `actionsSuppressedBy` and `actionsSuppressedReason` fields.
```
{
"version": "0",
"id": "d3dfc86d-384d-24c8-0345-9f7986db0b80",
"detail-type": "CloudWatch Alarm State Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2022-07-22T15:57:45Z",
"region": "us-east-1",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:ServiceAggregatedAlarm"
],
"detail": {
"alarmName": "ServiceAggregatedAlarm",
"state": {
"actionsSuppressedBy": "WaitPeriod",
"actionsSuppressedReason": "Actions suppressed by WaitPeriod",
"value": "ALARM",
"reason": "arn:aws:cloudwatch:us-east-1:123456789012:alarm:SuppressionDemo.EventBridge.FirstChild transitioned to ALARM at Friday 22 July, 2022 15:57:45 UTC",
"reasonData": "{\"triggeringAlarms\":[{\"arn\":\"arn:aws:cloudwatch:us-east-1:123456789012:alarm:ServerCpuTooHigh\",\"state\":{\"value\":\"ALARM\",\"timestamp\":\"2022-07-22T15:57:45.394+0000\"}}]}",
"timestamp": "2022-07-22T15:57:45.394+0000"
},
"previousState": {
"value": "OK",
"reason": "arn:aws:cloudwatch:us-east-1:123456789012:alarm:SuppressionDemo.EventBridge.Main was created and its alarm rule evaluates to OK",
"reasonData": "{\"triggeringAlarms\":[{\"arn\":\"arn:aws:cloudwatch:us-east-1:123456789012:alarm:TotalNetworkTrafficTooHigh\",\"state\":{\"value\":\"OK\",\"timestamp\":\"2022-07-14T16:28:57.770+0000\"}},{\"arn\":\"arn:aws:cloudwatch:us-east-1:123456789012:alarm:ServerCpuTooHigh\",\"state\":{\"value\":\"OK\",\"timestamp\":\"2022-07-14T16:28:54.191+0000\"}}]}",
"timestamp": "2022-07-22T15:56:14.552+0000"
},
"configuration": {
"alarmRule": "ALARM(ServerCpuTooHigh) OR ALARM(TotalNetworkTrafficTooHigh)",
"actionsSuppressor": "ServiceMaintenanceAlarm",
"actionsSuppressorWaitPeriod": 120,
"actionsSuppressorExtensionPeriod": 180
},
"muteDetail": {
"mutedByArn": "arn:aws:cloudwatch:us-east-1:1234567890:alarm-mute-rule:testMute",
"muteWindowStart": "2026-01-01T10:00:00.000+0000",
"muteWindowEnd": "2026-01-01T12:00:00.000+0000"
}
}
}
```
------
#### [ Multi Time Series Alarm ]
Events generated when an Alarm Contributor or an Alarm changes state. Alarm Contributor state change events contain the id and attributes of the Alarm Contributor as well as the most recent datapoint that breached the threshold. Alarm State change events have a summary of the number of contributors that caused the alarm to transition in their state reason.
**Alarm Contributor Example**
```
{
"version": "0",
"id": "6d226bbc-07f0-9a31-3359-1736968f8ded",
"detail-type": "CloudWatch Alarm Contributor State Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2025-12-01T13:42:04Z",
"region": "us-east-1",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:DynamoDBInsightsAlarm"
],
"detail": {
"alarmName": "DynamoDBInsightsAlarm",
"alarmContributor": {
"id": "6d442278dba546f6",
"attributes": {
"TableName": "example-dynamodb-table-name"
}
},
"state": {
"value": "ALARM",
"reason": "Threshold Crossed: 1 datapoint was less than the threshold (1.0). The most recent datapoint which crossed the threshold: [0.0 (01/12/25 13:34:00)].",
"timestamp": "2025-12-01T13:42:04.919+0000"
},
"configuration": {
"metrics": [
{
"id": "m1",
"expression": "SELECT AVG(ConsumedWriteCapacityUnits) FROM \"AWS/DynamoDB\" GROUP BY TableName ORDER BY MAX() DESC",
"returnData":true,
"period": 60
}
],
"description": "Metrics Insights alarm for DynamoDB ConsumedWriteCapacity per TableName"
},
"muteDetail": {
"mutedByArn": "arn:aws:cloudwatch:us-east-1:1234567890:alarm-mute-rule:testMute",
"muteWindowStart": "2026-01-01T10:00:00.000+0000",
"muteWindowEnd": "2026-01-01T12:00:00.000+0000"
}
}
}
```
**Alarm Example**
```
{
"version": "0",
"id": "80ddd249-dedf-7c4d-0708-0eb78132dd78",
"detail-type": "CloudWatch Alarm State Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2025-12-01T13:42:04Z",
"region": "us-east-1",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:DynamoDBInsightsAlarm"
],
"detail": {
"alarmName": "DynamoDBInsightsAlarm",
"state": {
"value": "ALARM",
"reason": "6 out of 6 time series evaluated to ALARM",
"timestamp": "2025-12-01T13:42:04.919+0000"
},
"previousState": {
"value": "INSUFFICIENT_DATA",
"reason": "Unchecked: Initial alarm creation",
"timestamp": "2025-12-01T13:40:50.600+0000"
},
"configuration": {
"metrics": [
{
"id": "m1",
"expression": "SELECT AVG(ConsumedWriteCapacityUnits) FROM \"AWS/DynamoDB\" GROUP BY TableName ORDER BY MAX() DESC",
"returnData": true,
"period": 60
}
],
"description": "Metrics Insights alarm for DynamoDB ConsumedWriteCapacity per TableName"
}
}
}
```
------
## Alarm Configuration Change Events
This section shows example events sent to EventBridge when an alarm's configuration changes. Configuration changes include creating, updating, or deleting alarms.
------
#### [ Creation Events ]
Events generated when new alarms are created. These events include the initial alarm configuration in the `configuration` field, with `operation` set to "create".
**Composite Alarm Example**
```
{
"version": "0",
"id": "91535fdd-1e9c-849d-624b-9a9f2b1d09d0",
"detail-type": "CloudWatch Alarm Configuration Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2022-03-03T17:06:22Z",
"region": "us-east-1",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:ServiceAggregatedAlarm"
],
"detail": {
"alarmName": "ServiceAggregatedAlarm",
"operation": "create",
"state": {
"value": "INSUFFICIENT_DATA",
"timestamp": "2022-03-03T17:06:22.289+0000"
},
"configuration": {
"alarmRule": "ALARM(ServerCpuTooHigh) OR ALARM(TotalNetworkTrafficTooHigh)",
"alarmName": "ServiceAggregatedAlarm",
"description": "Aggregated monitor for instance",
"actionsEnabled": true,
"timestamp": "2022-03-03T17:06:22.289+0000",
"okActions": [],
"alarmActions": [],
"insufficientDataActions": []
}
}
}
```
**Composite Alarm with Suppressor Example**
```
{
"version": "0",
"id": "454773e1-09f7-945b-aa2c-590af1c3f8e0",
"detail-type": "CloudWatch Alarm Configuration Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2022-07-14T13:59:46Z",
"region": "us-east-1",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:ServiceAggregatedAlarm"
],
"detail": {
"alarmName": "ServiceAggregatedAlarm",
"operation": "create",
"state": {
"value": "INSUFFICIENT_DATA",
"timestamp": "2022-07-14T13:59:46.425+0000"
},
"configuration": {
"alarmRule": "ALARM(ServerCpuTooHigh) OR ALARM(TotalNetworkTrafficTooHigh)",
"actionsSuppressor": "ServiceMaintenanceAlarm",
"actionsSuppressorWaitPeriod": 120,
"actionsSuppressorExtensionPeriod": 180,
"alarmName": "ServiceAggregatedAlarm",
"actionsEnabled": true,
"timestamp": "2022-07-14T13:59:46.425+0000",
"okActions": [],
"alarmActions": [],
"insufficientDataActions": []
}
}
}
```
------
#### [ Update Events ]
Events generated when existing alarms are modified. These events contain both `configuration` and `previousConfiguration` fields to show what changed.
**Metric Alarm Example**
```
{
"version": "0",
"id": "bc7d3391-47f8-ae47-f457-1b4d06118d50",
"detail-type": "CloudWatch Alarm Configuration Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2022-03-03T17:06:34Z",
"region": "us-east-1",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:ServerCpuTooHigh"
],
"detail": {
"alarmName": "ServerCpuTooHigh",
"operation": "update",
"state": {
"value": "INSUFFICIENT_DATA",
"timestamp": "2022-03-03T17:06:13.757+0000"
},
"configuration": {
"evaluationPeriods": 1,
"threshold": 80,
"comparisonOperator": "GreaterThanThreshold",
"treatMissingData": "ignore",
"metrics": [
{
"id": "86bfa85f-b14c-ebf7-8916-7da014ce23c0",
"metricStat": {
"metric": {
"namespace": "AWS/EC2",
"name": "CPUUtilization",
"dimensions": {
"InstanceId": "i-12345678901234567"
}
},
"period": 300,
"stat": "Average"
},
"returnData": true
}
],
"alarmName": "ServerCpuTooHigh",
"description": "Goes into alarm when server CPU utilization is too high!",
"actionsEnabled": true,
"timestamp": "2022-03-03T17:06:34.267+0000",
"okActions": [],
"alarmActions": [],
"insufficientDataActions": []
},
"previousConfiguration": {
"evaluationPeriods": 1,
"threshold": 70,
"comparisonOperator": "GreaterThanThreshold",
"treatMissingData": "ignore",
"metrics": [
{
"id": "d6bfa85f-893e-b052-a58b-4f9295c9111a",
"metricStat": {
"metric": {
"namespace": "AWS/EC2",
"name": "CPUUtilization",
"dimensions": {
"InstanceId": "i-12345678901234567"
}
},
"period": 300,
"stat": "Average"
},
"returnData": true
}
],
"alarmName": "ServerCpuTooHigh",
"description": "Goes into alarm when server CPU utilization is too high!",
"actionsEnabled": true,
"timestamp": "2022-03-03T17:06:13.757+0000",
"okActions": [],
"alarmActions": [],
"insufficientDataActions": []
}
}
}
```
**Metric Alarm with Suppressor Example**
```
{
"version": "0",
"id": "4c6f4177-6bd5-c0ca-9f05-b4151c54568b",
"detail-type": "CloudWatch Alarm Configuration Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2022-07-14T13:59:56Z",
"region": "us-east-1",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:ServiceAggregatedAlarm"
],
"detail": {
"alarmName": "ServiceAggregatedAlarm",
"operation": "update",
"state": {
"actionsSuppressedBy": "WaitPeriod",
"value": "ALARM",
"timestamp": "2022-07-14T13:59:46.425+0000"
},
"configuration": {
"alarmRule": "ALARM(ServerCpuTooHigh) OR ALARM(TotalNetworkTrafficTooHigh)",
"actionsSuppressor": "ServiceMaintenanceAlarm",
"actionsSuppressorWaitPeriod": 120,
"actionsSuppressorExtensionPeriod": 360,
"alarmName": "ServiceAggregatedAlarm",
"actionsEnabled": true,
"timestamp": "2022-07-14T13:59:56.290+0000",
"okActions": [],
"alarmActions": [], Remove
"insufficientDataActions": []
},
"previousConfiguration": {
"alarmRule": "ALARM(ServerCpuTooHigh) OR ALARM(TotalNetworkTrafficTooHigh)",
"actionsSuppressor": "ServiceMaintenanceAlarm",
"actionsSuppressorWaitPeriod": 120,
"actionsSuppressorExtensionPeriod": 180,
"alarmName": "ServiceAggregatedAlarm",
"actionsEnabled": true,
"timestamp": "2022-07-14T13:59:46.425+0000",
"okActions": [],
"alarmActions": [],
"insufficientDataActions": []
}
}
}
```
------
#### [ Deletion Events ]
Events generated when alarms are deleted. These events include the final alarm configuration and set `operation` to "delete".
**Metric Math Alarm Example**
```
{
"version": "0",
"id": "f171d220-9e1c-c252-5042-2677347a83ed",
"detail-type": "CloudWatch Alarm Configuration Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2022-03-03T17:07:13Z",
"region": "us-east-*",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:TotalNetworkTrafficTooHigh"
],
"detail": {
"alarmName": "TotalNetworkTrafficTooHigh",
"operation": "delete",
"state": {
"value": "INSUFFICIENT_DATA",
"timestamp": "2022-03-03T17:06:17.672+0000"
},
"configuration": {
"evaluationPeriods": 1,
"threshold": 10000,
"comparisonOperator": "GreaterThanThreshold",
"treatMissingData": "ignore",
"metrics": [{
"id": "m1",
"metricStat": {
"metric": {
"namespace": "AWS/EC2",
"name": "NetworkIn",
"dimensions": {
"InstanceId": "i-12345678901234567"
}
},
"period": 300,
"stat": "Maximum"
},
"returnData": false
},
{
"id": "m2",
"metricStat": {
"metric": {
"namespace": "AWS/EC2",
"name": "NetworkOut",
"dimensions": {
"InstanceId": "i-12345678901234567"
}
},
"period": 300,
"stat": "Maximum"
},
"returnData": false
},
{
"id": "e1",
"expression": "SUM(METRICS())",
"label": "Total Network Traffic",
"returnData": true
}
],
"alarmName": "TotalNetworkTrafficTooHigh",
"description": "Goes into alarm if total network traffic exceeds 10Kb",
"actionsEnabled": true,
"timestamp": "2022-03-03T17:06:17.672+0000",
"okActions": [],
"alarmActions": [],
"insufficientDataActions": []
}
}
}
```
**Metric Math Alarm with Suppressor Example**
```
{
"version": "0",
"id": "e34592a1-46c0-b316-f614-1b17a87be9dc",
"detail-type": "CloudWatch Alarm Configuration Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2022-07-14T14:00:01Z",
"region": "us-east-*",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:ServiceAggregatedAlarm"
],
"detail": {
"alarmName": "ServiceAggregatedAlarm",
"operation": "delete",
"state": {
"actionsSuppressedBy": "WaitPeriod",
"value": "ALARM",
"timestamp": "2022-07-14T13:59:46.425+0000"
},
"configuration": {
"alarmRule": "ALARM(ServerCpuTooHigh) OR ALARM(TotalNetworkTrafficTooHigh)",
"actionsSuppressor": "ServiceMaintenanceAlarm",
"actionsSuppressorWaitPeriod": 120,
"actionsSuppressorExtensionPeriod": 360,
"alarmName": "ServiceAggregatedAlarm",
"actionsEnabled": true,
"timestamp": "2022-07-14T13:59:56.290+0000",
"okActions": [],
"alarmActions": [],
"insufficientDataActions": []
}
}
}
```
------