# Using Amazon SQS, Amazon SNS, and Lambda Using SQS, SNS, and Lambda Enabling notifications is a bucket-level operation. You store notification configuration information in the *notification* subresource that's associated with a bucket. After you create or change the bucket notification configuration, it usually takes about five minutes for the changes to take effect. When the notification is first enabled, an `s3:TestEvent` occurs. You can use any of the following methods to manage notification configuration: + **Using the Amazon S3 console** — You can use the console UI to set a notification configuration on a bucket without having to write any code. For more information, see [Enabling and configuring event notifications using the Amazon S3 console](enable-event-notifications.md). + **Programmatically using the AWS SDKs** — Internally, both the console and the SDKs call the Amazon S3 REST API to manage *notification* subresources that are associated with the bucket. For examples of notification configurations that use AWS SDK, see [Walkthrough: Configuring a bucket for notifications (SNS topic or SQS queue)](ways-to-add-notification-config-to-bucket.md). **Note** You can also make the Amazon S3 REST API calls directly from your code. However, this can be cumbersome because to do so you must write code to authenticate your requests. Regardless of the method that you use, Amazon S3 stores the notification configuration as XML in the *notification* subresource that's associated with a bucket. For information about bucket subresources, see [General purpose buckets configuration options](UsingBucket.md#bucket-config-options-intro). **Note** If you have multiple failed event notifications due to deleted destinations you may receive the **Unable to validate the following destination configurations** when trying to delete them. You can resolve this in the S3 console by deleting all the failed notifications at the same time. **Topics** + [ # Granting permissions to publish event notification messages to a destination ](grant-destinations-permissions-to-s3.md) + [ # Enabling and configuring event notifications using the Amazon S3 console ](enable-event-notifications.md) + [ ## Configuring event notifications programmatically ](#event-notification-configuration) + [ # Walkthrough: Configuring a bucket for notifications (SNS topic or SQS queue) ](ways-to-add-notification-config-to-bucket.md) + [ # Configuring event notifications using object key name filtering ](notification-how-to-filtering.md) + [ # Event message structure ](notification-content-structure.md) # Granting permissions to publish event notification messages to a destination Granting permissions You must grant the Amazon S3 principal the necessary permissions to call the relevant API to publish messages to an SNS topic, an SQS queue, or a Lambda function. This is so that Amazon S3 can publish event notification messages to a destination. To troubleshoot publishing event notification messages to a destination, see [ Troubleshoot to publish Amazon S3 event notifications to an Amazon Simple Notification Service topic ](https://repost.aws/knowledge-center/sns-not-receiving-s3-event-notifications). **Topics** + [ ## Granting permissions to invoke an AWS Lambda function ](#grant-lambda-invoke-permission-to-s3) + [ ## Granting permissions to publish messages to an SNS topic or an SQS queue ](#grant-sns-sqs-permission-for-s3) ## Granting permissions to invoke an AWS Lambda function Amazon S3 publishes event messages to AWS Lambda by invoking a Lambda function and providing the event message as an argument. When you use the Amazon S3 console to configure event notifications on an Amazon S3 bucket for a Lambda function, the console sets up the necessary permissions on the Lambda function. This is so that Amazon S3 has permissions to invoke the function from the bucket. For more information, see [Enabling and configuring event notifications using the Amazon S3 console](enable-event-notifications.md). You can also grant Amazon S3 permissions from AWS Lambda to invoke your Lambda function. For more information, see [Tutorial: Using AWS Lambda with Amazon S3](https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html) in the *AWS Lambda Developer Guide*. ## Granting permissions to publish messages to an SNS topic or an SQS queue To grant Amazon S3 permissions to publish messages to the SNS topic or SQS queue, attach an AWS Identity and Access Management (IAM) policy to the destination SNS topic or SQS queue. For an example of how to attach a policy to an SNS topic or an SQS queue, see [Walkthrough: Configuring a bucket for notifications (SNS topic or SQS queue)](ways-to-add-notification-config-to-bucket.md). For more information about permissions, see the following topics: + [Example cases for Amazon SNS access control](https://docs.aws.amazon.com/sns/latest/dg/AccessPolicyLanguage_UseCases_Sns.html) in the *Amazon Simple Notification Service Developer Guide* + [Identity and access management in Amazon SQS](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/UsingIAM.html) in the *Amazon Simple Queue Service Developer Guide* ### IAM policy for a destination SNS topic The following is an example of an AWS Identity and Access Management (IAM) policy that you attach to the destination SNS topic. For instructions on how to use this policy to set up a destination Amazon SNS topic for event notifications, see [Walkthrough: Configuring a bucket for notifications (SNS topic or SQS queue)](ways-to-add-notification-config-to-bucket.md). ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Id": "example-ID", "Statement": [ { "Sid": "Example SNS topic policy", "Effect": "Allow", "Principal": { "Service": "s3.amazonaws.com" }, "Action": [ "SNS:Publish" ], "Resource": "arn:aws:sns:us-east-1:111122223333:example-sns-topic", "Condition": { "ArnEquals": { "aws:SourceArn": "arn:aws:s3:::amzn-s3-demo-bucket" }, "StringEquals": { "aws:SourceAccount": "bucket-owner-123456789012" } } } ] } ``` ------ ### IAM policy for a destination SQS queue The following is an example of an IAM policy that you attach to the destination SQS queue. For instructions on how to use this policy to set up a destination Amazon SQS queue for event notifications, see [Walkthrough: Configuring a bucket for notifications (SNS topic or SQS queue)](ways-to-add-notification-config-to-bucket.md). To use this policy, you must update the Amazon SQS queue ARN, bucket name, and bucket owner's AWS account ID. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Id": "example-ID", "Statement": [ { "Sid": "example-statement-ID", "Effect": "Allow", "Principal": { "Service": "s3.amazonaws.com" }, "Action": [ "SQS:SendMessage" ], "Resource": "arn:aws:sqs:us-east-1:111122223333:queue-name", "Condition": { "ArnLike": { "aws:SourceArn": "arn:aws:s3:*:*:amzn-s3-demo-bucket" }, "StringEquals": { "aws:SourceAccount": "bucket-owner-123456789012" } } } ] } ``` ------ For both the Amazon SNS and Amazon SQS IAM policies, you can specify the `StringLike` condition in the policy instead of the `ArnLike` condition. When `ArnLike` is used, the partition, service, account ID, resource type, and partial resource ID portions of the ARN must exactly match to the ARN in the request context. Only the Region and resource path allow partial matching. When `StringLike` is used instead of `ArnLike`, matching ignores the ARN structure and allows partial matching, regardless of the portion that's replaced by the wildcard character. For more information, see [IAM JSON policy elements](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition_operators.html) in the *IAM User Guide*. ``` "Condition": {        "StringLike": { "aws:SourceArn": "arn:aws:s3:*:*:amzn-s3-demo-bucket" } } ``` ### AWS KMS key policy If the SQS queue or SNS topics are encrypted with an AWS Key Management Service (AWS KMS) customer managed key, you must grant the Amazon S3 service principal permission to work with the encrypted topics or queue. To grant the Amazon S3 service principal permission, add the following statement to the key policy for the customer managed key. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Id": "example-ID", "Statement": [ { "Sid": "example-statement-ID", "Effect": "Allow", "Principal": { "Service": "s3.amazonaws.com" }, "Action": [ "kms:GenerateDataKey", "kms:Decrypt" ], "Resource": "*" } ] } ``` ------ For more information about AWS KMS key policies, see [Using key policies in AWS KMS](https://docs.aws.amazon.com/kms/latest/developerguide/key-policies.html) in the *AWS Key Management Service Developer Guide*. For more information about using server-side encryption with AWS KMS for Amazon SQS and Amazon SNS, see the following: + [Key management](https://docs.aws.amazon.com/sns/latest/dg/sns-key-management.html) in the *Amazon Simple Notification Service Developer Guide*. + [Key management](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-key-management.html) in the *Amazon Simple Queue Service Developer Guide*. + [Encrypting messages published to Amazon SNS with AWS KMS](https://aws.amazon.com/blogs/compute/encrypting-messages-published-to-amazon-sns-with-aws-kms/) in the *AWS Compute Blog*. # Enabling and configuring event notifications using the Amazon S3 console Enabling notifications in the S3 console You can enable certain Amazon S3 general purpose bucket events to send a notification message to a destination whenever those events occur. This section explains how to use the Amazon S3 console to enable event notifications. For information about how to use event notifications with the AWS SDKs and the Amazon S3 REST APIs, see [Configuring event notifications programmatically](how-to-enable-disable-notification-intro.md#event-notification-configuration). **Prerequisites**: Before you can enable event notifications for your bucket, you must set up one of the destination types and then configure permissions. For more information, see [Supported event destinations](notification-how-to-event-types-and-destinations.md#supported-notification-destinations) and [Granting permissions to publish event notification messages to a destination](grant-destinations-permissions-to-s3.md). **Note** Amazon Simple Queue Service FIFO (First-In-First-Out) queues aren't supported as an Amazon S3 event notification destination. To send a notification for an Amazon S3 event to an Amazon SQS FIFO queue, you can use Amazon EventBridge. For more information, see [Enabling Amazon EventBridge](enable-event-notifications-eventbridge.md). **Topics** + [ ## Enabling Amazon SNS, Amazon SQS, or Lambda notifications using the Amazon S3 console ](#enable-event-notifications-sns-sqs-lam) ## Enabling Amazon SNS, Amazon SQS, or Lambda notifications using the Amazon S3 console **To enable and configure event notifications for an S3 bucket** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **General purpose buckets**. 1. In the buckets list, choose the name of the bucket that you want to enable events for. 1. Choose **Properties**. 1. Navigate to the **Event Notifications** section and choose **Create event notification**. 1. In the **General configuration** section, specify descriptive event name for your event notification. Optionally, you can also specify a prefix and a suffix to limit the notifications to objects with keys ending in the specified characters. 1. Enter a description for the **Event name**. If you don't enter a name, a globally unique identifier (GUID) is generated and used for the name. 1. (Optional) To filter event notifications by prefix, enter a **Prefix**. For example, you can set up a prefix filter so that you receive notifications only when files are added to a specific folder (for example, `images/`). 1. (Optional) To filter event notifications by suffix, enter a **Suffix**. For more information, see [Configuring event notifications using object key name filtering](notification-how-to-filtering.md). 1. In the **Event types** section, select one or more event types that you want to receive notifications for. For a list of the different event types, see [Supported event types for SQS, SNS, and Lambda](notification-how-to-event-types-and-destinations.md#supported-notification-event-types). 1. In the **Destination** section, choose the event notification destination. **Note** Before you can publish event notifications, you must grant the Amazon S3 principal the necessary permissions to call the relevant API. This is so that it can publish notifications to a Lambda function, SNS topic, or SQS queue. 1. Select the destination type: **Lambda Function**, **SNS Topic**, or **SQS Queue**. 1. After you choose your destination type, choose a function, topic, or queue from the list. 1. Or, if you prefer to specify an Amazon Resource Name (ARN), select **Enter ARN** and enter the ARN. For more information, see [Supported event destinations](notification-how-to-event-types-and-destinations.md#supported-notification-destinations). 1. Choose **Save changes**, and Amazon S3 sends a test message to the event notification destination. ## Configuring event notifications programmatically By default, notifications aren't enabled for any type of event. Therefore, the *notification* subresource initially stores an empty configuration. ``` ``` To enable notifications for events of specific types, you replace the XML with the appropriate configuration that identifies the event types you want Amazon S3 to publish and the destination where you want the events published. For each destination, you add a corresponding XML configuration. **To publish event messages to an SQS queue** To set an SQS queue as the notification destination for one or more event types, add the `QueueConfiguration`. ``` optional-id-string sqs-queue-arn event-type event-type ... ... ``` **To publish event messages to an SNS topic** To set an SNS topic as the notification destination for specific event types, add the `TopicConfiguration`. ``` optional-id-string sns-topic-arn event-type event-type ... ... ``` **To invoke the AWS Lambda function and provide an event message as an argument** To set a Lambda function as the notification destination for specific event types, add the `CloudFunctionConfiguration`. ```    optional-id-string    cloud-function-arn         event-type       event-type       ...   ... ``` **To remove all notifications configured on a bucket** To remove all notifications configured on a bucket, save an empty `` element in the *notification* subresource. When Amazon S3 detects an event of the specific type, it publishes a message with the event information. For more information, see [Event message structure](notification-content-structure.md). For more information about configuring event notifications, see the following topics: + [Walkthrough: Configuring a bucket for notifications (SNS topic or SQS queue)](ways-to-add-notification-config-to-bucket.md). + [Configuring event notifications using object key name filtering](notification-how-to-filtering.md) # Walkthrough: Configuring a bucket for notifications (SNS topic or SQS queue) Walkthrough: Configuring SNS or SQS You can receive Amazon S3 notifications using Amazon Simple Notification Service (Amazon SNS) or Amazon Simple Queue Service (Amazon SQS). In this walkthrough, you add a notification configuration to your bucket using an Amazon SNS topic and an Amazon SQS queue. **Note** Amazon Simple Queue Service FIFO (First-In-First-Out) queues aren't supported as an Amazon S3 event notification destination. To send a notification for an Amazon S3 event to an Amazon SQS FIFO queue, you can use Amazon EventBridge. For more information, see [Enabling Amazon EventBridge](enable-event-notifications-eventbridge.md). **Topics** + [ ## Walkthrough summary ](#notification-walkthrough-summary) + [ ## Step 1: Create an Amazon SQS queue ](#step1-create-sqs-queue-for-notification) + [ ## Step 2: Create an Amazon SNS topic ](#step1-create-sns-topic-for-notification) + [ ## Step 3: Add a notification configuration to your bucket ](#step2-enable-notification) + [ ## Step 4: Test the setup ](#notification-walkthrough-1-test) ## Walkthrough summary This walkthrough helps you do the following: + Publish events of the `s3:ObjectCreated:*` type to an Amazon SQS queue. + Publish events of the `s3:ReducedRedundancyLostObject` type to an Amazon SNS topic. For information about notification configuration, see [Using Amazon SQS, Amazon SNS, and Lambda](how-to-enable-disable-notification-intro.md). You can do all these steps using the console, without writing any code. In addition, code examples using AWS SDKs for Java and .NET are also provided to help you add notification configurations programmatically. The procedure includes the following steps: 1. Create an Amazon SQS queue. Using the Amazon SQS console, create an SQS queue. You can access any messages Amazon S3 sends to the queue programmatically. But, for this walkthrough, you verify notification messages in the console. You attach an access policy to the queue to grant Amazon S3 permission to post messages. 1. Create an Amazon SNS topic. Using the Amazon SNS console, create an SNS topic and subscribe to the topic. That way, any events posted to it are delivered to you. You specify email as the communications protocol. After you create a topic, Amazon SNS sends an email. You use the link in the email to confirm the topic subscription. You attach an access policy to the topic to grant Amazon S3 permission to post messages. 1. Add notification configuration to a bucket. ## Step 1: Create an Amazon SQS queue Follow the steps to create and subscribe to an Amazon Simple Queue Service (Amazon SQS) queue. 1. Using the Amazon SQS console, create a queue. For instructions, see [Getting Started with Amazon SQS](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-getting-started.html) in the *Amazon Simple Queue Service Developer Guide*. 1. Replace the access policy that's attached to the queue with the following policy. 1. In the Amazon SQS console, in the **Queues** list, choose the queue name. 1. On the **Access policy** tab, choose **Edit**. 1. Replace the access policy that's attached to the queue. In it, provide your Amazon SQS ARN, source bucket name, and bucket owner account ID. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Id": "example-ID", "Statement": [ { "Sid": "example-statement-ID", "Effect": "Allow", "Principal": { "Service": "s3.amazonaws.com" }, "Action": [ "SQS:SendMessage" ], "Resource": "arn:aws:sqs:us-west-2:111122223333:s3-notification-queue", "Condition": { "ArnLike": { "aws:SourceArn": "arn:aws:s3:*:*:awsexamplebucket1" }, "StringEquals": { "aws:SourceAccount": "bucket-owner-123456789012" } } } ] } ``` ------ 1. Choose **Save**. 1. (Optional) If the Amazon SQS queue or the Amazon SNS topic is server-side encryption enabled with AWS Key Management Service (AWS KMS), add the following policy to the associated symmetric encryption customer managed key. You must add the policy to a customer managed key because you cannot modify the AWS managed key for Amazon SQS or Amazon SNS. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Id": "example-ID", "Statement": [ { "Sid": "example-statement-ID", "Effect": "Allow", "Principal": { "Service": "s3.amazonaws.com" }, "Action": [ "kms:GenerateDataKey", "kms:Decrypt" ], "Resource": "*" } ] } ``` ------ For more information about using SSE for Amazon SQS and Amazon SNS with AWS KMS, see the following: + [Key management](https://docs.aws.amazon.com/sns/latest/dg/sns-key-management.html) in the *Amazon Simple Notification Service Developer Guide*. + [Key management](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-key-management.html) in the *Amazon Simple Queue Service Developer Guide*. 1. Note the queue ARN. The SQS queue that you created is another resource in your AWS account. It has a unique Amazon Resource Name (ARN). You need this ARN in the next step. The ARN is of the following format: ``` arn:aws:sqs:aws-region:account-id:queue-name ``` ## Step 2: Create an Amazon SNS topic Follow the steps to create and subscribe to an Amazon SNS topic. 1. Using Amazon SNS console, create a topic. For instructions, see [Creating an Amazon SNS topic](https://docs.aws.amazon.com/sns/latest/dg/CreateTopic.html) in the *Amazon Simple Notification Service Developer Guide*. 1. Subscribe to the topic. For this exercise, use email as the communications protocol. For instructions, see [Subscribing to an Amazon SNS topic](https://docs.aws.amazon.com/sns/latest/dg/sns-create-subscribe-endpoint-to-topic.html) in the *Amazon Simple Notification Service Developer Guide*. You get an email requesting you to confirm your subscription to the topic. Confirm the subscription. 1. Replace the access policy attached to the topic with the following policy. In it, provide your SNS topic ARN, bucket name, and bucket owner's account ID. 1. Note the topic ARN. The SNS topic you created is another resource in your AWS account, and it has a unique ARN. You will need this ARN in the next step. The ARN will be of the following format: ``` arn:aws:sns:aws-region:account-id:topic-name ``` ## Step 3: Add a notification configuration to your bucket You can enable bucket notifications either by using the Amazon S3 console or programmatically by using AWS SDKs. Choose any one of the options to configure notifications on your bucket. This section provides code examples using the AWS SDKs for Java and .NET. ### Option A: Enable notifications on a bucket using the console Using the Amazon S3 console, add a notification configuration requesting Amazon S3 to do the following: + Publish events of the **All object create events** type to your Amazon SQS queue. + Publish events of the **Object in RRS lost** type to your Amazon SNS topic. After you save the notification configuration, Amazon S3 posts a test message, which you get via email. For instructions, see [Enabling and configuring event notifications using the Amazon S3 console](enable-event-notifications.md). ### Option B: Enable notifications on a bucket using the AWS SDKs ------ #### [ .NET ] The following C\$1 code example provides a complete code listing that adds a notification configuration to a bucket. You must update the code and provide your bucket name and SNS topic ARN. For information about setting up and running the code examples, see [Getting Started with the AWS SDK for .NET](https://docs.aws.amazon.com/sdk-for-net/latest/developer-guide/net-dg-setup.html) in the *AWS SDK for .NET Developer Guide*. ``` using Amazon; using Amazon.S3; using Amazon.S3.Model; using System; using System.Collections.Generic; using System.Threading.Tasks; namespace Amazon.DocSamples.S3 { class EnableNotificationsTest { private const string bucketName = "*** bucket name ***"; private const string snsTopic = "*** SNS topic ARN ***"; private const string sqsQueue = "*** SQS topic ARN ***"; // Specify your bucket region (an example region is shown). private static readonly RegionEndpoint bucketRegion = RegionEndpoint.USWest2; private static IAmazonS3 client; public static void Main() { client = new AmazonS3Client(bucketRegion); EnableNotificationAsync().Wait(); } static async Task EnableNotificationAsync() { try { PutBucketNotificationRequest request = new PutBucketNotificationRequest { BucketName = bucketName }; TopicConfiguration c = new TopicConfiguration { Events = new List { EventType.ObjectCreatedCopy }, Topic = snsTopic }; request.TopicConfigurations = new List(); request.TopicConfigurations.Add(c); request.QueueConfigurations = new List(); request.QueueConfigurations.Add(new QueueConfiguration() { Events = new List { EventType.ObjectCreatedPut }, Queue = sqsQueue }); PutBucketNotificationResponse response = await client.PutBucketNotificationAsync(request); } catch (AmazonS3Exception e) { Console.WriteLine("Error encountered on server. Message:'{0}' ", e.Message); } catch (Exception e) { Console.WriteLine("Unknown error encountered on server. Message:'{0}' ", e.Message); } } } } ``` ------ #### [ Java ] For examples of how to configure bucket notifications with the AWS SDK for Java, see [Process S3 event notifications](https://docs.aws.amazon.com/AmazonS3/latest/API/s3_example_s3_Scenario_ProcessS3EventNotification_section.html) in the *Amazon S3 API Reference*. ------ ## Step 4: Test the setup Now, you can test the setup by uploading an object to your bucket and verifying the event notification in the Amazon SQS console. For instructions, see [Receiving a Message](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-getting-started.htmlReceiveMessage.html) in the *Amazon Simple Queue Service Developer Guide "Getting Started" section*. # Configuring event notifications using object key name filtering Configuring notifications using object key name filtering When configuring an Amazon S3 event notification, you must specify which supported Amazon S3 event types cause Amazon S3 to send the notification. If an event type that you didn't specify occurs in your S3 bucket, Amazon S3 doesn't send the notification. You can configure notifications to be filtered by the prefix and suffix of the key name of objects. For example, you can set up a configuration where you're sent a notification only when image files with a "`.jpg`" file name extension are added to a bucket. Or, you can have a configuration that delivers a notification to an Amazon SNS topic when an object with the prefix "`images/`" is added to the bucket, while having notifications for objects with a "`logs/`" prefix in the same bucket delivered to an AWS Lambda function. **Note** A wildcard character ("\$1") can't be used in filters as a prefix or suffix. If your prefix or suffix contains a space, you must replace it with the "\$1" character. If you use any other special characters in the value of the prefix or suffix, you must enter them in [URL-encoded (percent-encoded) format](https://en.wikipedia.org/wiki/Percent-encoding). For a complete list of special characters that must be converted to URL-encoded format when used in a prefix or suffix for event notifications, see [Safe characters](object-keys.md#object-key-guidelines-safe-characters). You can set up notification configurations that use object key name filtering in the Amazon S3 console. You can do so by using Amazon S3 APIs through the AWS SDKs or the REST APIs directly. For information about using the console UI to set a notification configuration on a bucket, see [Enabling and configuring event notifications using the Amazon S3 console](enable-event-notifications.md). Amazon S3 stores the notification configuration as XML in the *notification* subresource associated with a bucket as described in [Using Amazon SQS, Amazon SNS, and Lambda](how-to-enable-disable-notification-intro.md). You use the `Filter` XML structure to define the rules for notifications to be filtered by the prefix or suffix of an object key name. For information about the `Filter` XML structure, see [PUT Bucket notification](https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTnotification.html) in the *Amazon Simple Storage Service API Reference*. Notification configurations that use `Filter` cannot define filtering rules with overlapping prefixes, overlapping suffixes, or prefix and suffix overlapping. The following sections have examples of valid notification configurations with object key name filtering. They also contain examples of notification configurations that are not valid because of prefix and suffix overlapping. **Topics** + [ ## Examples of valid notification configurations with object key name filtering ](#notification-how-to-filtering-example-valid) + [ ## Examples of notification configurations with invalid prefix and suffix overlapping ](#notification-how-to-filtering-examples-invalid) ## Examples of valid notification configurations with object key name filtering The following notification configuration contains a queue configuration identifying an Amazon SQS queue for Amazon S3 to publish events to of the `s3:ObjectCreated:Put` type. The events are published whenever an object that has a prefix of `images/` and a `jpg` suffix is PUT to a bucket. ``` 1 prefix images/ suffix jpg arn:aws:sqs:us-west-2:444455556666:s3notificationqueue s3:ObjectCreated:Put ``` The following notification configuration has multiple non-overlapping prefixes. The configuration defines that notifications for PUT requests in the `images/` folder go to queue-A, while notifications for PUT requests in the `logs/` folder go to queue-B. ``` 1 prefix images/ arn:aws:sqs:us-west-2:444455556666:sqs-queue-A s3:ObjectCreated:Put 2 prefix logs/ arn:aws:sqs:us-west-2:444455556666:sqs-queue-B s3:ObjectCreated:Put ``` The following notification configuration has multiple non-overlapping suffixes. The configuration defines that all `.jpg` images newly added to the bucket are processed by Lambda cloud-function-A, and all newly added `.png` images are processed by cloud-function-B. The `.png` and `.jpg` suffixes aren't overlapping even though they have the same last letter. If a given string can end with both suffixes, the two suffixes are considered overlapping. A string can't end with both `.png` and `.jpg`, so the suffixes in the example configuration aren't overlapping suffixes. ``` 1 suffix .jpg arn:aws:lambda:us-west-2:444455556666:cloud-function-A s3:ObjectCreated:Put 2 suffix .png arn:aws:lambda:us-west-2:444455556666:cloud-function-B s3:ObjectCreated:Put ``` Your notification configurations that use `Filter` can't define filtering rules with overlapping prefixes for the same event types. They can only do so, if the overlapping prefixes that are used with suffixes that don't overlap. The following example configuration shows how objects created with a common prefix but non-overlapping suffixes can be delivered to different destinations. ``` 1 prefix images suffix .jpg arn:aws:lambda:us-west-2:444455556666:cloud-function-A s3:ObjectCreated:Put 2 prefix images suffix .png arn:aws:lambda:us-west-2:444455556666:cloud-function-B s3:ObjectCreated:Put ``` ## Examples of notification configurations with invalid prefix and suffix overlapping For the most part, your notification configurations that use `Filter` can't define filtering rules with overlapping prefixes, overlapping suffixes, or overlapping combinations of prefixes and suffixes for the same event types. You can have overlapping prefixes as long as the suffixes don't overlap. For an example, see [Configuring event notifications using object key name filtering](#notification-how-to-filtering). You can use overlapping object key name filters with different event types. For example, you can create a notification configuration that uses the prefix `image/` for the `ObjectCreated:Put` event type and the prefix `image/` for the `ObjectRemoved:*` event type. You get an error if you try to save a notification configuration that has invalid overlapping name filters for the same event types when using the Amazon S3 console or API. This section shows examples of notification configurations that aren't valid because of overlapping name filters. Any existing notification configuration rule is assumed to have a default prefix and suffix that match any other prefix and suffix, respectively. The following notification configuration isn't valid because it has overlapping prefixes. Specifically, the root prefix overlaps with any other prefix. The same thing is true if you use a suffix instead of a prefix in this example. The root suffix overlaps with any other suffix. ``` arn:aws:sns:us-west-2:444455556666:sns-notification-one s3:ObjectCreated:* arn:aws:sns:us-west-2:444455556666:sns-notification-two s3:ObjectCreated:* prefix images ``` The following notification configuration isn't valid because it has overlapping suffixes. If a given string can end with both suffixes, the two suffixes are considered overlapping. A string can end with `jpg` and `pg`. So, the suffixes overlap. The same is true for prefixes. If a given string can begin with both prefixes, the two prefixes are considered overlapping. ``` arn:aws:sns:us-west-2:444455556666:sns-topic-one s3:ObjectCreated:* suffix jpg arn:aws:sns:us-west-2:444455556666:sns-topic-two s3:ObjectCreated:Put suffix pg arn:aws:sns:us-west-2:444455556666:sns-topic-one s3:ObjectCreated:* prefix images suffix jpg arn:aws:sns:us-west-2:444455556666:sns-topic-two s3:ObjectCreated:Put suffix jpg ``` # Event message structure The notification message that Amazon S3 sends to publish an event is in the JSON format. For a general overview and instructions on configuring event notifications, see [Amazon S3 Event Notifications](EventNotifications.md). This example shows *version 2.1* of the event notification JSON structure. Amazon S3 uses *versions 2.1*, *2.2*, and *2.3* of this event structure. Amazon S3 uses version 2.2 for cross-Region replication event notifications. It uses version 2.3 for S3 Lifecycle, S3 Intelligent-Tiering, object ACL, object tagging, and object restoration delete events. These versions contain extra information specific to these operations. Versions 2.2 and 2.3 are otherwise compatible with version 2.1, which Amazon S3 currently uses for all other event notification types. ``` { "Records":[ { "eventVersion":"2.1", "eventSource":"aws:s3", "awsRegion":"us-west-2", "eventTime":"The time, in ISO-8601 format (for example, 1970-01-01T00:00:00.000Z) when Amazon S3 finished processing the request", "eventName":"The event type", "userIdentity":{ "principalId":"The unique ID of the IAM resource that caused the event" }, "requestParameters":{ "sourceIPAddress":"The IP address where the request came from" }, "responseElements":{ "x-amz-request-id":"The Amazon S3 generated request ID", "x-amz-id-2":"The Amazon S3 host that processed the request" }, "s3":{ "s3SchemaVersion":"1.0", "configurationId":"The ID found in the bucket notification configuration", "bucket":{ "name":"The name of the bucket, for example, amzn-s3-demo-bucket", "ownerIdentity":{ "principalId":"The Amazon retail customer ID of the bucket owner" }, "arn":"The bucket Amazon Resource Name (ARN)" }, "object":{ "key":"The object key name", "size":"The object size in bytes (as a number)", "eTag":"The object entity tag (ETag)", "versionId":"The object version if the bucket is versioning-enabled; null or not present if the bucket isn't versioning-enabled", "sequencer": "A string representation of a hexadecimal value used to determine event sequence; only used with PUT and DELETE requests" } }, "glacierEventData": { "restoreEventData": { "lifecycleRestorationExpiryTime": "The time, in ISO-8601 format (for example, 1970-01-01T00:00:00.000Z), when the temporary copy of the restored object expires", "lifecycleRestoreStorageClass": "The source storage class for restored objects" } } } ] } ``` Note the following about the event message structure: + The `eventVersion` key value contains a major and minor version in the form `major`.`minor`. The major version is incremented if Amazon S3 makes a change to the event structure that's not backward compatible. This includes removing a JSON field that's already present or changing how the contents of a field are represented (for example, a date format). The minor version is incremented if Amazon S3 adds new fields to the event structure. This might occur if new information is provided for some or all existing events. This might also occur if new information is provided only for newly introduced event types. To stay compatible with new minor versions of the event structure, we recommend that your applications ignore new fields. If new event types are introduced, but the structure of the event is otherwise unmodified, the event version doesn't change. To ensure that your applications can parse the event structure correctly, we recommend that you do an equal-to comparison on the major version number. To ensure that the fields that are expected by your application are present, we also recommend doing a greater-than-or-equal-to comparison on the minor version. + The `eventName` key value references the list of [event notification types](https://docs.aws.amazon.com/AmazonS3/latest/userguide/notification-how-to-event-types-and-destinations.html) but doesn't contain the `s3:` prefix. + The `userIdentity` key value references the unique ID of the AWS Identity and Access Management (IAM) resource (a user, role, group, and so on) that caused the event. For a definition of each IAM identification prefix (for example, AIDA, AROA, AGPA) and information about how to get the unique identifier, see [Unique identifiers](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_identifiers.html##identifiers-unique-ids) in the *IAM User Guide*. + The `responseElements` key value is useful if you want to trace a request by following up with AWS Support. Both `x-amz-request-id` and `x-amz-id-2` help Amazon S3 trace an individual request. These values are the same as those that Amazon S3 returns in the response to the request that initiates the events. Therefore, you can use these values to match the event to the request. + The `s3` key value provides information about the bucket and object involved in the event. The object key name value is URL encoded. For example, `red flower.jpg` becomes `red+flower.jpg`. (Amazon S3 returns "`application/x-www-form-urlencoded`" as the content type in the response.) The `ownerIdentity` key value corresponds to the Amazon retail (Amazon.com) customer ID of the bucket owner. This ID value is no longer used and is maintained only for backward compatibility. + The `sequencer` key value provides a way to determine the sequence of events. Event notifications aren't guaranteed to arrive in the same order that the events occurred. However, notifications from events that create objects (`PUT` requests) and delete objects contain a `sequencer`. You can use this value to determine the order of events for a given object key. If you compare the `sequencer` strings from two event notifications on the same object key, the event notification with the greater `sequencer` hexadecimal value is the event that occurred later. If you're using event notifications to maintain a separate database or index of your Amazon S3 objects, we recommend that you compare and store the `sequencer` values as you process each event notification. Note the following: + You can't use the `sequencer` key value to determine the order for events on different object keys. + The `sequencer` strings can be of different lengths. So, to compare these values, first left-pad the shorter value with zeros, and then do a lexicographical comparison. + The `glacierEventData` key value is only visible for `s3:ObjectRestore:Completed` events. + The `restoreEventData` key value contains attributes that are related to your restore request. + The `replicationEventData` key value is only visible for replication events. + The `intelligentTieringEventData` key value is only visible for S3 Intelligent-Tiering events. + The `lifecycleEventData` key value is only visible for S3 Lifecycle transition events. ## Example messages The following are examples of Amazon S3 event notification messages. **Amazon S3 test message** After you configure an event notification on a bucket, Amazon S3 sends the following test message. ``` 1. { 2. "Service":"Amazon S3", 3. "Event":"s3:TestEvent", 4. "Time":"2014-10-13T15:57:02.089Z", 5. "Bucket":"amzn-s3-demo-bucket", 6. "RequestId":"5582815E1AEA5ADF", 7. "HostId":"8cLeGAmw098X5cv4Zkwcmo8vvZa3eH3eKxsPzbB9wrR+YstdA6Knx4Ip8EXAMPLE" 8. } ``` **Note** The `s3:TestEvent` message uses a different format than regular S3 event notifications. Unlike other event notifications that use the `Records` array structure shown earlier, the test event uses a simplified format with direct fields. When implementing event handling, ensure your code can distinguish between and properly handle both message formats. **Example message when an object is created using a `PUT` request** The following is an example of a message that Amazon S3 sends to publish an `s3:ObjectCreated:Put` event. ``` 1. { 2. "Records":[ 3. { 4. "eventVersion":"2.1", 5. "eventSource":"aws:s3", 6. "awsRegion":"us-west-2", 7. "eventTime":"1970-01-01T00:00:00.000Z", 8. "eventName":"ObjectCreated:Put", 9. "userIdentity":{ 10. "principalId":"AIDAJDPLRKLG7UEXAMPLE" 11. }, 12. "requestParameters":{ 13. "sourceIPAddress":"172.16.0.1" 14. }, 15. "responseElements":{ 16. "x-amz-request-id":"C3D13FE58DE4C810", 17. "x-amz-id-2":"FMyUVURIY8/IgAtTv8xRjskZQpcIZ9KG4V5Wp6S7S/JRWeUWerMUE5JgHvANOjpD" 18. }, 19. "s3":{ 20. "s3SchemaVersion":"1.0", 21. "configurationId":"testConfigRule", 22. "bucket":{ 23. "name":"amzn-s3-demo-bucket", 24. "ownerIdentity":{ 25. "principalId":"A3NL1KOZZKExample" 26. }, 27. "arn":"arn:aws:s3:::amzn-s3-demo-bucket" 28. }, 29. "object":{ 30. "key":"HappyFace.jpg", 31. "size":1024, 32. "eTag":"d41d8cd98f00b204e9800998ecf8427e", 33. "versionId":"096fKKXTRTtl3on89fVO.nfljtsv6qko", 34. "sequencer":"0055AED6DCD90281E5" 35. } 36. } 37. } 38. ] 39. } ```