# Working with S3 Lifecycle for directory buckets S3 Lifecycle helps you store objects in S3 Express One Zone in directory buckets cost effectively by deleting expired objects on your behalf. To manage the lifecycle of your objects, create an S3 Lifecycle configuration for your directory bucket. An S3 Lifecycle configuration is a set of rules that define actions that Amazon S3 applies to a group of objects. You can set an Amazon S3 Lifecycle configuration on a directory bucket by using the AWS Command Line Interface (AWS CLI), the AWS SDKs, the Amazon S3 REST API and AWS CloudFormation. In your lifecycle configuration, you use rules to define actions that you want Amazon S3 to take on your objects. For objects stored in directory buckets, you can create lifecycle rules to expire objects as they age. You can also create lifecycle rules to delete incomplete multipart uploads in directory buckets at a daily frequency. When you add a Lifecycle configuration to a bucket, the configuration rules apply to both existing objects and objects that you add later. For example, if you add a Lifecycle configuration rule today with an expiration action that causes objects with a specific prefix to expire 30 days after creation, S3 will queue for removal any existing objects that are more than 30 days old and that have the specified prefix. ## How S3 Lifecycle for directory buckets is different For objects in directory buckets, you can create lifecycle rules to expire objects and delete incomplete multipart uploads. However, S3 Lifecycle for directory buckets doesn't support transition actions between storage classes. **CreateSession** Lifecycle uses public `DeleteObject` and `DeleteObjects` API operations to expire objects in directory buckets. To use these API operations, S3 Lifecycle will use the `CreateSession` API to establish temporary security credentials to access the objects in the directory buckets. For more information, see [`CreateSession`in the *Amazon S3 API Reference*.](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateSession.html) If you have an active policy that denies delete permissions to the lifecycle principal, this will prevent you from allowing S3 Lifecycle to delete objects on your behalf. ### Using a bucket policy to Grant permissions to the S3 Lifecycle service principal The following bucket policy grants the S3 Lifecycle service principal permission to create sessions for performing operations such as `DeleteObject` and `DeleteObjects`. When no session mode is specified in a `CreateSession` request, the session is created with the maximum allowable privilege by the permissions in (attempting `ReadWrite` first, then `ReadOnly` if `ReadWrite` is not permitted). However, `ReadOnly` sessions are insufficient for lifecycle operations that modify or delete objects. Therefore, this example explicitly requires a `ReadWrite` session mode by using the `s3express:SessionMode` condition key. **Example – Bucket policy to allow `CreateSession` calls with an explicit `ReadWrite` session mode for lifecycle operations** ``` { "Version":"2008-10-17", "Statement":[ { "Effect":"Allow", "Principal": { "Service":"lifecycle.s3.amazonaws.com" }, "Action":"s3express:CreateSession", "Condition": { "StringEquals": { "s3express:SessionMode": "ReadWrite" } }, "Resource":"arn:aws:s3express:us-east-2:412345678921:bucket/amzn-s3-demo-bucket--use2-az2--x-s3" } ] } ``` ### Monitoring lifecycle rules For objects stored in directory buckets, S3 Lifecycle generates AWS CloudTrail management and data event logs. For more information, see [CloudTrail log file examples for S3 Express One Zone](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-express-log-files.html). For more information about creating lifecycle configurations and troubleshooting S3 Lifecycle related issues, see the following topics: **Topics** # Creating and managing a Lifecycle configuration for your directory bucket You can create a lifecycle configuration for directory buckets by using the AWS Command Line Interface (AWS CLI), AWS SDKs and REST APIs. ## Using the AWS CLI You can use the following AWS CLI commands to manage S3 Lifecycle configurations: + `put-bucket-lifecycle-configuration` + `get-bucket-lifecycle-configuration` + `delete-bucket-lifecycle` For instructions on setting up the AWS CLI, see [Developing with Amazon S3 using the AWS CLI](https://docs.aws.amazon.com/AmazonS3/latest/API/setup-aws-cli.html) in the *Amazon S3 API Reference*. The Amazon S3 Lifecycle configuration is an XML file. But when you're using the AWS CLI, you cannot specify the XML format. You must specify the JSON format instead. The following are example XML lifecycle configurations and the equivalent JSON configurations that you can specify in an AWS CLIcommand. The following AWS CLI example puts a lifecycle configuration policy on a directory bucket. This policy specifies that all objects that have the flagged prefix (myprefix) and are the defined object size expire after 7 days. To use this example, replace each *user input placeholder* with your own information. Save the lifecycle configuration policy to a JSON file. In this example, the file is named lifecycle1.json. **Example** ``` { "Rules": [ { "Expiration": { "Days": 7 }, "ID": "Lifecycle expiration rule", "Filter": { "And": { "Prefix": "myprefix/", "ObjectSizeGreaterThan": 500, "ObjectSizeLessThan": 64000 } }, "Status": "Enabled" } ] } ``` Submit the JSON file as part of the `put-bucket-lifecycle-configuration` CLI command. To use this command, replace each *user input placeholder* with your own information. ``` aws s3api put-bucket-lifecycle-configuration --region us-west-2 --profile default --bucket amzn-s3-demo-bucket--usw2-az1--x-s3 --lifecycle-configuration file://lc-policy.json --checksum-algorithm crc32c ``` **Example** ``` Lifecycle expiration rule myprefix/ 500 64000 Enabled 7 ``` ## Using the AWS SDKs ------ #### [ SDK for Java ] **Example** ``` import software.amazon.awssdk.services.s3.model.PutBucketLifecycleConfigurationRequest; import software.amazon.awssdk.services.s3.model.PutBucketLifecycleConfigurationResponse; import software.amazon.awssdk.services.s3.model.ChecksumAlgorithm; import software.amazon.awssdk.services.s3.model.BucketLifecycleConfiguration; import software.amazon.awssdk.services.s3.model.LifecycleRule; import software.amazon.awssdk.services.s3.model.LifecycleRuleFilter; import software.amazon.awssdk.services.s3.model.LifecycleExpiration; import software.amazon.awssdk.services.s3.model.LifecycleRuleAndOperator; import software.amazon.awssdk.services.s3.model.GetBucketLifecycleConfigurationResponse; import software.amazon.awssdk.services.s3.model.GetBucketLifecycleConfigurationRequest; import software.amazon.awssdk.services.s3.model.DeleteBucketLifecycleRequest; import software.amazon.awssdk.services.s3.model.DeleteBucketLifecycleResponse; import software.amazon.awssdk.services.s3.model.AbortIncompleteMultipartUpload; // PUT a Lifecycle policy LifecycleRuleFilter objectExpirationFilter = LifecycleRuleFilter.builder().and(LifecycleRuleAndOperator.builder().prefix("dir1/").objectSizeGreaterThan(3L).objectSizeLessThan(20L).build()).build(); LifecycleRuleFilter mpuExpirationFilter = LifecycleRuleFilter.builder().prefix("dir2/").build(); LifecycleRule objectExpirationRule = LifecycleRule.builder().id("lc").filter(objectExpirationFilter).status("Enabled").expiration(LifecycleExpiration.builder() .days(10) .build()) .build(); LifecycleRule mpuExpirationRule = LifecycleRule.builder().id("lc-mpu").filter(mpuExpirationFilter).status("Enabled").abortIncompleteMultipartUpload(AbortIncompleteMultipartUpload.builder() .daysAfterInitiation(10) .build()) .build(); PutBucketLifecycleConfigurationRequest putLifecycleRequest = PutBucketLifecycleConfigurationRequest.builder() .bucket("amzn-s3-demo-bucket--usw2-az1--x-s3") .checksumAlgorithm(ChecksumAlgorithm.CRC32) .lifecycleConfiguration( BucketLifecycleConfiguration.builder() .rules(objectExpirationRule, mpuExpirationRule) .build() ).build(); PutBucketLifecycleConfigurationResponse resp = client.putBucketLifecycleConfiguration(putLifecycleRequest); // GET the Lifecycle policy GetBucketLifecycleConfigurationResponse getResp = client.getBucketLifecycleConfiguration(GetBucketLifecycleConfigurationRequest.builder().bucket("amzn-s3-demo-bucket--usw2-az1--x-s3").build()); // DELETE the Lifecycle policy DeleteBucketLifecycleResponse delResp = client.deleteBucketLifecycle(DeleteBucketLifecycleRequest.builder().bucket("amzn-s3-demo-bucket--usw2-az1--x-s3").build()); ``` ------ #### [ SDK for Go ] **Example** ``` package main import ( "context" "log" "github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/config" "github.com/aws/aws-sdk-go-v2/service/s3" "github.com/aws/aws-sdk-go-v2/service/s3/types" ) // PUT a Lifecycle policy func putBucketLifecycleConfiguration(client *s3.Client, bucketName string ) error { lifecycleConfig := &s3.PutBucketLifecycleConfigurationInput{ Bucket: aws.String(bucketName), LifecycleConfiguration: &types.BucketLifecycleConfiguration{ Rules: []types.LifecycleRule{ { ID: aws.String("lc"), Filter: &types.LifecycleRuleFilter{ And: &types.LifecycleRuleAndOperator{ Prefix: aws.String("foo/"), ObjectSizeGreaterThan: aws.Int64(1000000), ObjectSizeLessThan: aws.Int64(100000000), }, }, Status: types.ExpirationStatusEnabled, Expiration: &types.LifecycleExpiration{ Days: aws.Int32(int32(1)), }, }, { ID: aws.String("abortmpu"), Filter: &types.LifecycleRuleFilter{ Prefix: aws.String("bar/"), }, Status: types.ExpirationStatusEnabled, AbortIncompleteMultipartUpload: &types.AbortIncompleteMultipartUpload{ DaysAfterInitiation: aws.Int32(int32(5)), }, }, }, }, } _, err := client.PutBucketLifecycleConfiguration(context.Background(), lifecycleConfig) return err } // Get the Lifecycle policy func getBucketLifecycleConfiguration(client *s3.Client, bucketName string) error { getLifecycleConfig := &s3.GetBucketLifecycleConfigurationInput{ Bucket: aws.String(bucketName), } resp, err := client.GetBucketLifecycleConfiguration(context.Background(), getLifecycleConfig) if err != nil { return err } return nil } // Delete the Lifecycle policy func deleteBucketLifecycleConfiguration(client *s3.Client, bucketName string) error { deleteLifecycleConfig := &s3.DeleteBucketLifecycleInput{ Bucket: aws.String(bucketName), } _, err := client.DeleteBucketLifecycle(context.Background(), deleteLifecycleConfig) return err } func main() { cfg, err := config.LoadDefaultConfig(context.Background(), config.WithRegion("us-west-2")) // Specify your region here if err != nil { log.Fatalf("unable to load SDK config, %v", err) } s3Client := s3.NewFromConfig(cfg) bucketName := "amzn-s3-demo-bucket--usw2-az1--x-s3" putBucketLifecycleConfiguration(s3Client, bucketName) getBucketLifecycleConfiguration(s3Client, bucketName) deleteBucketLifecycleConfiguration(s3Client, bucketName) getBucketLifecycleConfiguration(s3Client, bucketName) } ``` ------ #### [ SDK for .NET ] **Example** ``` using Amazon; using Amazon.S3; using Amazon.S3.Model; using System; using System.Collections.Generic; using System.Threading.Tasks; namespace Amazon.DocSamples.S3 { class LifecycleTest { private const string bucketName = "amzn-s3-demo-bucket--usw2-az1--x-s3"; // Specify your bucket region (an example region is shown). private static readonly RegionEndpoint bucketRegion = RegionEndpoint.USWest2; private static IAmazonS3 client; public static void Main() { client = new AmazonS3Client(bucketRegion); AddUpdateDeleteLifecycleConfigAsync().Wait(); } private static async Task AddUpdateDeleteLifecycleConfigAsync() { try { var lifeCycleConfiguration = new LifecycleConfiguration() { Rules = new List { new LifecycleRule { Id = "delete rule", Filter = new LifecycleFilter() { LifecycleFilterPredicate = new LifecyclePrefixPredicate() { Prefix = "projectdocs/" } }, Status = LifecycleRuleStatus.Enabled, Expiration = new LifecycleRuleExpiration() { Days = 10 } } } }; // Add the configuration to the bucket. await AddExampleLifecycleConfigAsync(client, lifeCycleConfiguration); // Retrieve an existing configuration. lifeCycleConfiguration = await RetrieveLifecycleConfigAsync(client); // Add a new rule. lifeCycleConfiguration.Rules.Add(new LifecycleRule { Id = "mpu abort rule", Filter = new LifecycleFilter() { LifecycleFilterPredicate = new LifecyclePrefixPredicate() { Prefix = "YearlyDocuments/" } }, Expiration = new LifecycleRuleExpiration() { Days = 10 }, AbortIncompleteMultipartUpload = new LifecycleRuleAbortIncompleteMultipartUpload() { DaysAfterInitiation = 10 } }); // Add the configuration to the bucket. await AddExampleLifecycleConfigAsync(client, lifeCycleConfiguration); // Verify that there are now two rules. lifeCycleConfiguration = await RetrieveLifecycleConfigAsync(client); Console.WriteLine("Expected # of rulest=2; found:{0}", lifeCycleConfiguration.Rules.Count); // Delete the configuration. await RemoveLifecycleConfigAsync(client); // Retrieve a nonexistent configuration. lifeCycleConfiguration = await RetrieveLifecycleConfigAsync(client); } catch (AmazonS3Exception e) { Console.WriteLine("Error encountered ***. Message:'{0}' when writing an object", e.Message); } catch (Exception e) { Console.WriteLine("Unknown encountered on server. Message:'{0}' when writing an object", e.Message); } } static async Task AddExampleLifecycleConfigAsync(IAmazonS3 client, LifecycleConfiguration configuration) { PutLifecycleConfigurationRequest request = new PutLifecycleConfigurationRequest { BucketName = bucketName, Configuration = configuration }; var response = await client.PutLifecycleConfigurationAsync(request); } static async Task RetrieveLifecycleConfigAsync(IAmazonS3 client) { GetLifecycleConfigurationRequest request = new GetLifecycleConfigurationRequest { BucketName = bucketName }; var response = await client.GetLifecycleConfigurationAsync(request); var configuration = response.Configuration; return configuration; } static async Task RemoveLifecycleConfigAsync(IAmazonS3 client) { DeleteLifecycleConfigurationRequest request = new DeleteLifecycleConfigurationRequest { BucketName = bucketName }; await client.DeleteLifecycleConfigurationAsync(request); } } } ``` ------ #### [ SDK for Python ] **Example** ``` import boto3 client = boto3.client("s3", region_name="us-west-2") bucket_name = 'amzn-s3-demo-bucket--usw2-az1--x-s3' client.put_bucket_lifecycle_configuration( Bucket=bucket_name, ChecksumAlgorithm='CRC32', LifecycleConfiguration={ 'Rules': [ { 'ID': 'lc', 'Filter': { 'And': { 'Prefix': 'foo/', 'ObjectSizeGreaterThan': 1000000, 'ObjectSizeLessThan': 100000000, } }, 'Status': 'Enabled', 'Expiration': { 'Days': 1 } }, { 'ID': 'abortmpu', 'Filter': { 'Prefix': 'bar/' }, 'Status': 'Enabled', 'AbortIncompleteMultipartUpload': { 'DaysAfterInitiation': 5 } } ] } ) result = client.get_bucket_lifecycle_configuration( Bucket=bucket_name ) client.delete_bucket_lifecycle( Bucket=bucket_name ) ``` ------ # Troubleshooting S3 Lifecycle issues for directory buckets **Topics** + [ ## I set up my lifecycle configuration but objects in my directory bucket are not expiring ](#troubleshoot-directory-bucket-lifecycle-1) + [ ## How do I monitor the actions taken by my lifecycle rules? ](#troubleshoot-directory-bucket-lifecycle-2) ## I set up my lifecycle configuration but objects in my directory bucket are not expiring S3 Lifecycle for directory buckets utilizes public APIs to delete objects in S3 Express One Zone. To use object level public APIs, you must grant permission to `CreateSession` and allow S3 Lifecycle permission to delete your objects. If you have an active policy that denies deletes, this will prevent you from allowing S3 Lifecycle to delete objects on your behalf. It’s important to configure your bucket policies correctly to ensure that the objects that you want to delete are eligible for expiration. You can check your AWS CloudTrail logs for `AccessDenied` Trails for `CreateSession` API invocations in CloudTrail to verify if access has been denied. Checking your CloudTrail logs can assist you in troubleshooting access issues and identifying the root cause of access denied errors. You can then fix your incorrect access controls by updating the relevant policies. If you confirm that your bucket policies are set correctly and you are still experiencing issues, we recommend that you review the lifecycle rules to ensure that they are applied to the right subset of objects. ## How do I monitor the actions taken by my lifecycle rules? You can use AWS CloudTrail data event logs to monitor actions taken by S3 Lifecycle in directory buckets. For more information, see [CloudTrail log file examples](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-express-log-files.html).