# Defining sensitive data exceptions with allow lists Defining sensitive data exceptions with allow lists With allow lists in Amazon Macie, you can define specific text and text patterns that you want Macie to ignore when it inspects Amazon Simple Storage Service (Amazon S3) objects for sensitive data. These are typically sensitive data exceptions for your particular scenarios or environment. If data matches text or a text pattern in an allow list, Macie doesn’t report the data. This is the case even if the data matches the criteria of a [managed data identifier](managed-data-identifiers.md) or a [custom data identifier](custom-data-identifiers.md). By using allow lists, you can refine your analysis of Amazon S3 data and reduce noise. You can create and use two types of allow lists in Macie: + **Predefined text** – For this type of list, you specify certain character sequences to ignore. For example, you might specify the names of public representatives for your organization, specific phone numbers, or specific sample data that your organization uses for testing. If you use this type of list, Macie ignores text that exactly matches an entry in the list. This type of allow list is helpful if you want to specify words, phrases, and other kinds of character sequences that aren’t sensitive, aren’t likely to change, and don’t necessarily adhere to a common pattern. + **Regular expression** – For this type of list, you specify a regular expression (*regex*) that defines a text pattern to ignore. For example, you might specify the pattern for your organization's public phone numbers, email addresses for your organization’s domain, or patterned sample data that your organization uses for testing. If you use this type of list, Macie ignores text that completely matches the pattern defined by the list. This type of allow list is helpful if you want to specify text that isn’t sensitive but varies or is likely to change while also adhering to a common pattern. After you create an allow list, you can [create and configure sensitive data discovery jobs](discovery-jobs-create.md) to use it, or [add it to your settings for automated sensitive data discovery](discovery-asdd-account-configure.md). Macie then uses the list when it analyzes data. If Macie finds text that matches an entry or pattern in an allow list, Macie doesn’t report that occurrence of text in sensitive data findings, statistics, and other types of results. You can manage and use allow lists in all the AWS Regions where Macie is currently available except the Asia Pacific (Osaka) Region. **Topics** + [Configuration options for allow lists](allow-lists-options.md) + [Creating an allow list](allow-lists-create.md) + [Checking the status of an allow list](allow-lists-status-check.md) + [Changing an allow list](allow-lists-change.md) + [Deleting an allow list](allow-lists-delete.md) # Configuration options and requirements for allow lists Configuration options for allow lists In Amazon Macie, you can use allow lists to specify text or text patterns that you want Macie to ignore when it inspects Amazon Simple Storage Service (Amazon S3) objects for sensitive data. Macie provides options for two types of allow lists, predefined text and regular expressions. A list of predefined text is helpful if you want Macie to ignore specific words, phrases, and other kinds of character sequences that you don't consider sensitive. Examples are: the names of public representatives for your organization, specific phone numbers, or specific sample data that your organization uses for testing. If Macie finds text that matches the criteria of a managed or custom data identifier and the text also matches an entry in an allow list, Macie doesn't report that occurrence of text in sensitive data findings, statistics, and other types of results. A regular expression (*regex*) is helpful if you want Macie to ignore text that varies or is likely to change while also adhering to a common pattern. The regex specifies a text pattern to ignore. Examples are: public phone numbers for your organization, email addresses for your organization's domain, or patterned sample data that your organization uses for testing. If Macie finds text that matches the criteria of a managed or custom data identifier and the text also matches a regex pattern in an allow list, Macie doesn't report that occurrence of text in sensitive data findings, statistics, and other types of results. You can create and use both types of allow lists in all the AWS Regions where Macie is currently available except the Asia Pacific (Osaka) Region. As you create and manage allow lists, keep the following options and requirements in mind. Also note that list entries and regex patterns for mailing addresses aren't supported. **Contents** + [ ## Options and requirements for lists of predefined text ](#allow-lists-options-s3list) + [ ### Syntax requirements ](#allow-lists-options-s3list-syntax) + [ ### Storage requirements ](#allow-lists-options-s3list-storage) + [ ### Encryption/Decryption requirements ](#allow-lists-options-s3list-encryption) + [ ### Design considerations and recommendations ](#allow-lists-options-s3list-notes) + [ ## Options and requirements for regular expressions ](#allow-lists-options-regex) + [ ### Syntax support and recommendations ](#allow-lists-options-regex-syntax) + [ ### Examples ](#allow-lists-options-regex-examples) ## Options and requirements for lists of predefined text Options and requirements for lists of predefined text For this type of allow list, you provide a line-delimited plaintext file that lists specific character sequences to ignore. The list entries are typically words, phrases, and other kinds of character sequences that you don’t consider sensitive, aren’t likely to change, and don’t necessarily adhere to a specific pattern. If you use this type of list, Amazon Macie doesn't report occurrences of text that exactly match an entry in the list. Macie treats each list entry as a string literal value. To use this type of allow list, start by creating the list in a text editor and saving it as a plaintext file. Then upload the list to an S3 general purpose bucket. Also ensure that the storage and encryption settings for the bucket and the object allow Macie to retrieve and decrypt the list. Then [create and configure settings for the list](allow-lists-create.md) in Macie. After you configure the settings in Macie, we recommend that you test the allow list with a small, representative set of data for your account or organization. To test a list, you can [create a one-time job](discovery-jobs-create.md). Configure the job to use the list in addition to the managed and custom data identifiers that you typically use to analyze data. You can then review the job's results—sensitive data findings, sensitive data discovery results, or both. If the job's results differ from what you expect, you can change and test the list until the results are what you expect. After you finish configuring and testing an allow list, you can create and configure additional jobs to use it, or add it to your settings for automated sensitive data discovery. When those jobs start to run or the next automated discovery analysis cycle starts, Macie retrieves the latest version of the list from Amazon S3 and stores it in temporary memory. Macie then uses this temporary copy of the list when it inspects S3 objects for sensitive data. When a job finishes running or the analysis cycle is complete, Macie permanently deletes its copy of the list from memory. The list doesn't persist in Macie. Only the list's settings persist in Macie. **Important** Because lists of predefined text don't persist in Macie, it's important to [check the status of your allow lists](allow-lists-status-check.md) periodically. If Macie can’t retrieve or parse a list that you configured a job or automated discovery to use, Macie doesn’t use the list. This might produce unexpected results, such as sensitive data findings for text that you specified in the list. **Topics** + [ ### Syntax requirements ](#allow-lists-options-s3list-syntax) + [ ### Storage requirements ](#allow-lists-options-s3list-storage) + [ ### Encryption/Decryption requirements ](#allow-lists-options-s3list-encryption) + [ ### Design considerations and recommendations ](#allow-lists-options-s3list-notes) ### Syntax requirements Syntax requirements for lists of predefined text When you create this type of allow list, note the following requirements for the list's file: + The list must be stored as a plaintext (`text/plain`) file, such as a .txt, .text, or .plain file. + The list must use line breaks to separate individual entries. For example: ``` Akua Mansa John Doe Martha Rivera 425-555-0100 425-555-0101 425-555-0102 ``` Macie treats each line as a single, distinct entry in the list. The file can also contain blank lines to improve readability. Macie skips blank lines when it parses the file. + Each entry can contain 1–90 UTF–8 characters. + Each entry must be a complete, exact match for the text to ignore. Macie doesn't support use of wildcard characters or partial values for entries. Macie treats each entry as a string literal value. Matches aren't case sensitive. + The file can contain 1–100,000 entries. + The total storage size of the file can't exceed 35 MB. ### Storage requirements Storage requirements for lists of predefined text As you add and manage allow lists in Amazon S3, note the following storage requirements and recommendations: + **Regional support** – An allow list must be stored in a bucket that's in the same AWS Region as your Macie account. Macie can’t access an allow list if it’s stored in a different Region. + **Bucket ownership** – An allow list must be stored in a bucket that's owned by your AWS account. If you want other accounts to use the same allow list, consider creating an Amazon S3 replication rule to replicate the list to buckets that are owned by those accounts. For information about replicating S3 objects, see [Replicating objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication.html) in the *Amazon Simple Storage Service User Guide*. In addition, your AWS Identity and Access Management (IAM) identity must have read access to the bucket and object that store the list. Otherwise, you won't be allowed to create or update the list's settings or check the list's status by using Macie. + **Storage types and classes** – An allow list must be stored in a general purpose bucket, not a directory bucket. In addition, it must be stored using one of the following storage classes: Reduced Redundancy (RRS), S3 Glacier Instant Retrieval, S3 Intelligent-Tiering, S3 One Zone-IA, S3 Standard, or S3 Standard-IA. + **Bucket policies** – If you store an allow list in a bucket that has a restrictive bucket policy, ensure that the policy allows Macie to retrieve the list. To do this, you can add a condition for the Macie service-linked role to the bucket policy. For more information, see [Allowing Macie to access S3 buckets and objects](monitoring-restrictive-s3-buckets.md). Also ensure that the policy allows your IAM identity to have read access to the bucket. Otherwise, you won't be allowed to create or update the list's settings or check the list's status by using Macie. + **Object paths** – If you store more than one allow list in Amazon S3, the object path for each list must be unique. In other words, each allow list must be stored separately in its own S3 object. + **Versioning** – When you add an allow list to a bucket, we recommend that you also enable versioning for the bucket. You can then use date and time values to correlate versions of the list with the results of sensitive data discovery jobs and automated sensitive data discovery cycles that use the list. This can help with data privacy and protection audits or investigations that you perform. + **Object Lock** – To prevent an allow list from being deleted or overwritten for a certain amount of time or indefinitely, you can enable Object Lock for the bucket that stores the list. Enabling this setting doesn’t prevent Macie from accessing the list. For information about this setting, see [Locking objects with Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html) in the *Amazon Simple Storage Service User Guide*. ### Encryption/Decryption requirements Encryption/Decryption requirements for lists of predefined text If you encrypt an allow list in Amazon S3, the permissions policy for the [Macie service-linked role](service-linked-roles.md) typically grants Macie the permissions that it needs to decrypt the list. However, this depends on the type of encryption that’s used: + If a list is encrypted using server-side encryption with an Amazon S3 managed key (SSE-S3), Macie can decrypt the list. The service-linked role for your Macie account grants Macie the permissions that it needs. + If a list is encrypted using server-side encryption with an AWS managed AWS KMS key (DSSE-KMS or SSE-KMS), Macie can decrypt the list. The service-linked role for your Macie account grants Macie the permissions that it needs. + If a list is encrypted using server-side encryption with a customer managed AWS KMS key (DSSE-KMS or SSE-KMS), Macie can decrypt the list only if you allow Macie to use the key. To learn how to do this, see [Allowing Macie to use a customer managed AWS KMS key](discovery-supported-encryption-types.md#discovery-supported-encryption-cmk-configuration). **Note** You can encrypt a list with a customer managed AWS KMS key in an external key store. However, the key might then be slower and less reliable than a key that’s managed entirely within AWS KMS. If latency or an availability issue prevents Macie from decrypting the list, Macie doesn’t use the list when it analyzes S3 objects. This might produce unexpected results, such as sensitive data findings for text that you specified in the list. To reduce this risk, consider storing the list in an S3 bucket that’s configured to use the key as an S3 Bucket Key. For information about using KMS keys in external key stores, see [External key stores](https://docs.aws.amazon.com/kms/latest/developerguide/keystore-external.html) in the *AWS Key Management Service Developer Guide*. For information about using S3 Bucket Keys, see [Reducing the cost of SSE-KMS with Amazon S3 Bucket Keys](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html) in the *Amazon Simple Storage Service User Guide*. + If a list is encrypted using server-side encryption with a customer-provided key (SSE-C) or client-side encryption, Macie can’t decrypt the list. Consider using SSE-S3, DSSE-KMS, or SSE-KMS encryption instead. If a list is encrypted with an AWS managed KMS key or a customer managed KMS key, your AWS Identity and Access Management (IAM) identity must also be allowed to use the key. Otherwise, you won't be allowed to create or update the list's settings or check the list's status by using Macie. To learn how to check or change the permissions for a KMS key, see [Key policies in AWS KMS](https://docs.aws.amazon.com/kms/latest/developerguide/key-policies.html) in the *AWS Key Management Service Developer Guide*. For detailed information about encryption options for Amazon S3 data, see [Protecting data with encryption](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingEncryption.html) in the *Amazon Simple Storage Service User Guide*. ### Design considerations and recommendations Design considerations for lists of predefined text In general, Macie treats each entry in an allow list as a string literal value. That is to say, Macie ignores each occurrence of text that exactly matches a complete entry in an allow list. Matches aren't case sensitive. However, Macie uses the entries as part of a larger data extraction and analysis framework. The framework includes machine learning and pattern matching functions that factor dimensions such as grammatical and syntactical variations and, in many cases, keyword proximity. The framework also factors an S3 object’s file type or storage format. Therefore, keep the following considerations and recommendations in mind as you add and manage the entries in an allow list. **Prepare for different file types and storage formats** For unstructured data, such as text in an Adobe Portable Document Format (.pdf) file, Macie ignores text that exactly matches a complete entry in an allow list, including text that spans multiple lines or pages. For structured data, such as columnar data in a CSV file or record-based data in a JSON file, Macie ignores text that exactly matches a complete entry in an allow list if all the text is stored in a single field, cell, or array. This requirement doesn’t apply to structured data that’s stored in an otherwise unstructured file, such as a table in a .pdf file. For example, consider the following content in a CSV file: ``` Name,Account ID Akua Mansa,111111111111 John Doe,222222222222 ``` If `Akua Mansa` and `John Doe` are entries in an allow list, Macie ignores those names in the CSV file. The complete text of each list entry is stored in a single `Name` field. Conversely, consider a CSV file that contains the following columns and fields: ``` First Name,Last Name,Account ID Akua,Mansa,111111111111 John,Doe,222222222222 ``` If `Akua Mansa` and `John Doe` are entries in an allow list, Macie doesn’t ignore those names in the CSV file. None of the fields in the CSV file contain the complete text of an entry in the allow list. **Include common variations** Add entries for common variations of numeric data, proper nouns, terms, and alphanumeric character sequences. For example, if you add names or phrases that contain only one space between words, also add variations that include two spaces between words. Similarly, add words and phrases that do and don’t contain special characters, and consider including common syntactical and semantic variations. For the US phone number *425-555-0100*, for example, you might add these entries to an allow list: ``` 425-555-0100 425.555.0100 (425) 555-0100 +1-425-555-0100 ``` For the date *February 1, 2022* in a multinational context, you might add entries that include common syntactical variations for English and French, including variations that do and don't include special characters: ``` February 1, 2022 1 février 2022 1 fevrier 2022 Feb 01, 2022 1 fév 2022 1 fev 2022 02/01/2022 01/02/2022 ``` For names of people, include entries for various forms of a name that you don't consider sensitive. For example, include: the first name followed by the last name; the last name followed by the first name, the first and last name separated by one space; the first and last name separated by two spaces; and nicknames. For the name *Martha Rivera*, for example, you might add: ``` Martha Rivera Martha Rivera Rivera, Martha Rivera, Martha Rivera Martha Rivera Martha ``` If you want to ignore variations of a specific name that contains many parts, create an allow list that uses a regular expression instead. For example, for the name *Dr. Martha Lyda Rivera, PhD*, you might use the following regular expression: `^(Dr. )?Martha\s(Lyda|L\.)?\s?Rivera,?( PhD)?$`. ## Options and requirements for regular expressions Options and requirements for regular expressions For this type of allow list, you specify a regular expression (*regex*) that defines a text pattern to ignore. For example, you might specify the pattern for your organization's public phone numbers, email addresses for your organization’s domain, or patterned sample data that your organization uses for testing. The regex defines a common pattern for a specific kind of data that you don’t consider sensitive. If you use this type of allow list, Amazon Macie doesn't report occurrences of text that completely match the specified pattern. Unlike an allow list that specifies predefined text to ignore, you create and store the regex and all other list settings in Macie. When you create or update this type of allow list, you can test the list’s regex with sample data before you save the list. We recommend that you do this with multiple sets of sample data. If you create a regex that’s too general, Macie might ignore occurrences of text that you consider sensitive. If a regex is too specific, Macie might not ignore occurrences of text that you don’t consider sensitive. To protect against malformed or long-running expressions, Macie also compiles and tests the regex against a collection of sample text automatically, and notifies you of issues to address. For additional testing, we recommend that you also test the list’s regex with a small, representative set of data for your account or organization. To do this, you can [create a one-time job](discovery-jobs-create.md). Configure the job to use the list in addition to the managed and custom data identifiers that you typically use to analyze data. You can then review the job's results—sensitive data findings, sensitive data discovery results, or both. If the job's results differ from what you expect, you can change and test the regex until the results are what you expect. After you configure and test an allow list, you can create and configure additional jobs to use it, or add it to your settings for automated sensitive data discovery. When those job run or Macie performs automated discovery, Macie uses the latest version of the list's regex to analyze data. **Topics** + [ ### Syntax support and recommendations ](#allow-lists-options-regex-syntax) + [ ### Examples ](#allow-lists-options-regex-examples) ### Syntax support and recommendations Syntax support and recommendations for regular expressions in allow lists An allow list can specify a regular expression (*regex*) that contains as many as 512 characters. Macie supports a subset of the regex pattern syntax provided by the [Perl Compatible Regular Expressions (PCRE) library](https://www.pcre.org/). Of the constructs provided by the PCRE library, Macie doesn’t support the following pattern elements: + Backreferences + Capturing groups + Conditional patterns + Embedded code + Global pattern flags, such as `/i`, `/m`, and `/x` + Recursive patterns + Positive and negative look-behind and look-ahead zero-width assertions, such as `?=`, `?!`, `?<=`, and `? aws macie2 create-allow-list ^ --criteria={\"s3WordsList\":{\"bucketName\":\"amzn-s3-demo-bucket\",\"objectKey\":\"allowlists/macie/mylist.txt\"}} ^ --name my_allow_list ^ --description "Lists public phone numbers and names for Example Corp." ``` When you submit your request, Macie tests the list's settings. Macie also verifies that it can retrieve the list from Amazon S3 and parse the list's content. If an error occurs, your request fails and Macie returns a message that describes the error. For detailed information that can help you troubleshoot the error, see [Options and requirements for lists of predefined text](allow-lists-options.md#allow-lists-options-s3list). If Macie can retrieve and parse the list, your request succeeds and you receive output similar to the following. ``` { "arn": "arn:aws:macie2:us-west-2:123456789012:allow-list/nkr81bmtu2542yyexample", "id": "nkr81bmtu2542yyexample" } ``` Where `arn` is the Amazon Resource Name (ARN) of the allow list that was created, and `id` is the unique identifier for the list. ------ After you save the list's settings, you can [create and configure sensitive data discovery jobs](discovery-jobs-create.md) to use the list, or [add the list to your settings for automated sensitive data discovery](discovery-asdd-account-configure.md). Each time those jobs start to run or an automated discovery analysis cycle starts, Macie retrieves the latest version of the list from Amazon S3. Macie then uses that version of the list when it analyzes data. ## Regular expression When you create an allow list that specifies a regular expression (*regex*), you define the regex and all other list settings directly in Macie. For the regex, Macie supports a subset of the pattern syntax provided by the [Perl Compatible Regular Expressions (PCRE) library](https://www.pcre.org/). For more information, see [Syntax support and recommendations](allow-lists-options.md#allow-lists-options-regex-syntax). You can create this type of list by using the Amazon Macie console or the Amazon Macie API. ------ #### [ Console ] Follow these steps to create an allow list by using the Amazon Macie console. **To create an allow list by using the console** 1. Open the Amazon Macie console at [https://console.aws.amazon.com/macie/](https://console.aws.amazon.com/macie/). 1. In the navigation pane, under **Settings**, choose **Allow lists**. 1. On the **Allow lists** page, choose **Create**. 1. Under **Select a list type**, choose **Regular expression**. 1. Under **List settings**, use the following options to enter additional settings for the allow list: + For **Name**, enter a name for the list. The name can contain as many as 128 characters. + For **Description**, optionally enter a brief description of the list. The description can contain as many as 512 characters. + For **Regular expression**, enter the regex that defines the text pattern to ignore. The regex can contain as many as 512 characters. 1. (Optional) For **Evaluate**, enter up to 1,000 characters in the **Sample data** box, and then choose **Test** to test the regex. Macie evaluates the sample data and reports the number of occurrences of text that match the regex. You can repeat this step as many times as you like to refine and optimize the regex. **Note** We recommend that you test and refine the regex with multiple sets of sample data. If you create a regex that’s too general, Macie might ignore occurrences of text that you consider sensitive. If a regex is too specific, Macie might not ignore occurrences of text that you don’t consider sensitive. 1. (Optional) Under **Tags**, choose **Add tag**, and then enter as many as 50 tags to assign to the allow list. A *tag* is a label that you define and assign to certain types of AWS resources. Each tag consists of a required tag key and an optional tag value. Tags can help you identify, categorize, and manage resources in different ways, such as by purpose, owner, environment, or other criteria. To learn more, see [Tagging Macie resources](tagging-resources.md). 1. When you finish, choose **Create**. Macie tests the list's settings. Macie also tests the regex to verify that it can compile the expression. If an error occurs, Macie displays a message that describes the error. For detailed information that can help you troubleshoot the error, see [Options and requirements for regular expressions](allow-lists-options.md#allow-lists-options-regex). After you address any errors, you can save the allow list. ------ #### [ API ] Before you create this type of allow list in Macie, we recommend that you test and refine the regex with multiple sets of sample data. If you create a regex that’s too general, Macie might ignore occurrences of text that you consider sensitive. If a regex is too specific, Macie might not ignore occurrences of text that you don’t consider sensitive. To test an expression with Macie, you can use the [TestCustomDataIdentifier](https://docs.aws.amazon.com/macie/latest/APIReference/custom-data-identifiers-test.html) operation of the Amazon Macie API or, for the AWS CLI, run the [test-custom-data-identifier](https://docs.aws.amazon.com/cli/latest/reference/macie2/test-custom-data-identifier.html) command. Macie uses the same underlying code to compile expressions for allow lists and custom data identifiers. If you test an expression in this way, be sure to specify values only for the `regex` and `sampleText` parameters. Otherwise, you'll receive inaccurate results. When you're ready to create this type of allow list, use the [CreateAllowList](https://docs.aws.amazon.com/macie/latest/APIReference/allow-lists.html) operation of the Amazon Macie API and specify the appropriate values for the required parameters. For the `criteria` parameter, use the `regex` field to specify the regular expression that defines the text pattern to ignore. The expression can contain as many as 512 characters. To create this type of list by using the AWS CLI, run the [create-allow-list](https://docs.aws.amazon.com/cli/latest/reference/macie2/create-allow-list.html) command and specify the appropriate values for the required parameters. The following examples create an allow list named *my\$1allow\$1list*. The regex is designed to ignore all email addresses that a custom data identifier might otherwise detect for the `example.com` domain. This example is formatted for Linux, macOS, or Unix, and it uses the backslash (\$1) line-continuation character to improve readability. ``` $ aws macie2 create-allow-list \ --criteria '{"regex":"[a-z]@example.com"}' \ --name my_allow_list \ --description "Ignores all email addresses for Example Corp." ``` This example is formatted for Microsoft Windows and it uses the caret (^) line-continuation character to improve readability. ``` C:\> aws macie2 create-allow-list ^ --criteria={\"regex\":\"[a-z]@example.com\"} ^ --name my_allow_list ^ --description "Ignores all email addresses for Example Corp." ``` When you submit your request, Macie tests the list's settings. Macie also tests the regex to verify that it can compile the expression. If an error occurs, the request fails and Macie returns a message that describes the error. For detailed information that can help you troubleshoot the error, see [Options and requirements for regular expressions](allow-lists-options.md#allow-lists-options-regex). If Macie can compile the expression, the request succeeds and you receive output similar to the following: ``` { "arn": "arn:aws:macie2:us-west-2:123456789012:allow-list/km2d4y22hp6rv05example", "id": "km2d4y22hp6rv05example" } ``` Where `arn` is the Amazon Resource Name (ARN) of the allow list that was created, and `id` is the unique identifier for the list. ------ After you save the list, you can [create and configure sensitive data discovery jobs](discovery-jobs-create.md) to use it, or [add it to your settings for automated sensitive data discovery](discovery-asdd-account-configure.md). When those jobs run or Macie performs automated discovery, Macie uses the latest version of the list's regex to analyze data. # Checking the status of an allow list Checking the status of an allow list If you create an allow list, it's important to check its status periodically. Otherwise, errors might cause Amazon Macie to produce unexpected analysis results for your Amazon Simple Storage Service (Amazon S3) data. For example, Macie might create sensitive data findings for text that you specified in an allow list. If you configure a sensitive data discovery job to use an allow list and Macie can't access or use the list when the job starts to run, the job continues to run. However, Macie doesn't use the list when it analyzes S3 objects. Similarly, if an analysis cycle starts for automated sensitive data discovery and Macie can't access or use a specified allow list, the analysis continues but Macie doesn't use the list. Errors are unlikely to occur for an allow list that specifies a regular expression (*regex*). This is partly because Macie automatically tests the regex when you create or update the list's settings. In addition, you store the regex and all other list settings in Macie. However, errors can occur for an allow list that specifies predefined text, partly because you store the list in Amazon S3 instead of Macie. Common causes of errors are: + The S3 bucket or object is deleted. + The S3 bucket or object is renamed and the list's settings in Macie don't specify the new name. + The S3 bucket's permissions settings are changed and Macie loses access to the bucket and the object. + The encryption settings for the S3 bucket are changed and Macie can't decrypt the object that stores the list. + The policy for the encryption key is changed and Macie loses access to the key. Macie can't decrypt the S3 object that stores the list. **Important** Because these errors affect your analyses' results, we recommend that you check the status of all of your allow lists periodically. We recommend that you also do this if you change the permissions or encryption settings for an S3 bucket that stores an allow list, or you change the policy for an AWS Key Management Service (AWS KMS) key that's used to encrypt a list. For detailed information that can help you troubleshoot errors that occur, see [Options and requirements for lists of predefined text](allow-lists-options.md#allow-lists-options-s3list). **To check the status of an allow list** You can check the status of an allow list by using the Amazon Macie console or the Amazon Macie API. On the console, you can use a single page to check the status of all of your allow lists at the same time. If you use the Amazon Macie API, you can check the status of individual allow lists, one at a time. ------ #### [ Console ] Follow these steps to check the status of your allow lists by using the Amazon Macie console. **To check the status of your allow lists** 1. Open the Amazon Macie console at [https://console.aws.amazon.com/macie/](https://console.aws.amazon.com/macie/). 1. In the navigation pane, under **Settings**, choose **Allow lists**. 1. On the **Allow lists** page, choose refresh (![\[The refresh button, which is a button that displays an empty blue circle with an arrow.\]](http://docs.aws.amazon.com/macie/latest/user/images/btn-refresh-data.png)). Macie tests the settings for all of your allow lists and updates the **Status** field to indicate the current status of each list. If a list specifies a regular expression, its status is typically **OK**. This means that Macie can compile the expression. If a list specifies predefined text, its status can be any of the following values. **OK** Macie can retrieve and parse the contents of the list. **Access denied** Macie isn't allowed to access the S3 object that stores the list. Amazon S3 denied the request to retrieve the object. A list can also have this status if the object is encrypted with a customer managed AWS KMS key that Macie isn't allowed to use. To address this error, review the bucket policy and other permissions settings for the bucket and the object. Ensure that Macie is allowed to access and retrieve the object. If the object is encrypted with a customer managed AWS KMS key, also review the key policy and ensure that Macie is allowed to use the key. **Error** A transient or internal error occurred when Macie attempted to retrieve or parse the contents of the list. An allow list can also have this status if it's encrypted with an encryption key that Amazon S3 and Macie can't access or use. To address this error, wait a few minutes and then choose refresh (![\[The refresh button, which is a button that displays an empty blue circle with an arrow.\]](http://docs.aws.amazon.com/macie/latest/user/images/btn-refresh-data.png)) again. If the status continues to be **Error**, check the encryption settings for the S3 object. Ensure that the object is encrypted with a key that Amazon S3 and Macie can access and use. **Object is empty** Macie can retrieve the list from Amazon S3 but the list doesn't contain any content. To address this error, download the object from Amazon S3 and ensure that it contains the correct entries. If the entries are correct, review the list's settings in Macie. Ensure that the specified bucket and object names are correct. **Object not found** The list doesn't exist in Amazon S3. To address this error, review the list's settings in Macie. Ensure that the specified bucket and object names are correct. **Quota exceeded** Macie can access the list in Amazon S3. However, the number of entries in the list or the storage size of the list exceeds the quota for an allow list. To address this error, break the list into multiple files. Ensure that each file contains fewer than 100,000 entries. Also ensure that the size of each file is less than 35 MB. Then, upload each file to Amazon S3. When you finish, configure allow list settings in Macie for each file. You can have as many as five lists of predefined text in each supported AWS Region. **Throttled** Amazon S3 throttled the request to retrieve the list. To address this error, wait a few minutes and then choose refresh (![\[The refresh button, which is a button that displays an empty blue circle with an arrow.\]](http://docs.aws.amazon.com/macie/latest/user/images/btn-refresh-data.png)) again. **User access denied** Amazon S3 denied the request to retrieve the object. If the specified object exists, you're not allowed to access it or it's encrypted with an AWS KMS key that you're not allowed to use. To address this error, work with your AWS administrator to ensure that the list's settings specify the correct bucket and object names, and you have read access to the bucket and the object. If the object is encrypted, also ensure that it's encrypted with a key that you're allowed to use. 1. To review the settings and status of a specific list, choose the list's name. ------ #### [ API ] To check the status of an allow list programmatically, use the [GetAllowList](https://docs.aws.amazon.com/macie/latest/APIReference/allow-lists-id.html) operation of the Amazon Macie API. Or, if you're using the AWS CLI, run the [get-allow-list](https://docs.aws.amazon.com/cli/latest/reference/macie2/get-allow-list.html) command. For the `id` parameter, specify the unique identifier for the allow list whose status you want to check. To get this identifier, you can use the [ListAllowLists](https://docs.aws.amazon.com/macie/latest/APIReference/allow-lists.html) operation. The **ListAllowLists** operation retrieves information about all the allow lists for your account. If you're using the AWS CLI, you can run the [list-allow-lists](https://docs.aws.amazon.com/cli/latest/reference/macie2/list-allow-lists.html) command to retrieve this information. When you submit a **GetAllowList** request, Macie tests all the settings for the allow list. If the settings specify a regular expression (`regex`), Macie verifies that it can compile the expression. If the settings specify a list of predefined text (`s3WordsList`), Macie verifies that it can retrieve and parse the list. Macie then returns a `GetAllowListResponse` object that provides the details of the allow list. In the `GetAllowListResponse` object, the `status` object indicates the current status of the list: a status code (`code`) and, depending on the status code, a brief description of the list's status (`description`). If the allow list specifies a regex, the status code is typically `OK` and there isn't an associated description. This means that Macie compiled the expression successfully. If the allow list specifies predefined text, the status code varies depending on the test results: + If Macie retrieved and parsed the list successfully, the status code is `OK` and there isn't an associated description. + If an error prevented Macie from retrieving or parsing the list, the status code and description indicate the nature of the error that occurred. For a list of possible status codes and a description of each one, see [AllowListStatus](https://docs.aws.amazon.com/macie/latest/APIReference/allow-lists-id.html#allow-lists-id-model-allowliststatus) in the *Amazon Macie API Reference*. ------ # Changing an allow list Changing an allow list After you create an allow list, you can change most of the list's settings in Amazon Macie. For example, you can change the list's name and description. You can also add and edit tags for the list. The only setting that you can't change is a list's type. For example, if an existing list specifies a regular expression (*regex*), you can't change its type to predefined text. If an allow list specifies predefined text, you can also change the entries in the list. To do this, update the file that contains the entries. Then upload the new version of the file to Amazon Simple Storage Service (Amazon S3). The next time Macie prepares to use the list, Macie retrieves the latest version of the file from Amazon S3. When you upload the new file, ensure that you store it in the same S3 bucket and object. Or, if you change the name of the bucket or object, ensure that you update the list's settings in Macie. **To change the settings for an allow list** You can change the settings for an allow list by using the Amazon Macie console or the Amazon Macie API. ------ #### [ Console ] Follow these steps to change an allow list's settings by using the Amazon Macie console. **To change an allow list's settings by using the console** 1. Open the Amazon Macie console at [https://console.aws.amazon.com/macie/](https://console.aws.amazon.com/macie/). 1. In the navigation pane, under **Settings**, choose **Allow lists**. 1. On the **Allow lists** page, choose the name of the allow list that you want to change. The allow list page opens and displays the current settings for the list. 1. To add or edit tags for the allow list, choose **Manage tags** in the **Tags** section. Then change the tags as necessary. When you finish, choose **Save**. 1. To change other settings for the allow list, choose **Edit** in the **List settings** section. Then change the settings that you want: + **Name** – Enter a new name for the list. The name can contain as many as 128 characters. + **Description** – Enter a new description of the list. The description can contain as many as 512 characters. + If the allow list specifies predefined text: + **S3 bucket name** – Enter the name of the bucket that stores the list. In Amazon S3, you can find this value in the **Name** field of the bucket's properties. This value is case sensitive. In addition, don't use wildcard characters or partial values when you enter the name. + **S3 object name** – Enter the name of the S3 object that stores the list. In Amazon S3, you can find this value in the **Key** field of the object's properties. If the name includes a path, be sure to include the complete path when you enter the name, for example **allowlists/macie/mylist.txt**. This value is case sensitive. In addition, don't use wildcard characters or partial values when you enter the name. + If the allow list specifies a regular expression (*regex*), enter a new regex in the **Regular expression** box. The regex can contain as many as 512 characters. After you enter the new regex, optionally test it. To do this, enter up to 1,000 characters in the **Sample data** box, and then choose **Test**. Macie evaluates the sample data and reports the number of occurrences of text that match the regex. You can repeat this step as many times as you like to refine and optimize the regex before you save your changes. 1. When you finish, choose **Save**. Macie tests the list's settings. For a list of predefined text, Macie also verifies that it can retrieve the list from Amazon S3 and parse the list's content. For a regex, Macie also verifies that it can compile the expression. If an error occurs, Macie displays a message that describes the error. For detailed information that can help you troubleshoot the error, see [Configuration options and requirements for allow lists](allow-lists-options.md). After you address any errors, you can save your changes. ------ #### [ API ] To change an allow list's settings programmatically, use the [UpdateAllowList](https://docs.aws.amazon.com/macie/latest/APIReference/allow-lists-id.html) operation of the Amazon Macie API. Or, if you're using the AWS CLI, run the [update-allow-list](https://docs.aws.amazon.com/cli/latest/reference/macie2/update-allow-list.html) command. In your request, use the supported parameters to specify a new value for each setting that you want to change. Note that the `criteria`, `id`, and `name` parameters are required. If you don't want to change the value for a required parameter, specify the current value for the parameter. For example, the following command changes the name and description of an existing allow list. The example is formatted for Microsoft Windows and it uses the caret (^) line-continuation character to improve readability. ``` C:\> aws macie2 update-allow-list ^ --id km2d4y22hp6rv05example ^ --name my_allow_list-email ^ --criteria={\"regex\":\"[a-z]@example.com\"} ^ --description "Ignores all email addresses for the example.com domain" ``` Where: + *km2d4y22hp6rv05example* is the unique identifier for the list. + *my\$1allow\$1list-email* is the new name for the list. + *[a-z]@example.com* is the list's criteria, a regular expression. + *Ignores all email addresses for the example.com domain* is the new description for the list. When you submit your request, Macie tests the list's settings. If the list specifies predefined text (`s3WordsList`), this includes verifying that Macie can retrieve the list from Amazon S3 and parse the list's content. If the list specifies a regex (`regex`), this includes verifying that Macie can compile the expression. If an error occurs when Macie tests the settings, your request fails and Macie returns a message that describes the error. For detailed information that can help you troubleshoot the error, see [Configuration options and requirements for allow lists](allow-lists-options.md). If the request fails for another reason, Macie returns an HTTP 4*xx* or 500 response that indicates why the operation failed. If your request succeeds, Macie updates the list's settings and you receive output similar to the following. ``` { "arn": "arn:aws:macie2:us-west-2:123456789012:allow-list/km2d4y22hp6rv05example", "id": "km2d4y22hp6rv05example" } ``` Where `arn` is the Amazon Resource Name (ARN) of the allow list that was updated, and `id` is the unique identifier for the list. ------ # Deleting an allow list Deleting an allow list When you delete an allow list in Amazon Macie, you permanently delete all the list's settings. These settings can't be recovered after they're deleted. If the settings specify a list of predefined text that you store in Amazon Simple Storage Service (Amazon S3), Macie doesn't delete the S3 object that stores the list. Only the settings in Macie are deleted. If you configure sensitive data discovery jobs to use an allow list that you subsequently delete, the jobs will run as scheduled. However, your job results, both sensitive data findings and sensitive data discovery results, might report text that you previously specified in the allow list. Similarly, if you configure automated sensitive data discovery to use a list that you subsequently delete, daily analyses cycles will proceed. However, sensitive data findings, statistics, and other types of results might report text that you previously specified in the allow list. Before you delete an allow list, we recommend that you [review your job inventory](discovery-jobs-manage-view.md) to identify jobs that use the list and are scheduled to run in the future. In the inventory, the details panel indicates whether a job is configured to use any allow lists and, if so, which ones. We recommend that you also [check your settings for automated sensitive data discovery](discovery-asdd-account-configure.md). You might determine that it's best to change a list instead of deleting it. As an additional safeguard, Macie checks the settings for all of your jobs when you try to delete an allow list. If you configured jobs to use the list and any of those jobs have a status other than **Complete** or **Cancelled**, Macie doesn't delete the list unless you provide additional confirmation. **To delete an allow list** You can delete an allow list by using the Amazon Macie console or the Amazon Macie API. ------ #### [ Console ] Follow these steps to delete an allow list by using the Amazon Macie console. **To delete an allow list by using the console** 1. Open the Amazon Macie console at [https://console.aws.amazon.com/macie/](https://console.aws.amazon.com/macie/). 1. In the navigation pane, under **Settings**, choose **Allow lists**. 1. On the **Allow lists** page, select the checkbox for the allow list that you want to delete. 1. On the **Actions** menu, choose **Delete**. 1. When prompted for confirmation, enter **delete**, and then choose **Delete**. ------ #### [ API ] To delete an allow list programmatically, use the [DeleteAllowList](https://docs.aws.amazon.com/macie/latest/APIReference/allow-lists-id.html) operation of the Amazon Macie API. For the `id` parameter, specify the unique identifier for the allow list to delete. You can get this identifier by using the [ListAllowLists](https://docs.aws.amazon.com/macie/latest/APIReference/allow-lists.html) operation. The **ListAllowLists** operation retrieves information about all the allow lists for your account. If you're using the AWS CLI, you can run the [list-allow-lists](https://docs.aws.amazon.com/cli/latest/reference/macie2/list-allow-lists.html) command to retrieve this information. For the `ignoreJobChecks` parameter, specify whether to force deletion of the list, even if sensitive data discovery jobs are configured to use the list: + If you specify `false`, Macie checks the settings for all of your jobs that have a status other than `COMPLETE` or `CANCELLED`. If none of those jobs are configured to use the list, Macie deletes the list permanently. If any of those jobs are configured to use the list, Macie rejects your request and returns an HTTP 400 (`ValidationException`) error. The error message indicates the number of applicable jobs for up to 200 jobs. + If you specify `true`, Macie deletes the list permanently without checking the settings for any of your jobs. To delete an allow list by using the AWS CLI, run the [delete-allow-list](https://docs.aws.amazon.com/cli/latest/reference/macie2/delete-allow-list.html) command. For example: ``` C:\> aws macie2 delete-allow-list --id nkr81bmtu2542yyexample --ignore-job-checks false ``` Where *nkr81bmtu2542yyexample* is the unique identifier for the allow list to delete. If your request succeeds, Macie returns an empty HTTP 200 response. Otherwise, Macie returns an HTTP 4*xx* or 500 response that indicates why the operation failed. ------ If the allow list specified predefined text, you can optionally delete the S3 object that stores the list. However, keeping this object can help ensure that you have an immutable history of sensitive data findings and discovery results for data privacy and protection audits or investigations.