# AWS Config Resource Compliance Dashboard AWS Config Resource Compliance Dashboard ## Introduction AWS Config is a fully managed service that provides you with resource inventory, configuration history, and configuration change notifications for security and governance. The Amazon Web Services (AWS) Config Resource Compliance Dashboard (CRCD) shows the inventory of your AWS resources, along with their compliance status, across multiple AWS accounts and Regions by leveraging your AWS Config data. [![AWS Videos](http://img.youtube.com/vi/https://www.youtube.com/embed/709-y8q81Q8?rel=0/0.jpg)](http://www.youtube.com/watch?v=https://www.youtube.com/embed/709-y8q81Q8?rel=0) ## Links ### Demo Dashboard Get more familiar with the dashboard using the live, interactive demo dashboard following this [link](https://cid.workshops.aws.dev/demo/?dashboard=cid-crcd). ### GitHub Project See the source code and the changelog at our GitHub [project](https://github.com/aws-samples/config-resource-compliance-dashboard). ## Advantages ### Compliance tracking Track compliance of your AWS Config rules and conformance packs per service, AWS Region, account, resource. Identify resources that require compliance remediation and establish a process for continuous compliance review. Verify that your tagging strategy is consistently applied across accounts and Regions. ### Democratize security and compliance visibility The AWS Config Dashboard helps security teams establish a compliance practice and offers visibility over security compliance to field teams, without them accessing AWS Config service or dedicated security tooling accounts. ### Shift-left security and compliance practices Field teams will see their non-compliant resources as quickly as security teams. This creates a short feedback loop that helps keep non-compliant resources to a minimum and helps organizations establish a consistent compliance review process with a shorter path to get to green compliance. ### A simplified Configuration Management Database (CMDB) experience in AWS Avoid investment in a dedicated external CMDB system or third-party tools. Access the inventory of resources in a single pane of glass, without accessing the AWS Management Console on each account and Region. Filter resources by account, Region, and fields that are specific to the resource such as IP address. If you tag consistently your resources, for example to map them to the application, owning team and environment, specify those tags to the dashboard and they will be displayed alongside other resource-specific information, and used for filtering your configuration items. Manage and plan the upgrade of Amazon RDS DB engines and AWS Lambda runtimes. ## Dashboard features ### AWS Config compliance + At-a-glance status of compliant and non-compliant resources and AWS Config rules. + Month-by-month compliance trend for resources and AWS Config rules. + Compliance breakdown by service, account, and Region. + Compliance tracking for AWS Config rules and conformance packs. ### Resource inventory management ![\[CRCD Dashboard\]](http://docs.aws.amazon.com/guidance/latest/cloud-intelligence-dashboards/images/images/dashboards/crcd-ec2-inventory.png) Inventory of Amazon EC2, Amazon EBS, Amazon S3, Amazon Relational Database Service (RDS) and AWS Lambda resources with filtering on account, Region and resource-specific fields (e.g. IP addresses for EC2, Lambda runtime, RDS database engine). Furthermore, the dashboard supports filtering of these resources by the custom tags that you use to categorize workloads and resources, such as Application, Owner and Environment. The name of the tags will be provided by you during installation. #### Resource inventory and EC2 Availability Zone dashboards Graphs that report summarized insights about resource configuration data, including detailed information about EC2 and EBS. Evaluate your resilience to AZ-level events by checking the distribution of your EC2 instances across Availability Zones. ### Tag compliance Visualize the results of AWS Config Managed Rule [required-tags](https://docs.aws.amazon.com/config/latest/developerguide/required-tags.html). You can deploy this rule to find resources in your accounts that were not launched with your desired tag configurations by specifying which resource types should have tags and the expected value for each tag. The rule can be deployed multiple times in AWS Config. To display data on the dashboard, the rules must have a name that starts with `required-tags`, `required-tag`, `requiredtags` or `requiredtag` (this is case insensitive). ![\[CRCD Dashboard\]](http://docs.aws.amazon.com/guidance/latest/cloud-intelligence-dashboards/images/images/dashboards/crcd-tag-compliance-summary.png) ### Contributors to AWS Config costs AWS Config cost is driven by the number of rule evaluations and configuration item changes being recorded. AWS Config cost are complex and calculating them precisely is outside the scope of this dashboard. To help you analyze AWS Config cost contributors and reduce operational costs while maintaining robust security and compliance monitoring, the dashboard reports the number of configuration items changes that are recorded and the number of AWS Config rule evaluations over time. The dashboard also covers other use cases that contribute to unnecessary AWS Config costs: + **Conformance pack rules that cannot be evaluated.** Conformance pack rules that have a compliance status of INSUFFICIENT\$1DATA do not have AWS resources in scope. Since you are charged for each rule evaluation regardless of the outcome, rules that return INSUFFICIENT\$1DATA still incur costs without delivering any compliance information. + **Redundant AWS Config rules.** While AWS Config provides multiple deployment options—including individual rules, conformance packs, Security Hub standards, and AWS Control Tower controls—many customers inadvertently implement duplicate rules across these services. This duplication leads to significant disadvantages: unnecessary costs from redundant evaluations, governance complexity that complicates compliance management, and potentially inconsistent remediation actions for the same compliance issues. To optimize compliance efforts and reduce costs, organizations should develop a strategic approach that eliminates rule duplication across their AWS environments. The dashboard will help you identify the rules that are deployed multiple times. ![\[CRCD Dashboard\]](http://docs.aws.amazon.com/guidance/latest/cloud-intelligence-dashboards/images/images/dashboards/crcd-cost-drivers.png) ### Configuration Item events The AWS Config Dashboards shows the timeline of your configuration changes. Find which resources were recently created, updated or deleted and see which accounts and Regions are delivering AWS Config data. Visualize the latest data imported into the dashboard and confirm that you are receiving data from all accounts and Regions. ![\[CRCD Dashboard\]](http://docs.aws.amazon.com/guidance/latest/cloud-intelligence-dashboards/images/images/dashboards/crcd-ci-events.png) ## Steps There are two possible ways to deploy the AWS Config dashboard on AWS Organizations. Read the [Perequisites](config-resource-prerequisites.md) page to understand which deployment setup is better for you. If you install the dashboard on a standalone account that is not part of an AWS Organization, follow the installation instructions in the Log Archive account. + [Prerequisites](config-resource-prerequisites.md) + [Deployment: Log Archive account](config-resource-log-archive.md) + [Deployment: Dashboard account](config-resource-dashboard-account.md) + [Optional post-deployment activities and FAQ](config-resource-post-deployment.md) + [Teardown](config-resource-teardown.md) **Note** These dashboards and their content: (a) are for informational purposes only, (b) represent current AWS product offerings and practices, which are subject to change without notice, and (c) does not create any commitments or assurances from AWS and its affiliates, suppliers or licensors. AWS content, products or services are provided "as is" without warranties, representations, or conditions of any kind, whether express or implied. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers. ## Update instructions If you already have installed the AWS Config Dasboard, you can check our [GitHub repository upgrade page](https://github.com/aws-samples/config-resource-compliance-dashboard/blob/main/documentation/upgrade.md) to see if there are instructions on how to upgrade to the latest version. ## Authors + Luca Casarini, Senior Technical Account Manager, AWS ## Contributors + Iakov Gan, Ex-Amazonian ## Feedback Support Follow [Feedback & Support](feedback-support.md) guide. # Prerequisites Prerequisites ## Architecture The AWS Config Resource Compliance Dashboard (CRCD) solution can be deployed in standalone AWS accounts or AWS accounts that are members of an AWS Organization. You can deploy the dashboard in a standalone account with [AWS Config enabled](https://docs.aws.amazon.com/config/latest/developerguide/getting-started.html). This option may be useful for proof of concept or testing purposes. In this case, all dashboard resources are deployed within the same AWS account. If you use AWS Organizations, AWS Config must be enabled with an [AWS Config delivery channel](https://docs.aws.amazon.com/config/latest/developerguide/manage-delivery-channel.html) sending files to a centralized Amazon S3 bucket (which we will call the Log Archive bucket) in a dedicated account (which we will call the Log Archive account). In this case, there are two possible ways to deploy the CRCD dashboard. 1. **Deploy in the Log Archive account** You can deploy the dashboard resources in the same Log Archive account where your AWS Config configuration files are delivered. The architecture in this case looks like this: ![\[CRCD Dashboard: deployment on AWS Organization\]](http://docs.aws.amazon.com/guidance/latest/cloud-intelligence-dashboards/images/images/dashboards/crcd-architecture-log-archive-account.png) 1. **Deploy in a separate Dashboard account** Alternatively, you can create a separate Dashboard account to deploy the dashboard resources. In this case, objects from the Log Archive bucket in the Log Archive account are replicated to another bucket in the Dashboard account. ![\[CRCD Dashboard: deployment on AWS Organization\]](http://docs.aws.amazon.com/guidance/latest/cloud-intelligence-dashboards/images/images/dashboards/crcd-architecture-dashboard-account.png) An Amazon Athena table is used to extract data from the AWS Config configuration files delivered to Amazon S3. Whenever a new object is added to the bucket, the Lambda Partitioner function is triggered. This function checks if the object is an AWS Config configuration update. If it is, the function adds a new partition to the corresponding Athena table with the new data; otherwise, the function ignores it. The solution provides Athena views, which are SQL queries that extract data from Amazon S3 using the schema defined in the Athena table. Finally, you can visualize the data in a Quick Sight dashboard that uses these views through Amazon Quick Sight datasets. ### Log Archive bucket encrypted with an AWS Key Management Service (KMS) key The deployment process supports Log Archive buckets encrypted using a customer-managed KMS key (SSE-KMS). In case of Log Archive account deployment: + Amazon Quick Sight will be granted permissions to use the KMS key for decrypt operations. This is done with an IAM policy. If you prefer, you can manually grant this permission directly on the key policy. See below for instructions. In case of Dashboard account deployment: + S3 replication must occur between buckets with the same type of encryption. + The Dashboard bucket will be encrypted with a KMS key which is created by the AWS CloudFormation template. + The S3 replication policy will have permissions to use the KMS keys of both buckets. **Note** If your Log Archive bucket is SSE-KMS encrypted, and you do not provide the ARN of the corresponding KMS key in the CloudFormation parameters, the dashboard resources will not have the necessary permissions to function correctly. ## Prerequisites 1. AWS Config enabled in the accounts and AWS Regions you want to track, with an AWS Config delivery channel sending files to a centralized Amazon S3 bucket (which we will call the Log Archive bucket) in a dedicated account (which we will call the Log Archive account). + We recommend that your AWS Config delivery channel delivers AWS Config configuration snapshot files every 24 hours for all accounts and Regions where AWS Config is active (see below for more information). 1. An AWS account where you’ll deploy the dashboard. 1. An IAM Role or IAM User with permissions to deploy the infrastructure using CloudFormation. 1. Sign up for [Amazon Quick Sight](https://docs.aws.amazon.com/quicksight/latest/user/signing-up.html) and create a user: 1. Select **Enterprise** edition. 1. For the **Get Paginated Reports add-on**, choose the option you prefer (this is not required for deploying the CRCD dashboard). 1. **Use IAM federated identities and Quick Sight-managed users**. 1. Select the Region where to deploy the dashboard. We recommend using the same Region of your Amazon S3 bucket. 1. Add a username and an e-mail where you’ll receive notifications about failed Quick Sight datasets updates. 1. Use the **Quick Sight-managed role (default)**. 1. Don’t modify the **Allow access and autodiscovery for these resources** section and click **Finish**. 1. Ensure you have SPICE capacity available in the Region where you’re deploying the dashboard. ### Account Names If you deployed other CUDOS dashboards, the dashboard will display account names. ## Before you start ### AWS Config considerations *Skip this paragraph if you have AWS Config enabled.* The solution leverages AWS Config data to build the visualizations on the dashboard. If you **do not** have AWS Config enabled, we strongly recommend building your strategy first: + Decide which accounts, Regions, and resources to monitor. + Define what "compliance" means to your organization, i.e. which [AWS Config rules](https://docs.aws.amazon.com/config/latest/developerguide/evaluate-config.html) or [conformance packs](https://docs.aws.amazon.com/config/latest/developerguide/conformance-packs.html) to activate. + Identify the account that will be delegated admin for AWS Config. + Keep in mind the paragraphs below when enabling AWS Config. **Note** Only when the AWS Config setup matches your needs should you consider deploying this dashboard. ### AWS Config delivery channel considerations The AWS Config delivery channel is a crucial component for managing and controlling where configuration updates are sent. It consists of an Amazon S3 bucket and an optional Amazon SNS topic, which is not needed by the AWS Config dashboard. The S3 bucket is used to store AWS Config configuration history and configuration snapshots files, while the SNS topic can be used for streaming configuration changes. A delivery channel is required to use AWS Config and is limited to one per Region per AWS account. When setting up a delivery channel, you can specify the name, the S3 bucket for file delivery, and the frequency of configuration snapshot delivery. A configuration **snapshot** provides a comprehensive view of all currently active recorded configuration items within a customer’s AWS account. In contrast, AWS Config delivers automatically a configuration **history** file to the S3 bucket every 6 hours. This file contains changes detected for each resource type since the last history file was delivered. Check this [blog post](https://aws.amazon.com/blogs/mt/configuration-history-configuration-snapshot-files-aws-config/) for more information on the difference between AWS Config configuration history and configuration snapshot files. The dashboard does not support [oversized configuration item change notifications](https://docs.aws.amazon.com/config/latest/developerguide/oversized-notification-example.html). To check your AWS Config delivery channel setup, you can use the AWS CLI command: ``` aws configservice describe-delivery-channels ``` This command will provide information about the delivery channel configuration on the account and Region where it is launched, including the S3 bucket where configuration updates are sent and the configuration snapshot delivery properties. Ensure the configuration is consistent across all accounts and Regions you want to record. The output of the CLI command should look like this: ``` { "DeliveryChannels": [ { "name": "[YOUR-DELIVERY-CHANNEL-NAME]", "s3BucketName": "[YOUR-LOG-ARCHIVE-BUCKET-NAME]", "s3KeyPrefix": "[OPTIONAL-S3-PREFIX-FOR-AWS-CONFIG-FILES]", "configSnapshotDeliveryProperties": { "deliveryFrequency": "TwentyFour_Hours" } } ] } ``` We recommend to have `configSnapshotDeliveryProperties` configured on your delivery channel with a delivery frequency of 24 hours, run the CLI command above to verify your setup. **Note** AWS Control Tower configures the AWS Config delivery channel with a 24-hour delivery frequency for configuration snapshot files. #### Click here to read what to do if your delivery channel is not configured to deliver configuration snapshots **How to add daily delivery of configuration snapshot files to your delivery channel** You have to configure this on every account and Region where you have AWS Config active. We’ll give an example below of how this can be achieved with the AWS CLI, but if your environment consists of several AWS accounts and Regions, we recommend using CloudFormation StackSets to ensure a consistent configuration. Here’s how you can use the AWS CLI to modify the existing settings and schedule the delivery of configuration snapshot files to your delivery channel configuration. 1. Log into the AWS Console in any account and Region, open AWS CloudShell. 1. Run the AWS CLI command `aws configservice describe-delivery-channels` and save the resulting JSON to a local file. Name it `deliveryChannel.json`. For example, your file may look like the one below. ``` { "name": "default", "s3BucketName": "config-bucket-123456789012", "snsTopicARN": "arn:aws:sns:us-east-1:123456789012:config-topic", "s3KeyPrefix": "my-prefix" } ``` 1. Verify the S3 bucket in `s3BucketName` is the name of your Log Archive bucket. 1. Edit the file to add the `configSnapshotDeliveryProperties` section: ``` { "name": "default", "s3BucketName": "config-bucket-123456789012", "snsTopicARN": "arn:aws:sns:us-east-1:123456789012:config-topic", "s3KeyPrefix": "my-prefix", "configSnapshotDeliveryProperties": { "deliveryFrequency": "TwentyFour_Hours" } } ``` You have to follow these steps consistently in every account and Region: 1. Log into the AWS Console of one account and Region, open AWS CloudShell. 1. Upload the `deliveryChannel.json` file containing the delivery channel configuration. 1. Use the `put-delivery-channel` AWS CLI [command](https://docs.aws.amazon.com/cli/latest/reference/configservice/put-delivery-channel.html) to update your delivery channel configuration according to the content of the JSON file. This command allows you to update or modify your current delivery channel settings. ``` aws configservice put-delivery-channel --delivery-channel file://deliveryChannel.json ``` Ensure this is done consistently in every account and Region. ### Regional considerations **Note** Data transfer costs will incur when Amazon Athena queries an Amazon S3 bucket across Regions. To avoid cross-region data transfer, Amazon Quick Sight and the Amazon S3 bucket containing AWS Config files must be deployed in the same Region. + If you have already deployed either resource, the other must use the same Region. If you haven’t deployed anything yet, you can choose a Region of your preference. + If you have deployed both resources in different Regions, we strongly recommend making changes so that both are in the same Region. + Once you have decided on the Region, deploy AWS resources supporting the dashboard (via CloudFormation) in the same Region. ### Tag Compliance: naming convention on the AWS Config rule This part of the dashboard visualizes the evaluation results of AWS Config Managed Rule [required-tags](https://docs.aws.amazon.com/config/latest/developerguide/required-tags.html). You can deploy this rule to find resources in your accounts that were not launched with your desired tag configurations by specifying which resource types should have tags and the expected value for each tag. The rule can be deployed multiple times in AWS Config. To display data on the dashboard, the rules must have a name that starts with `required-tags`, `required-tag`, `requiredtags` or `requiredtag` (this is case insensitive). ### Deployment architecture The most important decision is whether to deploy the dashboard on a dedicated Dashboard account or directly into the Log Archive account. These are the implications of each architecture. #### Log Archive account architecture | Pros | Cons | | --- | --- | | Keep your logs secure in the Log Archive account. | Your security team must deploy and maintain the AWS Config Dashboard resources, including user access to Quick Sight. Alternatively, you have to share access to the Log Archive account with other teams that will manage these resources. | | Avoid cost for data transfer and storing data on the Dashboard account. | The CRCD Dashboard adds complexity in user management if you already have Quick Sight dashboards deployed in the Log Archive account. | | | If you already have S3 object notification configured on your Config bucket, a part of the deployment process must be done manually. | #### Dashboard account architecture | Pros | Cons | | --- | --- | | Allow your DevOps or Platform teams independence in installing and maintaining the dashboard, as well as regulating user access. | Your security data will be copied to another AWS account. | | A limited number of resources must be deployed on Log Archive account. | Control Tower default installations may collect AWS Config and AWS CloudTrail on the same bucket. This means that all your security logs will be replicated to another account. | | | You will incur costs for the replication and storing a copy of your data on another Amazon S3 bucket. Cloud Trail logs will increase those costs needlessly, as they are not used by the dashboard. | | | If you already have S3 replication configured on your Log Archive bucket, a part of the deployment process must be done manually. | ### Deployment instructions + Follow [these instructions](config-resource-log-archive.md) to deploy the dashboard in the **Log Archive** account, or in a standalone AWS account. + Follow [these instructions](config-resource-dashboard-account.md) to deploy the dashboard in the **Dashboard** account. # Deployment: Log Archive account Deployment: Log Archive account ## Deployment Instructions The infrastructure needed to collect and process the data is defined in AWS CloudFormation. The dashboard resources are defined in a template file that can be installed using the [CID-CMD](https://github.com/aws-solutions-library-samples/cloud-intelligence-dashboards-framework/blob/main/CID-CMD.md) tool. ### Deployment on standalone account Follow the same installation instructions for the Log Archive account. ### Deployment on Log Archive account The installation process consists of two steps: 1. Data pipeline resources for the dashboard, via CloudFormation stack. 1. Quick Sight resources for the dashboard and the necessary Athena views, using the [CID-CMD](https://github.com/aws-solutions-library-samples/cloud-intelligence-dashboards-framework/blob/main/CID-CMD.md) command line tool. ![\[CRCD Dashboard: deployment steps on Log Archive account\]](http://docs.aws.amazon.com/guidance/latest/cloud-intelligence-dashboards/images/images/dashboards/crcd-deployment-steps-log-archive-account.png) #### Deployment Steps **Note** Ensure you are in the Region where both your Log Archive bucket and Amazon Quick Sight are deployed. ##### Step 1 1. Log into the AWS Management Console for your Log Archive account. 1. Click the Launch Stack button below to open the stack template in your CloudFormation console. This Stack will create the data pipeline resources for the dashboard. [https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?&templateURL=https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/cid-crcd-resources.yaml&stackName=config-dashboard-resources](https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?&templateURL=https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/cid-crcd-resources.yaml&stackName=config-dashboard-resources) 1. Specify the following parameters: + `Log Archive account ID` Enter the AWS account ID where you are currently logged in (Required). + `Log Archive bucket` Enter the name of the Amazon S3 bucket that collects AWS Config data (Required). + `ARN of the KMS key that encrypts the Log Archive bucket` Leave empty if the bucket is not encrypted with a KMS key. If you encrypt the bucket with a KMS key, copy the key’s ARN here. + CloudFormation will create an IAM policy that grants Amazon Quick Sight permissions to use the key for decrypt operations. + You may prefer managing key permissions on the key policy, rather than IAM. In his case, leave the parameter empty. You’ll have to manually grant key permissions in the key policy (more details below). + `Dashboard account ID` Enter the same value as the `Log Archive account ID` (Required). + `Dashboard bucket` Enter the same value as the `Log Archive bucket` (Required). + `ARN of the KMS key that encrypts the Dashboard bucket` Leave empty, this parameter is ignored in this deployment mode. + `Configure S3 event notification` This will configure S3 event notifications to trigger the Partitioner Lambda function, which creates the corresponding partition on Amazon Athena when a new AWS Config file is delivered to the Log Archive bucket (Required). + Select `yes` to configure S3 event notifications. + Select `no` if you have already configured event notifications on the Log Archive bucket. You’ll have to manually configure S3 event notifications (more details below). + The S3 event notification configuration is performed by an ad-hoc Lambda function (called `crcd-support-configure-s3-event-notification`) that will be called by the CloudFormation template automatically. **Note** The `crcd-support-configure-s3-event-notification` function will return an error (and the entire stack will fail) if you have already configured event notifications on the Log Archive bucket. In this case you must select `no` and run the stack again. + `Configure cross-account replication` Leave at the default value. This parameter is ignored in this deployment mode. + Leave all other parameters at their default value. 1. Run the CloudFormation template. 1. Note down the output values of the CloudFormation template. ##### Click here if you need to perform parts of this installation manually **Manual setup of KMS key permissions** **Note** Skip this section if you do not utilize a KMS key to encrypt your Dashboard bucket, or if you specified the key ARN in the CloudFormation parameter `ARN of the KMS key that encrypts the Dashboard bucket` in Step 1. Follow these steps to [edit](https://docs.aws.amazon.com/kms/latest/developerguide/key-policy-modifying.html) the key policy and grant the Quick Sight role permissions to use the key for decrypt operations. 1. Ensure you are logged into the AWS Management Console on the Log Archive account and Region where you created the KMS key that encrypts the Log Archive bucket. 1. Open the AWS Key Management Service console and click on the KMS key. 1. Add the following statement to the key policy: ``` { "Sid": "CRCD Dashboard allow Quick Sight Role access", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::ACCOUNT_ID:role/QUICKSIGHT_DATASOURCE_ROLE" }, "Action": [ "kms:Decrypt" ], "Resource": "*" } ``` Where: + `ACCOUNT_ID` is the AWS account ID where you installed thedashboard. + `QUICKSIGHT_DATASOURCE_ROLE` is the value of the output `Quick SightDataSourceRole` from the CloudFormation template. **Manual setup of S3 event notification** **Note** Skip this section if you selected `yes` in CloudFormation parameter `Configure S3 event notification` in Step 1. If you selected `no`, you must configure the Log Archive S3 bucket event notification to trigger the Lambda Partitioner function when objects are added to the bucket. CloudFormation has already deployed the necessary permissions for the Lambda function to access the Log Archive bucket. You can find the ARN of the Lambda Partitioner function in the output values of the CloudFormation template. We recommend that you configure your event notification to an SNS topic in these cases: 1. If your bucket already publishes events notifications to an SNS topic, [subscribe](https://docs.aws.amazon.com/lambda/latest/dg/with-sns.html#sns-trigger-console) the Lambda Partitioner function to the topic. 1. If your bucket sends event notifications to another Lambda function, change the notification to an SNS topic and [subscribe](https://docs.aws.amazon.com/lambda/latest/dg/with-sns.html#sns-trigger-console) both the existing function and the Lambda Partitioner function to that SNS topic. The S3 event notifications for this dashboard must meet the following requirements: 1. All object create events. 1. All prefixes. This may be a challenge depending on your current S3 event notification setup, as Amazon S3 [cannot have](https://docs.aws.amazon.com/AmazonS3/latest/userguide/notification-how-to-filtering.html#notification-how-to-filtering-examples-invalid) overlapping prefixes in two rules for the same event type. Follow [these instructions](https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-enable-disable-notification-intro.html) to add a notification configuration to your bucket using an Amazon SNS topic. Also, ensure that the Log Archive bucket is [granted permissions to publish event notification messages to your SNS topic](https://docs.aws.amazon.com/AmazonS3/latest/userguide/grant-destinations-permissions-to-s3.html). ##### Step 2 Remain logged into the AWS Management Console for your Log Archive account. **Note** At this step you will specify the tags to be used to display resources in the [Resource inventory management](config-resource-compliance-dashboard.md#config-resource-compliance-dashboard-inventory-management) part of the dashboard. Use the tags that classify workloads and resources in your organization. 1. Navigate to the AWS Management Console and open [AWS CloudShell](https://console.aws.amazon.com/cloudshell). Ensure to be in the correct Region. 1. Install the latest pip package of the [CID-CMD](https://github.com/aws-solutions-library-samples/cloud-intelligence-dashboards-framework/blob/main/CID-CMD.md) tool: ``` pip3 install --upgrade cid-cmd ``` If using [CloudShell](https://console.aws.amazon.com/cloudshell), use the following instead: ``` sudo yum install python3.11-pip -y python3.11 -m pip install -U cid-cmd ``` 1. Deploy the dashboard by running the following command (replace the parameters accordingly): + `--tag1` The name of the first tag you use to categorize resources. + `--tag2` The name of the second tag you use to categorize resources. + `--tag3` The name of the third tag you use to categorize resources. + `--tag4` The name of the fourth tag you use to categorize resources. + Notice that tag parameters are case sensitive and cannot be empty. If you do not use a tag, pass a short default value, e.g. `--tag4 'NA'`. + Leave all other parameters at their default value. ``` cid-cmd deploy \ --resources 'https://raw.githubusercontent.com/aws-samples/config-resource-compliance-dashboard/refs/heads/main/dashboard_template/cid-crcd.yaml' \ --dashboard-id 'cid-crcd' \ --athena-database 'cid_crcd_database' \ --athena-workgroup 'crcd-dashboard' \ --tag1 'REPLACE_WITH_CUSTOM_TAG_1' \ --tag2 'REPLACE_WITH_CUSTOM_TAG_2' \ --tag3 'REPLACE_WITH_CUSTOM_TAG_3' \ --tag4 'REPLACE_WITH_CUSTOM_TAG_4' ``` 1. The CID-CMD tool will prompt you to select a datasource: `[quicksight-datasource-id] Please choose DataSource (Select the first one if not sure):`. + If you have installed other CID/CUDOS dashboards, select the existing datasource `CID-CMD-Athena`. + Otherwise select `CID-CMD-Athena `. 1. When prompted `[quicksight-datasource-role] Please choose a Quick Sight role. It must have access to Athena:` select `CidCmdQuick SightDataSourceRole ` or `CidCmdQuick SightDataSourceRole` (the second option will appear by default if you have other CID/CUDOS dashboards). 1. In certain cases the installer will show an updated IAM policy JSON code and prompt `? [confirm-policy-AthenaAccess] Please confirm:`. Select `yes`. 1. When prompted `[timezone] Please select timezone for datasets scheduled refresh.:` select the time zone for dataset scheduled refresh in your Region (it is already preselected). 1. When prompted `Select taxonomy fields to add as dashboard filters and group by fields` select `Looks good` without adding taxonomy items. Taxonomy is not yet supported by the dashboard. 1. When prompted `[share-with-account] Share this dashboard with everyone in the account?:` select the option that works for you. ## Visualize the dashboard 1. Navigate to Quick Sight and then `Dashboards`. 1. Ensure you are in the correct Region. 1. Click on the **AWS Config Resource Compliance Dashboard (CRCD)** dashboard. # Deployment: Dashboard account Deployment: Dashboard account ## Deployment Instructions The infrastructure needed to collect and process the data is defined in AWS CloudFormation. The dashboard resources are defined in a template file that can be installed using the [CID-CMD](https://github.com/aws-solutions-library-samples/cloud-intelligence-dashboards-framework/blob/main/CID-CMD.md) tool. ## Installation on dedicated Dashboard account The installation process consists of three steps: 1. On the Dashboard account, deploy data pipeline resources for the dashboard using a CloudFormation stack. 1. On the Log Archive account, configure the S3 replication rule that copies AWS Config files from the Log Archive bucket to the Dashboard bucket using a CloudFormation stack. 1. On the Dashboard account, deploy Quick Sight resources for the dashboard and the necessary Athena views using the [CID-CMD](https://github.com/aws-solutions-library-samples/cloud-intelligence-dashboards-framework/blob/main/CID-CMD.md) command line tool. ![\[CRCD Dashboard: deployment steps on Dashboard account\]](http://docs.aws.amazon.com/guidance/latest/cloud-intelligence-dashboards/images/images/dashboards/crcd-deployment-steps-dashboard-account.png) **Note** The S3 replication rule configured at step 2 is valid only for new AWS Config files delivered to the Log Archive bucket, i.e. it **will not** replicate files that previously existed on the Log Archive bucket. ## Deployment Steps **Note** Ensure you are in the AWS Region where both your Log Archive bucket and Amazon Quick Sight are deployed. ## Step 1 [in Dashboard account] 1. Log into the AWS Management Console for your Dashboard account. 1. Ensure you are in the same Region as the Log Archive bucket. 1. Click the Launch Stack button below to open the stack template in your CloudFormation console. This Stack will create the data pipeline resources for the dashboard. [https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?&templateURL=https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/cid-crcd-resources.yaml&stackName=config-dashboard-resources](https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?&templateURL=https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/cid-crcd-resources.yaml&stackName=config-dashboard-resources) 1. Specify the following parameters: + `Log Archive account ID` Enter the AWS account ID of the Log Archive account. Notice this in **not** where you are currently logged in (Required). + `Log Archive bucket` Enter the name of the Amazon S3 bucket that collects AWS Config data (Required). + `ARN of the KMS key that encrypts the Log Archive bucket` If you encrypt the Log Archive bucket with a KMS key, copy the key’s ARN here. + If a KMS key ARN is passed here, the CloudFormation template will create a new KMS key and use it to encrypt the the Dashboard bucket. + `Dashboard account ID` Enter the AWS account ID where you are currently logged in (Required). + `Dashboard bucket` Enter the name of the Amazon S3 bucket that will collect AWS Config data. The CloudFormation template will create this bucket on the Dashboard account (Required). + `ARN of the KMS key that encrypts the Dashboard bucket` Leave empty. This parameter is ignored in this deployment mode. + `Configure S3 event notification` Leave at the default value. This parameter is ignored in this deployment mode. + `Configure cross-account replication` Leave at the default value. This parameter is ignored in this deployment mode. + Leave all other parameters at their default value. 1. Run the CloudFormation template. 1. Note down the output values of the CloudFormation template. 1. If you encrypt the Log Archive bucket with a KMS key, the template will create a KMS key to encrypt the Dashboard bucket. Note down its ARN from the output value `DashboardBucketKmsKeyArn`. You will use it at the next step. ## Step 2 [in Log Archive account] 1. Log into the AWS Management Console for your Log Archive account. 1. Click the Launch Stack button below to open the stack template in your CloudFormation console. This Stack will create the data pipeline resources for the dashboard. [https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?&templateURL=https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/cid-crcd-resources.yaml&stackName=config-dashboard-resources](https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?&templateURL=https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/cid-crcd-resources.yaml&stackName=config-dashboard-resources) 1. Specify the following parameters: + `Log Archive account ID` Enter the AWS account ID where you are currently logged in (Required). + `Log Archive bucket` Enter the name of the Amazon S3 bucket that collects AWS Config data (Required). + `ARN of the KMS key that encrypts the Log Archive bucket` If you encrypt the Log Archive bucket with a KMS key, copy the key’s ARN here. + `Dashboard account ID` Insert the ID of the Dashboard account that you specified in this field at Step 1 (Required). + `Dashboard bucket` Insert the bucket name that you specified in this field at Step 1 (Required). + `ARN of the KMS key that encrypts the Dashboard bucket` This parameter is used only at this step of the Dashboard account deployment. If you encrypt the Log Archive bucket with a KMS key, insert the ARN of the KMS key created in Step 1 (it’s `DashboardBucketKmsKeyArn` on the CloudFormation Outputs). + `Configure S3 event notification` Leave at the default value. This parameter is ignored in this deployment mode. + `Configure cross-account replication` (Required) + Select `yes` to configure S3 replication from the Log Archive bucket to the Dashboard bucket. + Select `no` if you already have configured S3 replication rules on the Log Archive bucket. You will have to setup S3 replication manually (see below). + The S3 replication configuration is performed by an ad-hoc Lambda function (**Configure bucket replication** in the diagram above) that will be called by the CloudFormation template automatically. **Note** If you select `yes`, and you have existing S3 replication configurations, the **Configure bucket replication** function will return an error and the entire stack will fail. In this case you must select `no` and run the stack again. + Leave all other parameters at their default value. 1. Run the CloudFormation template. 1. Note down the output values of the CloudFormation template. ### Click here if your need to manually setup S3 replication **Manual setup of S3 replication** 1. Log onto the Log Archive account and open the Amazon S3 console. 1. You can replicate AWS Config files from the centralized Log Archive bucket to the Dashboard bucket through an Amazon S3 Replication configuration, follow these [instructions](https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication-walkthrough-2.html). 1. Specify the IAM role created by the CloudFormation template at Step 2, as reported in the output value `ReplicationRoleArn` of the template. If your Log Archive bucket is SSE-KMS encrypted, the replication role will have the necessary permissions, no need for additional steps. **Note** The S3 replication rule configured at step 2 is valid only for new AWS Config files delivered to the Log Archive bucket, i.e. it **will not** replicate files that previously existed on the Log Archive bucket. ## Step 3 [in Dashboard account] Log back into the AWS Management Console for your Dashboard account. **Note** At this step you will specify the tags to be used to display resources in the [Resource inventory management](config-resource-compliance-dashboard.md#config-resource-compliance-dashboard-inventory-management) part of the dashboard. Use the tags that classify workloads and resources in your organization. 1. Navigate to the AWS Management Console and open [AWS CloudShell](https://console.aws.amazon.com/cloudshell). Ensure to be in the correct Region. 1. Install the latest pip package of the [CID-CMD](https://github.com/aws-solutions-library-samples/cloud-intelligence-dashboards-framework/blob/main/CID-CMD.md) tool: ``` pip3 install --upgrade cid-cmd ``` If using [CloudShell](https://console.aws.amazon.com/cloudshell), use the following instead: ``` sudo yum install python3.11-pip -y python3.11 -m pip install -U cid-cmd ``` 1. Deploy the dashboard by running the following command (replace the parameters accordingly): + `--tag1` The name of the first tag you use to categorize resources. + `--tag2` The name of the second tag you use to categorize resources. + `--tag3` The name of the third tag you use to categorize resources. + `--tag4` The name of the fourth tag you use to categorize resources. + Notice that tag parameters are case sensitive and cannot be empty. If you do not use a tag, pass a short default value, e.g. `--tag4 'NA'`. + Leave all other parameters at their default value. ``` cid-cmd deploy \ --resources 'https://raw.githubusercontent.com/aws-samples/config-resource-compliance-dashboard/refs/heads/main/dashboard_template/cid-crcd.yaml' \ --dashboard-id 'cid-crcd' \ --athena-database 'cid_crcd_database' \ --athena-workgroup 'crcd-dashboard' \ --tag1 'REPLACE_WITH_CUSTOM_TAG_1' \ --tag2 'REPLACE_WITH_CUSTOM_TAG_2' \ --tag3 'REPLACE_WITH_CUSTOM_TAG_3' \ --tag4 'REPLACE_WITH_CUSTOM_TAG_4' ``` 1. The CID-CMD tool will prompt you to select a datasource: `[quicksight-datasource-id] Please choose DataSource (Select the first one if not sure):`. + If you have installed other CID/CUDOS dashboards, select the existing datasource `CID-CMD-Athena`. + Otherwise select `CID-CMD-Athena `. 1. When prompted `[quicksight-datasource-role] Please choose a Quick Sight role.` select `CidCmdQuick SightDataSourceRole ` or `CidCmdQuick SightDataSourceRole` (the second option will appear as default if you have other CID/CUDOS dashboards). 1. In certain cases the installer will show an updated IAM policy JSON code and prompt `? [confirm-policy-AthenaAccess] Please confirm:`. Select `yes`. 1. When prompted `[timezone] Please select timezone for datasets scheduled refresh.:` select the time zone for dataset scheduled refresh in your Region (it is already preselected). 1. When prompted `Select taxonomy fields to add as dashboard filters and group by fields` select `Looks good` without adding taxonomy items. Taxonomy is not yet supported by the dashboard. 1. When prompted `[share-with-account] Share this dashboard with everyone in the account?:` select the option that works for you. ## Visualize the dashboard 1. Navigate to Quick Sight and then `Dashboards`. 1. Ensure you are in the correct Region. 1. Click on the **AWS Config Resource Compliance Dashboard (CRCD)** dashboard. # Optional post-deployment activities and FAQ Optional post-deployment activities and FAQ ## Post-deployment configuration ### Lambda Partitioner function #### Amazon CloudWatch Log Group Retention The logs of the Partitioner function are kept for 14 days. If needed, [change the retention period](https://docs.aws.amazon.com/solutions/latest/security-insights-on-aws/change-the-cloudwatch-log-group-retention-period.html) directly on the Amazon CloudWatch console. ### Quick Sight #### Configure dataset refresh schedule By default, the datasets for the CRCD dashboard are refreshed once a day. You can optionally configure the Refresh Schedule in Quick Sight with a different frequency: 1. Navigate to Quick Sight and then `Datasets`. 1. All the datasets for this dashboard have the prefix `config_`. 1. Click on a dataset, and then open the `Refresh` tab. 1. Click on `ADD NEW SCHEDULE`, and configure as needed. ## FAQ ### I installed the dashboard successfully, but there’s no data If you followed our recommendations in the [prerequisites](config-resource-prerequisites.md), AWS Config delivers a configuration snapshot file every 24 hours, so you will probably start seeing data in a couple of days, depending on when the configuration snapshot files are generated and when the Quick Sight datasets are refreshed. AWS Config generates history records approximately 6 hours after a resource is changed. These records will be loaded on the dasboard faster, and be visible on the **Configuration Item Events** tab. Follow these steps to have AWS Config generate a configuration snapshot and visualize its data on the dashboard: 1. Log into the AWS Management Console of an account of you organization. 1. Open [AWS CloudShell](https://console.aws.amazon.com/cloudshell) in the AWS Region whose data you want to export. 1. Run the following command: ``` aws configservice describe-delivery-channels ``` 1. This command will provide information about your current delivery channel configuration, including the S3 bucket where configuration updates are sent and the configuration snapshot delivery properties. The output of the CLI command should look like this: ``` { "DeliveryChannels": [ { "name": "[YOUR-DELIVERY-CHANNEL-NAME]", "s3BucketName": "[YOUR-LOG-ARCHIVE-BUCKET-NAME]", "s3KeyPrefix": "[OPTIONAL-S3-PREFIX-FOR-AWS-CONFIG-FILES]", "configSnapshotDeliveryProperties": { "deliveryFrequency": "TwentyFour_Hours" } } ] } ``` 1. Note down the name of your delivery channel. 1. Run this command to generate an AWS Config snapshot (replace `"YOUR-DELIVERY-CHANNEL-NAME"` with the name reported above): ``` aws configservice deliver-config-snapshot --delivery-channel-name "YOUR-DELIVERY-CHANNEL-NAME" ``` The snapshot file will be delivered to the Log Archive bucket, optionally replicated to the Dashboard bucket, and indexed by the Lambda Partitioner function. 1. Optionally repeat these steps on other AWS accounts/Regions. We recommend doing this only for test purposes, or for rapidly checking the AWS Config data of a few accounts of your interest. AWS Config will deliver a snapshot file for all your resources within 24 hours. 1. Open Athena and query the table (or any view) to see if the data has been indexed. Mind that some dashboards elements will still need time to visualize your data. ``` SELECT * FROM "cid_crcd_database"."cid_crcd_config" limit 10; ``` 1. Log onto Quick Sight and [refresh](https://docs.aws.amazon.com/quicksight/latest/user/refreshing-imported-data.html) your datasets before opening the dashboard. # Teardown Teardown ## Remove the AWS Config Resource Compliance Dashboard dashboard resources Follow these steps to remove the dashboard. ### Step 1: all deployment architectures 1. Log into the AWS Console of the account where you deployed the dashboard. This is the AWS account ID that you specified in the `Dashboard account ID` parameter of the CloudFormation template. 1. Open AWS CloudShell in the AWS Region where the dashboard is deployed. 1. You need to use the dahboard YAML file corresponding to your [CRCD release](https://github.com/aws-samples/config-resource-compliance-dashboard/releases). Upload both `cid-crcd.yaml` and `cid-crcd-definition.yaml` on the `dashboard_template` directory to AWS CloudShell. 1. Execute the following command to delete the dashboard: ``` cid-cmd delete --resources cid-crcd.yaml ``` 1. When prompted: + Select the `[cid-crcd] AWS Config Resource Compliance Dashboard (CRCD)` dashboard. + For each Quick Sight dataset, choose `yes` to delete the dataset. + If prompted, accept the default values for the S3 Path for the Athena table. + If prompted, accept the default values for the tags. ### Step 2: only for deployment on Log Archive or standalone account **Note** Follow these steps if you deployed the dashboard on the Log Archive account or a standalone AWS account. 1. Log into the AWS Console of the account where you deployed the dashboard resources with CloudFormation. This is the AWS account ID that you specified both in the `Log Archive account ID` and the `Dashboard account ID` parameters of the CloudFormation template. 1. Revert any manual configuration made during setup. 1. Open the S3 console and empty the Amazon S3 bucket for the Athena Query results. The bucket name is in the CloudFormation stack output. 1. In the same account, open CloudFormation and delete the stack that installed the data pipeline resources for the dashboard. ### Step 2: only for deployment on dedicated Dashboard account **Note** Follow these steps if you deployed the dashboard on a dedicated Dashboard account. #### Remove resources on Log Archive account 1. Log into the AWS Console of the Log Archive account. This is the AWS account ID that you specified in the `Log Archive account ID` parameter of the CloudFormation template. 1. Revert any manual configuration made during setup. 1. Open CloudFormation and delete the stack that installed the resources for the dashboard. #### Remove resources on Dashboard account 1. Log into the AWS Console of the account where you deployed the dashboard resources with CloudFormation. This is the AWS account ID that you specified in the `Dashboard account ID` parameter of the CloudFormation template. 1. Revert any manual configuration made during setup. 1. Open the S3 console and empty the Amazon S3 bucket for the Athena Query results. The bucket name is in the CloudFormation stack output. 1. Empty the Dashboard bucket, as well. This bucket contains a copy of the AWS Config files from the Log Archive account. The bucket name is in the CloudFormation stack output. 1. In the same account, open CloudFormation and delete the stack that installed the data pipeline resources for the dashboard.