# Centralize monitoring by using Amazon CloudWatch Observability Access Manager Centralize monitoring by using Observability Access Manager *Anand Krishna Varanasi, JAGDISH KOMAKULA, Ashish Kumar, Jimmy Morgan, Sarat Chandra Pothula, Vivek Thangamuthu, and Balaji Vedagiri, Amazon Web Services* ## Summary Observability is crucial to monitoring, understanding, and troubleshooting applications. Applications that span multiple accounts, as with AWS Control Tower or landing zone implementations, generate a large number of logs and trace data. To quickly troubleshoot problems or understand user analytics or business analytics, you need a common observability platform across all accounts. The Amazon CloudWatch Observability Access Manager gives you access to, and control over, multiple account logs from a central location. You can use the Observability Access Manager to view and manage observability data logs generated by source accounts. Source accounts are individual AWS accounts that generate observability data for their resources. Observability data is shared between source accounts and monitoring accounts. The shared observability data can include metrics in Amazon CloudWatch, logs in Amazon CloudWatch Logs, and traces in AWS X-Ray. For more information, see the [Observability Access Manager documentation](https://docs.aws.amazon.com/OAM/latest/APIReference/Welcome.html). This pattern is for users who have applications or infrastructure that run in multiple AWS accounts and need a common place to view logs. It explains how you can set up Observability Access Manager by using Terraform, to monitor the status and health of these applications or infrastructure. You can install this solution in multiple ways: + As a standalone Terraform module that you set up manually + By using a continuous integration and continuous delivery (CI/CD) pipeline + By integrating with other solutions such as [AWS Control Tower Account Factory for Terraform (AFT)](https://docs.aws.amazon.com/controltower/latest/userguide/aft-overview.html) The instructions in the [Epics](#centralize-monitoring-by-using-amazon-cloudwatch-observability-access-manager-epics) section cover the manual implementation. For AFT installation steps, see the README file for the GitHub [Observability Access Manager](https://github.com/aws-samples/cloudwatch-obervability-access-manager-terraform) repository. ## Prerequisites and limitations **Prerequisites** + [Terraform](https://www.terraform.io/) installed or referenced in your system or in automated pipelines. (We recommend that you use the [latest version](https://releases.hashicorp.com/terraform/).) + An account that you can use as a central monitoring account. Other accounts create links to the central monitoring account in order to view logs. + (Optional) A source code repository such as GitHub, AWS CodeCommit, Atlassian Bitbucket, or similar system. A source code repository isn’t necessary if you’re using automated CI/CD pipelines. + (Optional) Permissions to create pull requests (PRs) for code review and code collaboration in GitHub. **Limitations** Observability Access Manager has the following service quotas, which cannot be changed. Consider these quotas before you deploy this feature. For more information, see [CloudWatch service quotas](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html) in the CloudWatch documentation. + **Source account links**: You can link each source account to a maximum of five monitoring accounts. + **Sinks**: You can build multiple sinks for an account, but only one sink per AWS Region is allowed. In addition: + Sinks and links must be created in the same AWS Region; they cannot be cross-Region. **Cross-Region and cross-account monitoring** For cross-Region, cross-account monitoring, you can choose one of these options: + Create [cross-account and cross-Region CloudWatch dashboards](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html) for alarms and metrics. This option doesn’t support logs and traces. + Implement [centralized logging](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html) by using Amazon OpenSearch Service. + Create one sink per Region from all tenant accounts, push metrics to a centralized monitoring account (as described in this pattern), and then use [CloudWatch metric streams](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Metric-Streams.html) to send the data to a common external destination or to third-party monitoring products such as Datadog, Dynatrace, Sumo Logic, Splunk, or New Relic. ## Architecture **Components** CloudWatch Observability Access Manager consists of two major components that enable cross-account observability: + A *sink* provides the ability for source accounts to send observability data to the central monitoring account. A sink basically provides a gateway junction for source accounts to connect to. There can be only one sink gateway or connection, and multiple accounts can connect to it. + Each source account has a *link* to the sink gateway junction, and observability data is sent through this link. You must create a sink before you create links from each source account. **Architecture** The following diagram illustrates Observability Access Manager and its components. ![\[Architecture for cross-account observability with sinks and links.\]](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/images/pattern-img/00603763-4f99-456e-85e7-a80d803b087d/images/5188caf9-348b-4d91-b560-2b3d6ea81191.png) ## Tools **AWS services** + [Amazon CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html) helps you monitor the metrics of your AWS resources and the applications you run on AWS in real time. + [AWS Organizations](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_introduction.html) is an account management service that helps you consolidate multiple AWS accounts into an organization that you create and centrally manage. + [AWS Identity and Access Management (IAM)](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html) helps you securely manage access to your AWS resources by controlling who is authenticated and authorized to use them. **Tools** + [Terraform](https://www.terraform.io/) is an infrastructure as code (IaC) tool from HashiCorp that helps you create and manage cloud and on-premises resources. + [AWS Control Tower Account Factory for Terraform (AFT)](https://docs.aws.amazon.com/controltower/latest/userguide/aft-overview.html) sets up a Terraform pipeline to help you provision and customize accounts in AWS Control Tower. You can optionally use AFT to set up Observability Access Manager at scale across multiple accounts. **Code repository** The code for this pattern is available in the GitHub [Observability Access Manager](https://github.com/aws-samples/cloudwatch-obervability-access-manager-terraform) repository. ## Best practices + In AWS Control Tower environments, mark the logging account as the central monitoring account (sink). + If you have multiple organizations with multiple accounts in AWS Organizations, we recommend that you include the organizations instead of individual accounts in the configuration policy. If you have a small number of accounts or if the accounts aren’t part of an organization in the sink configuration policy, you might decide to include individual accounts instead. ## Epics ### Set up the sink module | Task | Description | Skills required | | --- | --- | --- | | Clone the repository. | Clone the GitHub Observability Access Manager repository:
git clone https://github.com/aws-samples/cloudwatch-obervability-access-manager-terraform
| AWS DevOps, Cloud administrator, AWS administrator | | Specify property values for the sink module. | In the `main.tf` file (in the `deployments/aft-account-customizations/LOGGING/terraform/`** **folder of the repository), specify values for the following properties:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/centralize-monitoring-by-using-amazon-cloudwatch-observability-access-manager.html)For more information, see [AWS::Oam::Sink](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-oam-sink.html) in the CloudFormation documentation. | AWS DevOps, Cloud administrator, AWS administrator | | Install the sink module. | Export the credentials of the AWS account that you have selected as the monitoring account, and install the Observability Access Manager sink module:
Terraform Init
Terrafom Plan
Terraform Apply
| AWS DevOps, Cloud administrator, AWS administrator | ### Set up the link module | Task | Description | Skills required | | --- | --- | --- | | Specify property values for the link module. | In the `main.tf `file (in the `deployments/aft-account-customizations/LOGGING/terraform/`** **folder of the repository), specify values for the following properties:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/centralize-monitoring-by-using-amazon-cloudwatch-observability-access-manager.html)For more information, see [AWS::Oam::Link](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-oam-link.html) in the CloudFormation documentation. | AWS DevOps, Cloud administrator, Cloud architect | | Install the link module for individual accounts. | Export the credentials of individual accounts and install the Observability Access Manager link module:
Terraform Plan
Terraform Apply
You can set up the link module individually for each account, or use [AFT](https://docs.aws.amazon.com/controltower/latest/userguide/aft-overview.html) to automatically install this module across a large number of accounts. | AWS DevOps, Cloud administrator, Cloud architect | ### Approve sink-to-link connections | Task | Description | Skills required | | --- | --- | --- | | Check the status message. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/centralize-monitoring-by-using-amazon-cloudwatch-observability-access-manager.html)On the right, you should see the status message **Monitoring account enabled** with a green checkmark. This means that the monitoring account has an Observability Access Manager sink that the links of other accounts will connect to. | | | Approve the link-to-sink connections. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/centralize-monitoring-by-using-amazon-cloudwatch-observability-access-manager.html)For more information, see [Link monitoring accounts with source accounts](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Unified-Cross-Account-Setup.html) in the CloudWatch documentation. | AWS DevOps, Cloud administrator, Cloud architect | ### Verify cross-account observability data | Task | Description | Skills required | | --- | --- | --- | | View cross-account data. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/centralize-monitoring-by-using-amazon-cloudwatch-observability-access-manager.html) | AWS DevOps, Cloud administrator, Cloud architect | ### (Optional) Enable source accounts to trust monitoring account | Task | Description | Skills required | | --- | --- | --- | | View metrics, dashboards, logs, widgets, and alarms from other accounts. | As an additional feature,** **you can share the CloudWatch metrics, dashboards, logs, widgets, and alarms with other accounts. Each account uses an IAM role called **CloudWatch-CrossAccountSharingRole **to gain access to this data.Source accounts that have a trust relationship with the central monitoring account can assume this role and view data from the monitoring account.CloudWatch provides a sample CloudFormation script to create the role. Choose **Manage role in IAM an**d run this script in the accounts where you want to view data.
{
    "Version": "2012-10-17",		 	 	 
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::XXXXXXXXX:root",
                    "arn:aws:iam::XXXXXXXXX:root",
                    "arn:aws:iam::XXXXXXXXX:root",
                    "arn:aws:iam::XXXXXXXXX:root"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
For more information, see [Enabling cross-account functionality in CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html#enable-cross-account-cross-Region) in the CloudWatch documentation. | AWS DevOps, Cloud administrator, Cloud architect | ### (Optional) View cross-account cross-Region from the monitoring account | Task | Description | Skills required | | --- | --- | --- | | Set up cross-account, cross-Region access. | In the central monitoring account, you can optionally add an account selector to easily switch between accounts and view their data without having to authenticate.[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/centralize-monitoring-by-using-amazon-cloudwatch-observability-access-manager.html)For more information, see [Cross-account cross-Region CloudWatch console](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html) in the CloudWatch documentation. | AWS DevOps, Cloud administrator, Cloud architect | ## Related resources + [CloudWatch cross-account observability](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Unified-Cross-Account.html) (Amazon CloudWatch documentation) + [Amazon CloudWatch Observability Access Manager API Reference](https://docs.aws.amazon.com/OAM/latest/APIReference/Welcome.html) (Amazon CloudWatch documentation) + [Resource: aws\$1oam\$1sink](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/oam_sink) (Terraform documentation) + [Data Source: aws\$1oam\$1link](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/oam_link) (Terraform documentation) + [CloudWatchObservabilityAccessManager](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/oam.html) (AWS Boto3 documentation)