# OpenTelemetry Collector
The OpenTelemetry Collector is an open source, vendor-agnostic agent that receives, processes, and exports telemetry data. It acts as a central pipeline between your applications and Amazon CloudWatch, collecting metrics, logs, and traces from multiple sources and sending them to CloudWatch using the OpenTelemetry Protocol (OTLP).
Using the OpenTelemetry Collector with CloudWatch provides the following benefits:
+ Collect telemetry from multiple applications and hosts through a single agent, reducing the number of connections to CloudWatch.
+ Process and filter telemetry before sending it to CloudWatch, including adding or removing attributes, batching data, and sampling traces.
+ Use the same collector configuration across AWS, on-premises, and other cloud environments, providing a consistent telemetry pipeline regardless of where your applications run.
+ Send metrics to CloudWatch with rich labels that are available to query using the Prometheus Query Language (PromQL) in CloudWatch Query Studio.
## Supported receivers
The OpenTelemetry Collector supports a wide range of receivers for ingesting telemetry data. You can use OpenTelemetry receivers such as the OTLP receiver for applications instrumented with OpenTelemetry SDKs, or Prometheus receivers to scrape metrics from existing Prometheus exporters. Common Prometheus receivers used with CloudWatch include:
+ Prometheus receiver, for scraping any Prometheus-compatible endpoint
+ Host Metrics receiver, for collecting system-level metrics from the host
+ Kubernetes Cluster receiver, for collecting cluster-level metrics from the Kubernetes API server
You can configure multiple receivers in a single collector, allowing you to collect both OpenTelemetry and Prometheus metrics and send them to CloudWatch through the same pipeline. For the full list of available receivers, see the OpenTelemetry Collector [repository](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver).
## Getting started
Prerequisite – If using the OTLP endpoint for tracing, ensure that Transaction Search is enabled.
Steps:
1. Download the latest release of the OpenTelemetry Collector distribution. For more information, see the OpenTelemetry Collector [releases](https://github.com/open-telemetry/opentelemetry-collector-releases/releases).
1. Install the OpenTelemetry Collector on your host. The collector runs on any operating system and platform. For more information, see [Install the Collector](https://opentelemetry.io/docs/collector/installation/).
1. Configure AWS credentials on your Amazon EC2 or on-premises host. The collector uses these credentials to authenticate with CloudWatch when sending telemetry data. See below for details.
------
#### [ Setup IAM permissions for Amazon EC2 ]
**Follow the below procedure to attach the `CloudWatchAgentServerPolicy` IAM policy to the IAM role of your Amazon EC2 instance.**
1. Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/).
1. Choose **Roles** and find and select the role used by your Amazon EC2 instance.
1. Under the **Permissions** tab, choose **Add permissions**, **Attach policies**.
1. Using the search box, search for `CloudWatchAgentServerPolicy` policy.
1. Select the **CloudWatchAgentServerPolicy** policy and choose **Add permissions**.
------
#### [ Setup IAM permissions for on-premise hosts ]
**You can create an IAM user that can be used to provide permissions to your on-premise hosts.**
1. Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/).
1. Choose **Users**, **Create User**.
1. Under **User details**,for **User name**, enter a name for the new IAM user. This is the sign-in name for AWS that will be used to authenticate your host.
1. Choose **Next**.
1. On the **Set permissions** page, under **Permissions options**, select **Attach policies directly**.
1. From the **Permissions policies** list, select the **CloudWatchAgentServerPolicy** policy to add to your user.
1. Choose **Next**.
1. On the **Review and create** page, ensure that you are satisfied with the user name and that the **CloudWatchAgentServerPolicy** policy is under the **Permissions summary**.
1. Choose **Create user**.
1. **Create and retrieve your AWS access key and secret key** – In the navigation pane in the IAM console, choose **Users** and then select the user name of the user that you created in the previous step.
1. On the user's page, choose the **Security credentials** tab.
1. Under the **Access keys** section, choose **Create access key**.
1. For **Create access key Step 1**, choose **Command Line Interface (CLI)**.
1. For **Create access key Step 2**, optionally enter a tag and then choose **Next**.
1. For **Create access key Step 3,** select **Download .csv file** to save a .csv file with your IAM user's access key and secret access key. You need this information for the next steps.
1. Choose **Done**.
1. Configure your AWS credentials in your on-premises host by entering the following command. Replace *ACCESS\$1KEY\$1ID * and *SECRET\$1ACCESS\$1ID * with your newly generated access key and secret access key from the .csv file that you downloaded in the previous step.
```
$ aws configure
AWS Access Key ID [None]: ACCESS_KEY_ID
AWS Secret Access Key [None]: SECRET_ACCESS_ID
Default region name [None]: MY_REGION
Default output format [None]: json
```
------
1. Setup AWS credentials for your Amazon EKS or Kubernetes clusters. The easiest way to get started with Amazon EKS is to use the EKS OTel Container Insights add-on. If you prefer to use the OpenTelemetry Collector directly, follow the procedure below to set up AWS credentials for your Amazon EKS or Kubernetes clusters to send telemetry to CloudWatch.
------
#### [ Setup IAM permissions for Amazon EKS ]
1. Create an IAM OIDC identity provider for your cluster using the following command.
```
eksctl utils associate-iam-oidc-provider --cluster ${CLUSTER_NAME} --region ${REGION} --approve
```
1. Assign IAM roles to Kubernetes service account for OTel Collector using the following command.
```
eksctl create iamserviceaccount \
--name ${COLLECTOR_SERVICE_ACCOUNT}\
--namespace ${NAMESPACE} \
--cluster ${CLUSTER_NAME} \
--region ${REGION} \
--attach-policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy \
--approve \
--override-existing-serviceaccounts
```
------
#### [ Setup IAM permissions for Kubernetes ]
1. Configure your AWS credentials in your on-premises host by entering the following command. Replace *ACCESS\$1KEY\$1ID* and *SECRET\$1ACCESS\$1ID* with your newly generated access key and secret access key from the .csv file that you downloaded in the previous step. By default, the credential file is saved under */home/user/.aws/credentials.*.
```
aws configure
AWS Access Key ID [None]: ACCESS_KEY_ID
AWS Secret Access Key [None]: SECRET_ACCESS_ID
Default region name [None]: MY_REGION
Default output format [None]: json
```
1. Edit OpenTelemetry Collector resource to add the newly created AWS credentials secret by using the command: `kubectl edit OpenTelemetryCollector otel_collector`
1. Using the file editor, add the AWS credentials into the OpenTelemetryCollector container by adding the following configuration to the top of the deployment. Replace the path */home/user/.aws/credentials* with the location of your local AWS credentials file.
```
spec:
volumeMounts:
- mountPath: /rootfs
volumeMounts:
- name: aws-credentials
mountPath: /root/.aws
readOnly: true
volumes:
- hostPath:
path: /home/user/.aws/credentials
name: aws-credentials
```
------
1. Configure the OTLP exporter in your collector configuration to send telemetry to the CloudWatch endpoint. See examples below.
## Collector configuration examples
Copy and paste the content below to configure your collector to send logs and traces to the OTLP endpoints.
```
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
exporters:
otlphttp/logs:
compression: gzip
logs_endpoint: logs_otlp_endpoint
headers:
x-aws-log-group: ency_log_group
x-aws-log-stream: default
auth:
authenticator: sigv4auth/logs
otlphttp/traces:
compression: gzip
traces_endpoint: traces_otlp_endpoint
auth:
authenticator: sigv4auth/traces
extensions:
sigv4auth/logs:
region: "region"
service: "logs"
sigv4auth/traces:
region: "region"
service: "xray"
service:
telemetry:
extensions: [sigv4auth/logs, sigv4auth/traces]
pipelines:
logs:
receivers: [otlp]
exporters: [otlphttp/logs]
traces:
receivers: [otlp]
exporters: [otlphttp/traces]
```
The following is an example to send logs and traces using sigv4 to us-east-1.
```
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
exporters:
otlphttp/logs:
compression: gzip
logs_endpoint: https://logs.us-east-1.amazonaws.com/v1/logs
headers:
x-aws-log-group: MyApplicationLogs
x-aws-log-stream: default
auth:
authenticator: sigv4auth/logs
otlphttp/traces:
compression: gzip
traces_endpoint: https://xray.us-east-1.amazonaws.com/v1/traces
auth:
authenticator: sigv4auth/traces
extensions:
sigv4auth/logs:
region: "us-east-1"
service: "logs"
sigv4auth/traces:
region: "us-east-1"
service: "xray"
service:
telemetry:
extensions: [sigv4auth/logs, sigv4auth/traces]
pipelines:
logs:
receivers: [otlp]
exporters: [otlphttp/logs]
traces:
receivers: [otlp]
exporters: [otlphttp/traces]
```
**Note**
Configure your OpenTelemetry SDKs to *always\$1on* sampling configuration to reliably record 100% spans and get full visibility into your critical applications with CloudWatch Application Signals. For more information, see an [OpenTelemetry Java SDK sampler configuration](https://opentelemetry.io/docs/languages/java/sdk/#sampler) example. For an example on setting up OpenTelemetry Collector with X-Ray OTLP endpoint, see the [application signals demo](https://github.com/aws-observability/application-signals-demo/blob/main/scripts/opentelemetry/otel_simple_setup/opentelemetry.yaml) repository.
Copy and paste the content below to configure your collector to send metrics to the OTLP endpoints.
```
receivers:
otlp:
protocols:
http:
endpoint: "0.0.0.0:4318"
processors:
batch:
send_batch_size: 200
timeout: 10s
exporters:
otlphttp:
tls:
insecure: false
endpoint: metrics_otlp_endpoint
auth:
authenticator: sigv4auth
extensions:
sigv4auth:
service: "monitoring"
region: "region"
service:
extensions: [sigv4auth]
pipelines:
metrics:
receivers: [otlp]
processors: [batch]
exporters: [otlphttp]
```
The following is an example to send metrics using sigv4 to us-east-1.
```
receivers:
otlp:
protocols:
http:
endpoint: "0.0.0.0:4318"
processors:
batch:
send_batch_size: 200
timeout: 10s
exporters:
otlphttp:
tls:
insecure: false
endpoint: "https://monitoring.us-east-1.amazonaws.com/v1/metrics:443"
auth:
authenticator: sigv4auth
extensions:
sigv4auth:
service: "monitoring"
region: "us-east-1"
service:
extensions: [sigv4auth]
pipelines:
metrics:
receivers: [otlp]
processors: [batch]
exporters: [otlphttp]
```