# Troubleshooting the CloudWatch agent
You can use the information in this section to troubleshoot issues you might encounter with the CloudWatch agent.
When encountering issues with the CloudWatch agent, you can use the `AWSSupport-TroubleshootCloudWatchAgent` automation runbook. The AWS troubleshooting tool can:
+ Verify IAM permissions and instance profiles
+ Check agent status and analyze logs
+ Test endpoint connectivity
+ Automatically collect and upload relevant logs to Amazon S3
For detailed information on the AWS troubleshooting tool, see [Support Automation Workflow (SAW) Runbook - Troubleshoot CloudWatch agent](https://repost.aws/articles/ARDFhNRgSMRcahrIbGJaIC4g/support-automation-workflow-saw-runbook-troubleshoot-amazon-cloudwatch-agent).
**Topics**
+ [
## CloudWatch agent command line parameters
](#CloudWatch-Agent-options-help)
+ [
## Install the CloudWatch agent using Run Command fails
](#CloudWatch-Agent-installation-fails)
+ [
## The CloudWatch agent won't start
](#CloudWatch-Agent-troubleshooting-cannot-start)
+ [
## Verify that the CloudWatch agent is running
](#CloudWatch-Agent-troubleshooting-verify-running)
+ [
## The CloudWatch agent won't start, and the error mentions an Amazon EC2 Region
](#CloudWatch-Agent-troubleshooting-EC2-region)
+ [
## The CloudWatch agent won't start on Windows Server
](#CloudWatch-Agent-troubleshooting-Windows-start)
+ [
## Where are the metrics?
](#CloudWatch-Agent-troubleshooting-no-metrics)
+ [
## The CloudWatch agent takes a long time to run in a container or logs a hop limit error
](#CloudWatch-Agent-container-slow)
+ [
## I updated my agent configuration but don’t see the new metrics or logs in the CloudWatch console
](#CloudWatch-Agent-troubleshooting-update-no-new-metrics)
+ [
## CloudWatch agent files and locations
](#CloudWatch-Agent-files-and-locations)
+ [
## Finding information about CloudWatch agent versions
](#CloudWatch-Agent-troubleshooting-agent-version)
+ [
## Logs generated by the CloudWatch agent
](#CloudWatch-Agent-troubleshooting-loginfo)
+ [
## Stopping and restarting the CloudWatch agent
](#CloudWatch-Agent-troubleshooting-stopping-restarting)
## CloudWatch agent command line parameters
To see the full list of parameters supported by the CloudWatch agent, enter the following at the command line at a computer where you have it installed:
```
amazon-cloudwatch-agent-ctl -help
```
## Install the CloudWatch agent using Run Command fails
To install the CloudWatch agent using Systems Manager Run Command, the SSM Agent on the target server must be version 2.2.93.0 or later of the SSM Agent agent. If your SSM Agent isn't the correct version, you might see errors that include the following messages:
```
no latest version found for package AmazonCloudWatchAgent on platform linux
```
```
failed to download installation package reliably
```
For information about updating your SSM Agent version, see [Installing and Configuring SSM Agent](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html) in the *AWS Systems Manager User Guide*.
## The CloudWatch agent won't start
If the CloudWatch agent fails to start, there might be an issue in your configuration. Configuration information is logged in the `configuration-validation.log` file. This file is located in `/opt/aws/amazon-cloudwatch-agent/logs/configuration-validation.log` on Linux servers and in `$Env:ProgramData\Amazon\AmazonCloudWatchAgent\Logs\configuration-validation.log` on servers running Windows Server.
## Verify that the CloudWatch agent is running
You can query the CloudWatch agent to find whether it's running or stopped. You can use AWS Systems Manager to do this remotely. You can also use the command line, but only to check the local server.
**To query the status of the CloudWatch agent using Run Command**
1. Open the Systems Manager console at [https://console.aws.amazon.com/systems-manager/](https://console.aws.amazon.com/systems-manager/).
1. In the navigation pane, choose **Run Command**.
-or-
If the AWS Systems Manager home page opens, scroll down and choose **Explore Run Command**.
1. Choose **Run command**.
1. In the **Command document** list, choose the button next to **AmazonCloudWatch-ManageAgent**.
1. In the **Action** list, choose **status**.
1. For **Optional Configuration Source** choose **default** and keep **Optional Configuration Location** blank.
1. In the **Target** area, choose the instance to check.
1. Choose **Run**.
If the agent is running, the output resembles the following.
```
{
"status": "running",
"starttime": "2017-12-12T18:41:18",
"version": "1.73.4"
}
```
If the agent is stopped, the `"status"` field displays `"stopped"`.
**To query the status of the CloudWatch agent locally using the command line**
+ On a Linux server, enter the following:
```
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
```
On a server running Windows Server, enter the following in PowerShell as an administrator:
```
& $Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1 -m ec2 -a status
```
## The CloudWatch agent won't start, and the error mentions an Amazon EC2 Region
If the agent doesn't start and the error message mentions an Amazon EC2 Region endpoint, you might have configured the agent to need access to the Amazon EC2 endpoint without granting that access.
For example, if you specify a value for the `append_dimensions` parameter in the agent configuration file that depends on Amazon EC2 metadata and you use proxies, you must make sure that the server can access the endpoint for Amazon EC2. For more information about these endpoints, see [Amazon Elastic Compute Cloud (Amazon EC2)](https://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region) in the *Amazon Web Services General Reference*.
## The CloudWatch agent won't start on Windows Server
On Windows Server, you might see the following error:
```
Start-Service : Service 'Amazon CloudWatch Agent (AmazonCloudWatchAgent)' cannot be started due to the following
error: Cannot start service AmazonCloudWatchAgent on computer '.'.
At C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1:113 char:12
+ $svc | Start-Service
+ ~~~~~~~~~~~~~
+ CategoryInfo : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service],
ServiceCommandException
+ FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand
```
To fix this, first make sure that the server service is running. This error can be seen if the agent tries to start when the server service isn't running.
If the server service is already running, the following may be the issue. On some Windows Server installations, the CloudWatch agent takes more than 30 seconds to start. Because Windows Server, by default, allows only 30 seconds for services to start, this causes the agent to fail with an error similar to the following:
To fix this issue, increase the service timeout value. For more information, see [ A service does not start, and events 7000 and 7011 are logged in the Windows event log](https://support.microsoft.com/en-us/help/922918/a-service-does-not-start-and-events-7000-and-7011-are-logged-in-window).
## Where are the metrics?
If the CloudWatch agent has been running but you can't find metrics collected by it in the AWS Management Console or the AWS CLI, confirm that you're using the correct namespace. By default, the namespace for metrics collected by the agent is `CWAgent`. You can customize this namespace using the `namespace` field in the `metrics` section of the agent configuration file. If you don't see the metrics that you expect, check the configuration file to confirm the namespace being used.
When you first download the CloudWatch agent package, the agent configuration file is `amazon-cloudwatch-agent.json`. This file is in the directory where you ran the configuration wizard, or you might have moved it to a different directory. If you use the configuration wizard, the agent configuration file output from the wizard is named `config.json`. For more information about the configuration file, including the `namespace` field, see [CloudWatch agent configuration file: Metrics section](CloudWatch-Agent-Configuration-File-Details.md#CloudWatch-Agent-Configuration-File-Metricssection).
## The CloudWatch agent takes a long time to run in a container or logs a hop limit error
When you run the CloudWatch agent as a container service and want to add Amazon EC2 metric dimensions to all metrics collected by the agent, you might see the following errors in version v1.247354.0 of the agent:
```
2022-06-07T03:36:11Z E! [processors.ec2tagger] ec2tagger: Unable to retrieve Instance Metadata Tags. This plugin must only be used on an EC2 instance.
2022-06-07T03:36:11Z E! [processors.ec2tagger] ec2tagger: Please increase hop limit to 2 by following this document https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-options.html#configuring-IMDS-existing-instances.
2022-06-07T03:36:11Z E! [telegraf] Error running agent: could not initialize processor ec2tagger: EC2MetadataRequestError: failed to get EC2 instance identity document
caused by: EC2MetadataError: failed to make EC2Metadata request
status code: 401, request id:
caused by:
```
You might see this error if the agent tries to get metadata from IMDSv2 inside a container without an appropriate hop limit. In versions of the agent earlier than v1.247354.0, you can experience this issue without seeing the log message.
To solve this, increase the hop limit to 2 by following the instructions in [ Configure the instance metadata options](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-options.html#configuring-IMDS-existing-instances.).
## I updated my agent configuration but don’t see the new metrics or logs in the CloudWatch console
If you update your CloudWatch agent configuration file, the next time that you start the agent, you need to use the **fetch-config** option. For example, if you stored the updated file on the local computer, enter the following command:
```
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -s -m ec2 -c file:configuration-file-path
```
## CloudWatch agent files and locations
The following table lists the files installed by and used with the CloudWatch agent, along with their locations on servers running Linux or Windows Server.
| File | Linux location | Windows Server location |
| --- | --- | --- |
| The control script that controls starting, stopping, and restarting the agent. | `/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl` or `/usr/bin/amazon-cloudwatch-agent-ctl` | `$Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1` |
| The log file the agent writes to. You might need to attach this when contacting AWS Support. | `/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log` or `/var/log/amazon/amazon-cloudwatch-agent/amazon-cloudwatch-agent.log` | `$Env:ProgramData\Amazon\AmazonCloudWatchAgent\Logs\amazon-cloudwatch-agent.log` |
| Agent configuration validation file. | `/opt/aws/amazon-cloudwatch-agent/logs/configuration-validation.log` or `/var/log/amazon/amazon-cloudwatch-agent/configuration-validation.log` | `$Env:ProgramData\Amazon\AmazonCloudWatchAgent\Logs\configuration-validation.log` |
| The JSON file used to configure the agent immediately after the wizard creates it. For more information, see [Create the CloudWatch agent configuration file](create-cloudwatch-agent-configuration-file.md). | `/opt/aws/amazon-cloudwatch-agent/bin/config.json` | `$Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\config.json` |
| The JSON file used to configure the agent if this configuration file has been downloaded from Parameter Store. | `/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json` or `/etc/amazon/amazon-cloudwatch-agent/amazon-cloudwatch-agent.json` | `$Env:ProgramData\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent.json` |
| The TOML file used to specify Region and credential information to be used by the agent, overriding system defaults. | `/opt/aws/amazon-cloudwatch-agent/etc/common-config.toml` or `/etc/amazon/amazon-cloudwatch-agent/common-config.toml` | `$Env:ProgramData\Amazon\AmazonCloudWatchAgent\common-config.toml` |
| The TOML file that contains the converted contents of the JSON configuration file. The `amazon-cloudwatch-agent-ctl` script generates this file. Users should not directly modify this file. It can be useful for verifying that JSON to TOML translation was successful. | `/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml` or `/etc/amazon/amazon-cloudwatch-agent/amazon-cloudwatch-agent.toml` | `$Env:ProgramData\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent.toml` |
| The YAML file that contains the converted contents of the JSON configuration file. The `amazon-cloudwatch-agent-ctl` script generates this file. You should not directly modify this file. This file can be useful for verifying that the JSON to YAML translation was successful. | `/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.yaml or /etc/amazon/amazon-cloudwatch-agent/amazon-cloudwatch-agent.yaml` | `$Env:ProgramData\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent.yaml` |
## Finding information about CloudWatch agent versions
To find the version number of the CloudWatch agent on a Linux server, enter the following command:
```
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a status
```
To find the version number of the CloudWatch agent on Windows Server, enter the following command:
```
& $Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1 -m ec2 -a status
```
**Note**
Using this command is the correct way to find the version of the CloudWatch agent. If you use **Programs and Features** in the Control Panel, you will see an incorrect version number.
You can also download a README file about the latest changes to the agent, and a file that indicates the version number that is currently available for download. These files are in the following locations:
+ `https://amazoncloudwatch-agent.s3.amazonaws.com/info/latest/RELEASE_NOTES` or `https://amazoncloudwatch-agent-us-east-1.s3.us-east-1.amazonaws.com/info/latest/RELEASE_NOTES`
+ `https://amazoncloudwatch-agent.s3.amazonaws.com/info/latest/CWAGENT_VERSION` or `https://amazoncloudwatch-agent-us-east-1.s3.us-east-1.amazonaws.com/info/latest/CWAGENT_VERSION`
## Logs generated by the CloudWatch agent
The agent generates a log while it runs. This log includes troubleshooting information. This log is the `amazon-cloudwatch-agent.log` file. This file is located in `/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log` on Linux servers and in `$Env:ProgramData\Amazon\AmazonCloudWatchAgent\Logs\amazon-cloudwatch-agent.log` on servers running Windows Server.
You can configure the agent to log additional details in the `amazon-cloudwatch-agent.log` file. In the agent configuration file, in the `agent` section, set the `debug` field to `true`, then reconfigure and restart the CloudWatch agent. To disable the logging of this extra information, set the `debug` field to `false`. Then, reconfigure and restart the agent. For more information, see [Manually create or edit the CloudWatch agent configuration file](CloudWatch-Agent-Configuration-File-Details.md).
In versions 1.247350.0 and later of the CloudWatch agent, you can optionally set the `aws_sdk_log_level` field in the `agent` section of the agent configuration file to one or more of the following options. Separate multiple options with the `|` character.
+ `LogDebug`
+ `LogDebugWithSigning`
+ `LogDebugWithHTTPBody`
+ `LogDebugRequestRetries`
+ `LogDebugWithEventStreamBody`
For more information about these options, see [LogLevelType](https://docs.aws.amazon.com/sdk-for-go/api/aws/#LogLevelType).
## Stopping and restarting the CloudWatch agent
You can manually stop the CloudWatch agent using either AWS Systems Manager or the command line.
**To stop the CloudWatch agent using Run Command**
1. Open the Systems Manager console at [https://console.aws.amazon.com/systems-manager/](https://console.aws.amazon.com/systems-manager/).
1. In the navigation pane, choose **Run Command**.
-or-
If the AWS Systems Manager home page opens, scroll down and choose **Explore Run Command**.
1. Choose **Run command**.
1. In the **Command document** list, choose **AmazonCloudWatch-ManageAgent**.
1. In the **Targets** area, choose the instance where you installed the CloudWatch agent.
1. In the **Action** list, choose **stop**.
1. Keep **Optional Configuration Source** and **Optional Configuration Location** blank.
1. Choose **Run**.
**To stop the CloudWatch agent locally using the command line**
+ On a Linux server, enter the following:
```
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a stop
```
On a server running Windows Server, enter the following in PowerShell as an administrator:
```
& $Env:ProgramFiles\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1 -m ec2 -a stop
```
To restart the agent, follow the instructions in [(Optional) Modify the common configuration and named profile for CloudWatch agent](installing-cloudwatch-agent-ssm.md#CloudWatch-Agent-profile-instance-fleet).