# Amazon ECS Managed Daemons
Amazon ECS Managed Daemons enable you to deploy and manage software agents, such as security, observability, and networking agents, across your container infrastructure on Amazon ECS Managed Instances. Managed Daemons decouple daemon lifecycle management from application operations. You can deploy, update, and monitor agents independently, without redeploying workloads or coordinating changes across services.
## How Managed Daemons work
To use Managed Daemons, first register a daemon task definition. A daemon task definition is a template that describes the containers that form a daemon. After you register a daemon task definition, create a daemon and associate it with a cluster and one or more Amazon ECS Managed Instances capacity providers. Amazon ECS then ensures that exactly one daemon task runs on every Amazon EC2 instance provisioned through those capacity providers.
Daemons do not launch instances independently. When you run an application task on a Amazon ECS Managed Instances capacity provider, Amazon ECS provisions an Amazon EC2 instance, starts the daemon task first, and only then transitions the application task to `RUNNING`. This ordering guarantees that cross-cutting functions like logging, tracing, and metrics collection are operational before your application begins processing requests.
Daemons are essential for instance health. If a daemon task stops, Amazon ECS automatically drains and replaces that container instance. This auto-repair behavior ensures reliable daemon coverage across all instances without manual intervention.
When you update a daemon to a new task definition revision, Amazon ECS performs a rolling deployment across all instances in the associated capacity providers. During the deployment, Amazon ECS drains a configurable percentage of instances simultaneously, provisions replacement instances with the updated daemon, and replaces your Amazon ECS service tasks automatically. Amazon ECS provides built-in circuit breaker protection. You can configure a bake time and CloudWatch alarms so that Amazon ECS monitors the deployment after it updates all instances and automatically rolls back if issues arise.
## Key benefits
+ **Decoupled lifecycle management** - Update daemons independently from application deployments.
+ **Guaranteed coverage** - Amazon ECS ensures daemon tasks start before application tasks on every instance, so cross-cutting functions are always available.
+ **Reliable version updates** - When you update a daemon version, Amazon ECS rolls it out across all instances in the associated capacity providers, with built-in circuit breaker protection and automatic rollback to ensure every instance runs the target revision.
+ **Improved resource utilization** - Running a single daemon task per instance eliminates the sidecar-per-task model, reducing resource overhead across your cluster.
+ **Automatic instance repair** - If a daemon task stops or becomes unhealthy, Amazon ECS automatically drains and replaces that container instance. This maintains reliable daemon coverage without manual intervention.
**Note**
Amazon ECS offers a DAEMON scheduling strategy for Amazon ECS services for the Amazon EC2 launch type. Managed Daemons is a new capability built for Amazon ECS Managed Instances to simplify daemon deployments and provide stronger daemon coverage guarantees.
## Getting started
To get started, register a daemon task definition specifying your container image, then create a daemon with associated capacity providers in your cluster. You can use the AWS Management Console, AWS CLI, CloudFormation, or AWS SDKs. For step-by-step instructions, see [Creating and managing daemons](managed-daemons-create-manage.md).
# Daemon task definitions
A daemon task definition is the blueprint for your daemon. Amazon ECS Managed Daemons use a dedicated daemon task definition resource, distinct from standard Amazon ECS task definitions. You register a daemon task definition to specify the container image, resource requirements, and configuration for your daemon.
## Creating a daemon task definition
You can create a daemon task definition by using the AWS Management Console or the AWS CLI.
### Using the AWS Management Console
1. Open the Amazon ECS console. In the left navigation, choose **Daemon task definitions**.
1. Choose **Create new daemon task definition**.
1. For **Daemon task definition family**, enter a unique name. The name can contain up to 255 alphanumeric characters, hyphens, and underscores.
1. (Optional) For **Task role**, choose an IAM role that grants permissions to the applications running in your containers. Leave this blank if your containers do not call AWS APIs.
1. (Optional) For **Task execution role**, select `ecsTaskExecutionRole`. Amazon ECS requires this role to pull container images and publish logs.
1. (Optional) For **Task size**, specify the CPU and memory to reserve for your daemon. For example, `0.25 vCPU` and `512 MB` memory.
1. Configure your container details:
+ **Container name** - Enter a name for your container. The name can contain up to 255 alphanumeric characters, hyphens, and underscores.
+ **Essential container** - Select **Yes**. Each daemon task definition requires at least one essential container. If an essential container fails, Amazon ECS stops the entire task.
+ **Image URI** - Enter the Docker image URI. You can browse Amazon ECR images or use public images from Docker Hub or other registries.
1. (Optional) Configure resource allocation, health check, environment variables, log collection, and tags as needed.
1. Review your configuration and choose **Create**.
### Using the AWS CLI
Register a daemon task definition by creating a JSON file and using the `register-daemon-task-definition` command.
The following is an example JSON file:
```
{
"family": "my-daemon-task",
"containerDefinitions": [
{
"name": "daemon-container",
"image": "public.ecr.aws/docker/library/busybox:latest",
"essential": true,
"command": ["sh", "-c", "while true; do echo 'Daemon running'; sleep 30; done"],
"memoryReservation": 512
}
]
}
```
Run the following command to register the daemon task definition:
```
aws ecs register-daemon-task-definition --cli-input-json file://daemon-taskdef.json
```
## Daemon task definition parameters
You can use the following parameters when you register a daemon task definition.
**Required parameters**
+ `family` - The task definition family name. This groups multiple revisions of the same daemon task definition.
+ `containerDefinitions` - An array of `DaemonContainerDefinition` objects that describe the containers in your daemon task.
**Optional parameters**
+ `cpu` - Task-level CPU units as a string (for example, `"256"`).
+ `memory` - Task-level memory in MiB as a string (for example, `"512"`).
+ `taskRoleArn` - The ARN of the IAM role that grants permissions to the containers in your daemon task.
+ `executionRoleArn` - The ARN of the IAM role that the Amazon ECS container agent uses to make AWS API calls on your behalf (for example, pulling images from Amazon ECR).
+ `volumes` - An array of `DaemonVolume` objects. Daemon volumes support only bind mounts with `host` and `sourcePath`. Daemon task definitions do not support Amazon EBS, Amazon EFS, FSx for Windows File Server, Docker volumes, or ephemeral storage.
+ `tags` - An array of key-value pairs to tag your daemon task definition.
## Network mode
Daemons use a special `daemon_bridge` network mode that Amazon ECS sets automatically. You cannot specify a network mode in your daemon task definition. All daemons on an instance share a single network namespace and are accessible locally via a static daemon bridge IP address (`169.254.172.2` for IPv4, or `fd00:ec2::172:2` for IPv6). Application tasks (non-daemons) can communicate with daemons at this address without additional network configuration.
Because daemons run in a separate network namespace from application tasks, non-daemon tasks don't need to worry about port conflicts with daemons. However, since all daemon tasks share the same namespace, daemons on the same instance cannot bind to the same port. When deploying multiple daemons, ensure each daemon uses a unique port.
## Privileged capabilities
On Amazon ECS Managed Instances, daemons support privileged Linux capabilities for system-level operations. Security agents, network monitoring tools, and observability agents often require kernel-level access to function correctly.
You can grant capabilities in two ways:
**Full privileged mode** grants all Linux capabilities to the container. Use this when your agent requires broad system access:
```
{
"containerDefinitions": [{
"name": "security-daemon",
"image": "my-security-agent:latest",
"privileged": true
}]
}
```
**Individual capabilities** grant only the permissions your daemon needs, following the principle of least privilege. Use the `linuxParameters.capabilities` field to add individual capabilities such as `SYS_ADMIN`, `NET_ADMIN`, `SYS_PTRACE`, and `BPF`:
```
{
"containerDefinitions": [{
"name": "monitoring-daemon",
"image": "my-monitoring-agent:latest",
"linuxParameters": {
"capabilities": {
"add": ["NET_ADMIN", "SYS_PTRACE"]
}
}
}]
}
```
## Volumes
Managed Daemons support bind mounts using host volumes with a `sourcePath` specification. This allows daemon containers to mount host directories to access logs, metrics, and system information on the underlying Amazon EC2 instance. Log collection agents, metrics exporters, and security scanners commonly use this to gain visibility into host-level data.
The daemon task definition supports host volume specifications with `sourcePath`. Daemon volumes persist for the lifecycle of the underlying container instance.
**Note**
Amazon EBS volumes, Amazon EFS volumes, FSx for Windows File Server volumes, Docker volumes, and ephemeral storage are not supported in daemon task definitions.
**Note**
Daemons do not support volume sharing between each other. Each daemon operates with its own independent volume configuration.
The following example shows a log collector daemon with a bind mount:
```
{
"containerDefinitions": [{
"name": "log-collector",
"image": "fluent/fluentd:latest",
"mountPoints": [{
"sourceVolume": "var-log",
"containerPath": "/var/log"
}]
}],
"volumes": [{
"name": "var-log",
"host": {
"sourcePath": "/var/log"
}
}]
}
```
## Supported container parameters
Daemon container definitions support the following parameters:
+ `image` (required) - Container image URI
+ `name` (required) - Container name
+ `cpu` - CPU units reserved for the container
+ `memory` / `memoryReservation` - Hard and soft memory limits
+ `essential` - Whether the container is essential (default: true)
+ `command` / `entryPoint` - Override the container command or entry point
+ `environment` / `environmentFiles` / `secrets` - Environment configuration
+ `privileged` - Run the container in privileged mode
+ `user` - User to run the container as
+ `workingDirectory` - Working directory inside the container
+ `readonlyRootFilesystem` - Mount the root filesystem as read-only
+ `mountPoints` - Volume mount points (bind mounts with host and sourcePath only)
+ `logConfiguration` - Logging configuration (supports awslogs, splunk, awsfirelens)
+ `healthCheck` - Container health check configuration
+ `dependsOn` - Container startup dependencies
+ `ulimits` - Resource limits
+ `systemControls` - Kernel parameters
+ `linuxParameters.capabilities` - Linux capabilities to add or drop
+ `linuxParameters.initProcessEnabled` - Run an init process in the container
+ `repositoryCredentials` - Credentials for private registries
+ `restartPolicy` - Container restart policy
# Creating and managing daemons
After you register a daemon task definition, create a daemon to deploy it across your Amazon ECS Managed Instances capacity providers. Amazon ECS automatically places one daemon task on every Amazon EC2 instance in the associated capacity providers and manages the daemon lifecycle.
## Prerequisites
Before you create a daemon, verify that you have the following:
+ An Amazon ECS cluster.
+ One or more Amazon ECS Managed Instances capacity providers associated with the cluster.
+ A registered daemon task definition.
## Creating a daemon
### AWS Management Console
1. Open the Amazon ECS console. In the left navigation, choose **Clusters**, then select your cluster.
1. Choose the **Daemons** tab, then choose **Create**.
1. For **Daemon task definition family**, select your daemon task definition from the dropdown.
1. For **Daemon task definition revision**, select the revision to use. Leave blank to use the latest revision.
1. For **Daemon name**, enter a unique name. The name can contain up to 255 alphanumeric characters, hyphens, and underscores.
1. For **Capacity providers**, select one or more Amazon ECS Managed Instances capacity providers. These determine which instances run your daemon tasks.
1. (Optional) Configure deployment settings:
+ **Drain percentage** - Percentage of instances to drain simultaneously during updates. Default: `25`.
+ **Use CloudWatch alarm(s)** - Turn on to monitor deployment health and automatically roll back if alarms trigger.
+ **Bake time** - The number of minutes that Amazon ECS waits after it updates all instances to the new daemon revision before it completes the deployment. During this period, Amazon ECS monitors CloudWatch alarms and automatically rolls back the deployment if any alarm triggers. Default: `0`.
1. (Optional) Add tags.
1. (Optional) Turn on **Enable ECS Exec** to run interactive commands in your daemon containers for troubleshooting.
1. Review your configuration and choose **Create**.
### AWS CLI
Create a JSON file with your daemon configuration and run the `create-daemon` command.
The following is an example JSON file:
```
{
"clusterArn": "arn:aws:ecs:us-east-1:123456789012:cluster/my-daemon-cluster",
"daemonName": "my-monitoring-daemon",
"daemonTaskDefinitionArn": "arn:aws:ecs:us-east-1:123456789012:daemon-task-definition/my-daemon-task:1",
"capacityProviderArns": [
"arn:aws:ecs:us-east-1:123456789012:capacity-provider/my-daemon-capacity-provider"
]
}
```
Run the following command to create the daemon:
```
aws ecs create-daemon --cli-input-json file://create-daemon.json
```
**Required fields**
+ `daemonName` - A unique name for the daemon.
+ `clusterArn` - The ARN of the cluster.
+ `daemonTaskDefinitionArn` - The ARN of the daemon task definition.
+ `capacityProviderArns` - An array of Amazon ECS Managed Instances capacity provider ARNs.
**Optional fields**
+ `deploymentConfiguration` - A `DaemonDeploymentConfiguration` object to customize deployment behavior.
+ `tags` - Key-value pairs for tagging.
+ `propagateTags` - Tag propagation setting.
+ `clientToken` - An idempotency token.
## Verifying daemon deployment
After you create a daemon, verify its status by using the AWS Management Console or the AWS CLI.
### AWS Management Console
1. Open the Amazon ECS console. In the left navigation, choose **Clusters**, then select your cluster.
1. Choose the **Daemons** tab.
1. Verify that your daemon shows **Active** status.
1. Choose the **Tasks** tab to confirm that one daemon task runs on each container instance.
### AWS CLI
Run the following commands to verify daemon status:
```
aws ecs list-daemons \
--cluster-arn arn:aws:ecs:us-east-1:123456789012:cluster/my-daemon-cluster
```
```
aws ecs describe-daemons \
--daemon-arn arn:aws:ecs:us-east-1:123456789012:daemon/my-daemon-cluster/my-monitoring-daemon
```
## Updating a daemon
To update a daemon with a new task definition revision or to add capacity providers, use the AWS Management Console or the AWS CLI. This triggers a rolling deployment across all instances.
### AWS Management Console
1. Open the Amazon ECS console. In the left navigation, choose **Clusters**, then select your cluster.
1. Choose the **Daemons** tab, then select the daemon you want to update.
1. Choose **Update**.
1. For **Daemon task definition revision**, select the new revision.
1. (Optional) Update the capacity providers or deployment settings.
1. Choose **Update** to start the rolling deployment.
### AWS CLI
Run the `update-daemon` command:
```
aws ecs update-daemon \
--daemon-arn arn:aws:ecs:us-east-1:123456789012:daemon/my-daemon-cluster/my-monitoring-daemon \
--daemon-task-definition-arn arn:aws:ecs:us-east-1:123456789012:daemon-task-definition/my-daemon-task:2 \
--capacity-provider-arns arn:aws:ecs:us-east-1:123456789012:capacity-provider/my-daemon-capacity-provider
```
**Important**
When you provide daemon configuration settings in an `UpdateDaemon` request, Amazon ECS uses your specified settings instead of the defaults. Daemon configuration settings, including tags and the enable execute command flag, are not persisted between updates. Each call to `UpdateDaemon` must include the full set of configuration settings you want applied. Any settings omitted from the request revert to their default values.
## Deleting a daemon
To delete a daemon, use the AWS Management Console or the AWS CLI. Wait for all daemon tasks to stop before you delete the capacity provider or cluster.
### AWS Management Console
1. Open the Amazon ECS console. In the left navigation, choose **Clusters**, then select your cluster.
1. Choose the **Daemons** tab, then select the daemon you want to delete.
1. Choose **Delete**.
1. In the confirmation dialog, choose **Delete** to confirm.
### AWS CLI
Run the `delete-daemon` command:
```
aws ecs delete-daemon \
--daemon-arn arn:aws:ecs:us-east-1:123456789012:daemon/my-daemon-cluster/my-monitoring-daemon
```
# Daemon deployments
When you create or update a daemon, Amazon ECS performs a rolling deployment to run daemon tasks on all instances in the associated capacity providers. This section describes how daemon deployments work and how to configure deployment behavior.
## How daemon deployments work
A daemon deployment follows a drain-provision-replace pattern. When you create a daemon on a capacity provider with existing instances, or update a daemon to a new task definition revision, Amazon ECS drains a batch of instances, provisions replacement instances with the updated daemon, and replaces application tasks automatically. This process repeats until all instances run the new daemon revision.
The deployment lifecycle transitions through the following states:
1. `PENDING` - Amazon ECS has created the deployment and is preparing to begin.
1. `IN_PROGRESS` - Amazon ECS is actively draining instances and provisioning replacements.
1. `SUCCESSFUL` - All instances are running the target daemon revision.
1. `STOPPED` - Amazon ECS has stopped the deployment because it failed or was replaced by a new deployment. The daemon may be in a mixed state where some instances run the old revision and others run the new revision.
Amazon ECS automatically rolls back to the previous daemon revision if the deployment circuit breaker detects failures or if a CloudWatch alarm triggers during the deployment.
## Deployment configuration parameters
You can customize deployment behavior by using the `deploymentConfiguration` parameter when you create or update a daemon.
+ `drainPercent` (1.0–100.0) - The percentage of instances to drain simultaneously during the deployment. Higher values speed up deployments but may temporarily reduce available capacity. For example, a value of `20.0` drains 20% of instances at a time. If not specified, the default is `25.0`.
+ `alarms` (`DaemonAlarmConfiguration`) - CloudWatch alarms to monitor during the deployment. Amazon ECS evaluates the specified alarms during the deployment and automatically rolls back if any alarm enters the `ALARM` state. Amazon ECS ignores alarms that are already in the `ALARM` state when the deployment begins.
+ `bakeTimeInMinutes` (0–1440) - The number of minutes that Amazon ECS waits after it updates all instances to the new daemon revision before it completes the deployment. During this period, Amazon ECS monitors CloudWatch alarms and automatically rolls back the deployment if any alarm triggers. If not specified, the default is `0`.
**Example with deployment configuration:**
```
aws ecs create-daemon \
--cluster-arn arn:aws:ecs:us-east-1:123456789012:cluster/my-daemon-cluster \
--daemon-name my-monitoring-daemon \
--daemon-task-definition-arn arn:aws:ecs:us-east-1:123456789012:daemon-task-definition/my-daemon-task:1 \
--capacity-provider-arns arn:aws:ecs:us-east-1:123456789012:capacity-provider/my-daemon-capacity-provider \
--deployment-configuration '{"drainPercent":20.0,"bakeTimeInMinutes":5}'
```
## Monitoring deployments
Track deployment progress by using the following commands:
```
aws ecs list-daemon-deployments \
--daemon-arn arn:aws:ecs:us-east-1:123456789012:daemon/my-daemon-cluster/my-monitoring-daemon
```
```
aws ecs describe-daemon-deployments \
--daemon-deployment-arns arn:aws:ecs:us-east-1:123456789012:daemon-deployment/my-daemon-cluster/abc123
```
The `describe-daemon-deployments` response includes the deployment status, the target daemon revision, and the `totalRunningInstanceCount` field that indicates how many instances run the target daemon.
## Daemon deployment scenarios
This section walks through common deployment scenarios for Managed Daemons.
+ [Deploy a daemon on an empty capacity provider](#managed-daemons-scenario-empty)
+ [Deploy a daemon on a capacity provider with existing instances](#managed-daemons-scenario-existing)
+ [Add a capacity provider to an existing daemon](#managed-daemons-scenario-add-cp)
+ [Update a daemon to a new revision](#managed-daemons-scenario-update)
### Deploy a daemon on an empty capacity provider
In this scenario, you deploy a daemon on a capacity provider with no existing instances. Instances launch when you schedule application tasks.
**Prerequisites:** A cluster and a Amazon ECS Managed Instances capacity provider with no running instances.
1. Register your daemon task definition.
1. Create the daemon. Amazon ECS completes the creation immediately even though no instances exist yet.
```
aws ecs create-daemon \
--cluster-arn arn:aws:ecs:us-east-1:123456789012:cluster/my-daemon-cluster \
--daemon-name my-monitoring-daemon \
--daemon-task-definition-arn arn:aws:ecs:us-east-1:123456789012:daemon-task-definition/my-daemon-task:1 \
--capacity-provider-arns arn:aws:ecs:us-east-1:123456789012:capacity-provider/my-daemon-capacity-provider
```
1. Create a service that uses the same capacity provider. When Amazon ECS provisions an instance for the service, it starts the daemon task first, then transitions the application task to `RUNNING`.
```
aws ecs create-service \
--cluster my-daemon-cluster \
--service-name my-app-service \
--task-definition my-app-task \
--desired-count 2 \
--capacity-provider-strategy capacityProvider=my-daemon-capacity-provider,weight=1
```
1. Verify that the daemon task is running on each instance by using the `describe-daemons` command or by checking the **Daemons** tab in the console.
**Note**
The daemon deployment may remain in `PENDING` state briefly while Amazon ECS provisions instances and starts daemon tasks. Wait for the deployment to reach `SUCCESSFUL` before verifying daemon task placement.
### Deploy a daemon on a capacity provider with existing instances
In this scenario, you deploy a daemon on a capacity provider that already has running instances and application tasks.
**Prerequisites:** A cluster with a Amazon ECS Managed Instances capacity provider that has running instances.
1. Register your daemon task definition.
1. Create the daemon. Amazon ECS starts a rolling deployment to place daemon tasks on all existing instances.
```
aws ecs create-daemon \
--cluster-arn arn:aws:ecs:us-east-1:123456789012:cluster/my-daemon-cluster \
--daemon-name my-monitoring-daemon \
--daemon-task-definition-arn arn:aws:ecs:us-east-1:123456789012:daemon-task-definition/my-daemon-task:1 \
--capacity-provider-arns arn:aws:ecs:us-east-1:123456789012:capacity-provider/my-daemon-capacity-provider \
--deployment-configuration '{"drainPercent":25.0,"bakeTimeInMinutes":3}'
```
Amazon ECS drains a batch of existing instances (based on the `drainPercent`), provisions replacement instances with the daemon, and replaces application tasks. This process repeats until all instances run the daemon.
1. Monitor the deployment progress:
```
aws ecs list-daemon-deployments \
--daemon-arn arn:aws:ecs:us-east-1:123456789012:daemon/my-daemon-cluster/my-monitoring-daemon
```
### Add a capacity provider to an existing daemon
In this scenario, you add a second capacity provider to an existing daemon. The daemon automatically deploys to instances in the new capacity provider.
**Prerequisites:** A daemon running on a first capacity provider, and a second capacity provider created and associated with the cluster.
1. Update the daemon to include both capacity providers:
```
aws ecs update-daemon \
--daemon-arn arn:aws:ecs:us-east-1:123456789012:daemon/my-daemon-cluster/my-monitoring-daemon \
--daemon-task-definition-arn arn:aws:ecs:us-east-1:123456789012:daemon-task-definition/my-daemon-task:1 \
--capacity-provider-arns \
arn:aws:ecs:us-east-1:123456789012:capacity-provider/my-daemon-capacity-provider \
arn:aws:ecs:us-east-1:123456789012:capacity-provider/my-daemon-capacity-provider-2
```
1. Monitor the deployment and verify that daemon tasks run on instances in both capacity providers.
### Update a daemon to a new revision
In this scenario, you update an existing daemon to use a new task definition revision.
1. Register a new revision of your daemon task definition with the updated container image or configuration.
1. Update the daemon to use the new revision:
```
aws ecs update-daemon \
--daemon-arn arn:aws:ecs:us-east-1:123456789012:daemon/my-daemon-cluster/my-monitoring-daemon \
--daemon-task-definition-arn arn:aws:ecs:us-east-1:123456789012:daemon-task-definition/my-daemon-task:2
```
Amazon ECS performs a rolling deployment. It drains instances running the old revision, provisions replacement instances with the new revision, and replaces application tasks automatically. If the circuit breaker detects failures, Amazon ECS rolls back to the previous revision.
1. Monitor the deployment and verify that all instances run the new revision:
```
aws ecs describe-daemon-deployments \
--daemon-deployment-arns arn:aws:ecs:us-east-1:123456789012:daemon-deployment/my-daemon-cluster/deployment-id
```
# Daemon events
Amazon ECS sends daemon events to to provide visibility into daemon task management and deployment lifecycle changes. You can use these events to monitor daemon health, track deployment progress, and trigger automated workflows.
Amazon ECS supports the following daemon event types:
+ [Daemon service action events](#managed-daemons-service-action-events) - Events related to daemon task placement and startup issues.
+ [Daemon deployment state change events](#managed-daemons-deployment-state-change-events) - Events related to daemon deployment lifecycle transitions.
To learn how to create rules for these events, see [Creating rules for daemon events](#managed-daemons-eventbridge-rules).
## Daemon service action events
Amazon ECS sends daemon service action events with the detail type `ECS Daemon Service Action`. These events notify you when Amazon ECS encounters issues starting daemon tasks on your container instances.
### DAEMON\$1TASK\$1START\$1IMPAIRED
Amazon ECS sends the `DAEMON_TASK_START_IMPAIRED` event when it is unable to successfully start a daemon task on a container instance. The event includes a `failureType` field that indicates the cause of the failure:
+ `TASK_FAILED_TO_RUN` - The daemon task was created but failed to reach `RUNNING` status. Common causes include container image pull failures, container health check failures, or essential container exits. The `taskArn` field is present in the event.
+ `TASK_CREATION_FAILED` - The daemon task could not be created on the container instance. Common causes include insufficient CPU, memory, or other resources on the instance. The `taskArn` field is not present in the event because no task was created.
### Example: TASK\$1FAILED\$1TO\$1RUN event
The following event shows a daemon task that was created but failed to reach `RUNNING` status due to a container image pull failure.
```
{
"version": "0",
"id": "12345678-1234-1234-1234-123456789012",
"detail-type": "ECS Daemon Service Action",
"source": "aws.ecs",
"account": "123456789012",
"time": "2026-03-24T12:00:00Z",
"region": "us-west-2",
"resources": [
"arn:aws:ecs:us-west-2:123456789012:task/my-cluster/a1b2c3d4e5f6"
],
"detail": {
"eventType": "WARNING",
"eventName": "DAEMON_TASK_START_IMPAIRED",
"clusterArn": "arn:aws:ecs:us-west-2:123456789012:cluster/my-cluster",
"containerInstanceArn": "arn:aws:ecs:us-west-2:123456789012:container-instance/my-cluster/a1b2c3d4e5f6",
"taskArn": "arn:aws:ecs:us-west-2:123456789012:task/my-cluster/a1b2c3d4e5f6",
"daemonRevisionArn": "arn:aws:ecs:us-west-2:123456789012:daemon-revision/my-cluster/my-daemon/a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"capacityProviderName": "my-capacity-provider",
"daemonArn": "arn:aws:ecs:us-west-2:123456789012:daemon/my-cluster/my-daemon",
"daemonTaskDefinitionArn": "arn:aws:ecs:us-west-2:123456789012:daemon-task-definition/my-daemon-td:1",
"failureType": "TASK_FAILED_TO_RUN",
"createdAt": "2026-03-24T12:00:00.000Z",
"reason": "Task failed to reach RUNNING status: CannotPullContainerError: pull image manifest has been retried 5 time(s)"
}
}
```
### Example: TASK\$1CREATION\$1FAILED event
The following event shows a daemon task that could not be created on the container instance due to insufficient CPU resources. The `taskArn` field is not present because no task was created.
```
{
"version": "0",
"id": "87654321-4321-4321-4321-210987654321",
"detail-type": "ECS Daemon Service Action",
"source": "aws.ecs",
"account": "123456789012",
"time": "2026-03-24T12:01:00Z",
"region": "us-west-2",
"resources": [],
"detail": {
"eventType": "WARNING",
"eventName": "DAEMON_TASK_START_IMPAIRED",
"clusterArn": "arn:aws:ecs:us-west-2:123456789012:cluster/my-cluster",
"containerInstanceArn": "arn:aws:ecs:us-west-2:123456789012:container-instance/my-cluster/b2c3d4e5f6a7",
"daemonRevisionArn": "arn:aws:ecs:us-west-2:123456789012:daemon-revision/my-cluster/my-daemon/a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"capacityProviderName": "my-capacity-provider",
"daemonArn": "arn:aws:ecs:us-west-2:123456789012:daemon/my-cluster/my-daemon",
"daemonTaskDefinitionArn": "arn:aws:ecs:us-west-2:123456789012:daemon-task-definition/my-daemon-td:1",
"failureType": "TASK_CREATION_FAILED",
"createdAt": "2026-03-24T12:01:00.000Z",
"reason": "RESOURCE:CPU - Unable to place daemon task on container instance: insufficient CPU"
}
}
```
## Daemon deployment state change events
Amazon ECS sends daemon deployment state change events with the detail type `ECS Daemon Deployment State Change`. Amazon ECS emits these events each time the state of a daemon deployment changes.
### Deployment state change event types
Amazon ECS categorizes deployment state change events by the following event types:
**INFO events**
+ `DAEMON_DEPLOYMENT_PENDING` - Amazon ECS has initiated a daemon deployment.
+ `DAEMON_DEPLOYMENT_IN_PROGRESS` - Amazon ECS has started the deployment and is actively replacing daemon tasks.
+ `DAEMON_DEPLOYMENT_SUCCESSFUL` - Amazon ECS has successfully completed the deployment with all daemon tasks running and healthy.
+ `DAEMON_DEPLOYMENT_STOPPED` - Amazon ECS has stopped the deployment. This occurs when a deployment has failed or has been replaced by a new deployment.
+ `DAEMON_DEPLOYMENT_STOP_REQUESTED` - Amazon ECS has stopped the deployment from moving forward and will now start to roll back.
+ `DAEMON_DEPLOYMENT_ROLLBACK_IN_PROGRESS` - Amazon ECS has initiated a rollback due to deployment failure or circuit breaker trigger.
+ `DAEMON_DEPLOYMENT_ROLLBACK_SUCCESSFUL` - Amazon ECS has successfully completed the deployment rollback.
**ERROR events**
+ `DAEMON_DEPLOYMENT_ROLLBACK_FAILED` - Amazon ECS was unable to complete the deployment rollback.
### Example: Deployment pending event
The following event shows a daemon deployment that Amazon ECS has initiated.
```
{
"version": "0",
"id": "3329f79b-3dca-07f8-b1c2-5fe99f0b5e87",
"detail-type": "ECS Daemon Deployment State Change",
"source": "aws.ecs",
"account": "111122223333",
"time": "2026-03-05T15:54:41Z",
"region": "us-west-2",
"resources": [
"arn:aws:ecs:us-west-2:111122223333:daemon/my-cluster/my-daemon"
],
"detail": {
"eventType": "INFO",
"eventName": "DAEMON_DEPLOYMENT_PENDING",
"clusterArn": "arn:aws:ecs:us-west-2:111122223333:cluster/my-cluster",
"daemonArn": "arn:aws:ecs:us-west-2:111122223333:daemon/my-cluster/my-daemon",
"daemonDeploymentArn": "arn:aws:ecs:us-west-2:111122223333:daemon-deployment/my-cluster/my-daemon/0EYSiB0qap8xf0N76FsbE",
"targetDaemonRevisionArn": "arn:aws:ecs:us-west-2:111122223333:daemon-revision/my-cluster/my-daemon/85707969-3732-4b6a-a37d-5cefddd7d7dd",
"updatedAt": "2026-03-05T15:54:41.618059641Z"
}
}
```
## Creating rules for daemon events
You can create rules to receive notifications when daemon events occur. For more information about creating rules, see [Creating a rule](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-create-rule.html) in the *Amazon EventBridge User Guide*.
The following example event patterns show how to filter daemon events.
### Example: Match all daemon service action events
```
{
"source": ["aws.ecs"],
"detail-type": ["ECS Daemon Service Action"]
}
```
### Example: Match a specific failure type
```
{
"source": ["aws.ecs"],
"detail-type": ["ECS Daemon Service Action"],
"detail": {
"eventName": ["DAEMON_TASK_START_IMPAIRED"],
"failureType": ["TASK_CREATION_FAILED"]
}
}
```
### Example: Match events for a specific cluster
```
{
"source": ["aws.ecs"],
"detail-type": ["ECS Daemon Service Action"],
"detail": {
"eventName": ["DAEMON_TASK_START_IMPAIRED"],
"clusterArn": ["arn:aws:ecs:us-west-2:123456789012:cluster/my-cluster"]
}
}
```
### Example: Match all deployment state change events
```
{
"source": ["aws.ecs"],
"detail-type": ["ECS Daemon Deployment State Change"]
}
```
# Daemon metrics
Amazon ECS publishes CloudWatch metrics for Managed Daemons. You can use these metrics to track CPU and memory utilization for daemon tasks across your cluster.
Daemon metrics use the same `AWS/ECS` namespace as service metrics. Amazon ECS distinguishes daemon metrics from service metrics by the `ServiceName` dimension value, which uses the prefix `daemon:` followed by the daemon name (for example, `daemon:my-daemon`).
## Daemon utilization metrics
Amazon ECS measures daemon CPU and memory utilization as the percentage of resources used by daemon tasks compared to the resources specified in the daemon task definition.
CloudWatch provides the following statistics for daemon utilization metrics:
+ **Average** - The average utilization across all daemon tasks in the cluster.
+ **Minimum** - The utilization of the daemon task with the lowest resource usage.
+ **Maximum** - The utilization of the daemon task with the highest resource usage.
### Utilization formulas
Amazon ECS calculates the Average statistic using the following formulas:
```
(Total CPU units used by daemon tasks) x 100
Daemon CPU utilization = -------------------------------------------------------------------------------
(Total CPU units specified in daemon task definition) x (Number of daemon tasks)
```
```
(Total MiB of memory used by daemon tasks) x 100
Daemon memory utilization = ------------------------------------------------------------------------------------
(Total MiB of memory specified in daemon task definition) x (Number of daemon tasks)
```
These formulas apply only to the Average statistic. The Minimum and Maximum statistics represent the individual daemon task with the lowest and highest resource utilization, respectively.
The Amazon ECS container agent samples CPU and memory usage for each running daemon task every 20 seconds. Each minute, the agent reports three samples to Amazon ECS. Amazon ECS aggregates the samples across all daemon tasks in the cluster and publishes the Average, Minimum, and Maximum values to CloudWatch as a percentage of the total resources specified in the daemon task definition.
### Example: Utilization calculation
A daemon task definition specifies 256 CPU units and 512 MiB of memory. The daemon has 2 running tasks across 2 container instances. One daemon task uses 200 CPU units and 400 MiB of memory. The other uses 100 CPU units and 256 MiB of memory. Amazon ECS reports the following metrics:
+ Average CPU utilization: ((200 \$1 100) x 100) / (256 x 2) = 58.6%
+ Average memory utilization: ((400 \$1 256) x 100) / (512 x 2) = 64.1%
## Viewing daemon metrics
You can view daemon metrics by using the AWS Management Console or the AWS CLI.
### AWS Management Console
1. Open the CloudWatch console.
1. In the left navigation, choose **Metrics**, then choose **All metrics**.
1. Choose the **AWS/ECS** namespace.
1. Choose the **ClusterName, ServiceName** dimension.
1. Filter for `ServiceName` values that begin with `daemon:` to locate your daemon metrics.
### AWS CLI
Run the `get-metric-statistics` command to retrieve daemon metrics from CloudWatch. Specify the `ServiceName` dimension with the `daemon:` prefix.
### Example: Viewing daemon CPU utilization
The following command retrieves the average CPU utilization for a daemon over a 12-hour period with 5-minute intervals.
```
aws cloudwatch get-metric-statistics \
--namespace AWS/ECS \
--metric-name CPUUtilization \
--dimensions Name=ClusterName,Value=my-cluster Name=ServiceName,Value=daemon:my-daemon \
--start-time 2026-03-24T00:00:00Z \
--end-time 2026-03-24T12:00:00Z \
--period 300 \
--statistics Average
```
### Example: Viewing daemon memory utilization
The following command retrieves the average memory utilization for a daemon over a 12-hour period with 5-minute intervals.
```
aws cloudwatch get-metric-statistics \
--namespace AWS/ECS \
--metric-name MemoryUtilization \
--dimensions Name=ClusterName,Value=my-cluster Name=ServiceName,Value=daemon:my-daemon \
--start-time 2026-03-24T00:00:00Z \
--end-time 2026-03-24T12:00:00Z \
--period 300 \
--statistics Average
```
# Daemon auto repair
Amazon ECS treats all daemons as critical to instance health. If any daemon task stops or becomes unhealthy, Amazon ECS considers the instance impaired and automatically drains and replaces it. The daemon auto repair actions are as follows:
1. Amazon ECS detects when a daemon task stops or becomes unhealthy.
1. Amazon ECS marks the instance as draining, which prevents it from accepting new application tasks.
1. Amazon ECS provisions a replacement instance and starts the daemon task on it.
1. After the daemon task reaches a healthy state, Amazon ECS schedules the application tasks from the draining instance onto the replacement.
1. Amazon ECS terminates the original instance.
**Important**
Daemon health checks are optional but highly recommended. Without a health check, Amazon ECS can only detect failures when the daemon task stops.
You can monitor daemon health using the `DescribeContainerInstances` API or `DescribeTasks` API.