# Runtime coverage and troubleshooting for Amazon EKS clusters After you enable Runtime Monitoring and install the GuardDuty security agent (add-on) for EKS either manually or through automated agent configuration, you can start assessing the coverage for your EKS clusters. **Topics** + [Reviewing coverage statistics](#reviewing-coverage-statistics-eks-runtime-monitoring) + [Coverage status change with EventBridge notifications](#eks-runtime-monitoring-coverage-status-change) + [Troubleshooting Amazon EKS runtime coverage issues](#eks-runtime-monitoring-coverage-issues-troubleshoot) ## Reviewing coverage statistics The coverage statistics for the EKS clusters associated with your own accounts or your member accounts is the percentage of the healthy EKS clusters over all EKS clusters in the selected AWS Region. The following equation represents this as: *(Healthy clusters/All clusters)\$1100* Choose one of the access methods to review the coverage statistics for your accounts. ------ #### [ Console ] + Sign in to the AWS Management Console and open the GuardDuty console at [https://console.aws.amazon.com/guardduty/](https://console.aws.amazon.com/guardduty/). + In the navigation pane, choose **Runtime Monitoring**. + Choose the **EKS clusters runtime coverage** tab. + Under the **EKS clusters runtime coverage** tab, you can view the coverage statistics aggregated by the coverage status that is available in the **Clusters list** table. + You can filter the **Clusters list** table by the following columns: + **Cluster name** + **Account ID** + **Agent management type** + **Coverage status** + **Add-on version** + If any of your EKS clusters have the **Coverage status** as **Unhealthy**, the **Issue** column may include additional information about the reason for the **Unhealthy** status. ------ #### [ API/CLI ] + Run the [ListCoverage](https://docs.aws.amazon.com/guardduty/latest/APIReference/API_ListCoverage.html) API with your own valid detector ID, Region, and service endpoint. You can filter and sort the cluster list using this API. + You can change the example `filter-criteria` with one of the following options for `CriterionKey`: + `ACCOUNT_ID` + `CLUSTER_NAME` + `RESOURCE_TYPE` + `COVERAGE_STATUS` + `ADDON_VERSION` + `MANAGEMENT_TYPE` + You can change the example `AttributeName` in `sort-criteria` with the following options: + `ACCOUNT_ID` + `CLUSTER_NAME` + `COVERAGE_STATUS` + `ISSUE` + `ADDON_VERSION` + `UPDATED_AT` + You can change the *max-results* (up to 50). + To find the `detectorId` for your account and current Region, see the **Settings** page in the [https://console.aws.amazon.com/guardduty/](https://console.aws.amazon.com/guardduty/) console, or run the [https://docs.aws.amazon.com/guardduty/latest/APIReference/API_ListDetectors.html](https://docs.aws.amazon.com/guardduty/latest/APIReference/API_ListDetectors.html) API. ``` aws guardduty --region us-east-1 list-coverage --detector-id 12abc34d567e8fa901bc2d34e56789f0 --sort-criteria '{"AttributeName": "EKS_CLUSTER_NAME", "OrderBy": "DESC"}' --filter-criteria '{"FilterCriterion":[{"CriterionKey":"ACCOUNT_ID", "FilterCondition":{"EqualsValue":"111122223333"}}] }' --max-results 5 ``` + Run the [GetCoverageStatistics](https://docs.aws.amazon.com/guardduty/latest/APIReference/API_GetCoverageStatistics.html) API to retrieve coverage aggregated statistics based on the `statisticsType`. + You can change the example `statisticsType` to one of the following options: + `COUNT_BY_COVERAGE_STATUS` – Represents coverage statistics for EKS clusters aggregated by coverage status. + `COUNT_BY_RESOURCE_TYPE` – Coverage statistics aggregated based on the type of AWS resource in the list. + You can change the example `filter-criteria` in the command. You can use the following options for `CriterionKey`: + `ACCOUNT_ID` + `CLUSTER_NAME` + `RESOURCE_TYPE` + `COVERAGE_STATUS` + `ADDON_VERSION` + `MANAGEMENT_TYPE` + To find the `detectorId` for your account and current Region, see the **Settings** page in the [https://console.aws.amazon.com/guardduty/](https://console.aws.amazon.com/guardduty/) console, or run the [https://docs.aws.amazon.com/guardduty/latest/APIReference/API_ListDetectors.html](https://docs.aws.amazon.com/guardduty/latest/APIReference/API_ListDetectors.html) API. ``` aws guardduty --region us-east-1 get-coverage-statistics --detector-id 12abc34d567e8fa901bc2d34e56789f0 --statistics-type COUNT_BY_COVERAGE_STATUS --filter-criteria '{"FilterCriterion":[{"CriterionKey":"ACCOUNT_ID", "FilterCondition":{"EqualsValue":"123456789012"}}] }' ``` ------ If the coverage status of your EKS cluster is **Unhealthy**, see [Troubleshooting Amazon EKS runtime coverage issues](#eks-runtime-monitoring-coverage-issues-troubleshoot). ## Coverage status change with EventBridge notifications The coverage status of an EKS cluster in your account may show up as **Unhealthy**. To detect when the coverage status becomes **Unhealthy**, we recommend you monitor the coverage status periodically and troubleshoot, if the status is **Unhealthy**. Alternatively, you can create an Amazon EventBridge rule to notify you when the coverage status changes from either `Unhealthy` to `Healthy` or otherwise. By default, GuardDuty publishes this in the [EventBridge bus](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-bus.html) for your account. ### Sample notification schema In an EventBridge rule, you can use the pre-defined sample events and event patterns to receive coverage status notification. For more information about creating an EventBridge rule, see [Create rule](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-get-started.html#eb-gs-create-rule) in the *Amazon EventBridge User Guide*. Additionally, you can create a custom event pattern by using the following example notification schema. Make sure to replace the values for your account. To get notified when the coverage status of your Amazon EKS cluster changes from `Healthy` to `Unhealthy`, the `detail-type` should be *GuardDuty Runtime Protection Unhealthy*. To get notified when the coverage status changes from `Unhealthy` to `Healthy`, replace the value of `detail-type` with *GuardDuty Runtime Protection Healthy*. ``` { "version": "0", "id": "event ID", "detail-type": "GuardDuty Runtime Protection Unhealthy", "source": "aws.guardduty", "account": "AWS account ID", "time": "event timestamp (string)", "region": "AWS Region", "resources": [ ], "detail": { "schemaVersion": "1.0", "resourceAccountId": "string", "currentStatus": "string", "previousStatus": "string", "resourceDetails": { "resourceType": "EKS", "eksClusterDetails": { "clusterName": "string", "availableNodes": "string", "desiredNodes": "string", "addonVersion": "string" } }, "issue": "string", "lastUpdatedAt": "timestamp" } } ``` ## Troubleshooting Amazon EKS runtime coverage issues If the coverage status for your EKS cluster is `Unhealthy`, you can view the corresponding error either under the **Issue** column in the GuardDuty console, or by using the [CoverageResource](https://docs.aws.amazon.com/guardduty/latest/APIReference/API_CoverageResource.html) data type. When working with inclusion or exclusion tags for monitoring your EKS clusters selectively, it may take some time for the tags to sync. This may impact the coverage status of the associated EKS cluster. You can try removing and adding the corresponding tag (inclusion or exclusion) again. For more information, see [Tagging your Amazon EKS resources](https://docs.aws.amazon.com/eks/latest/userguide/eks-using-tags.html) in the **Amazon EKS User Guide**. The structure of a coverage issue is `Issue type:Extra information`. Typically, the issues will have an optional *Extra information* that may include specific client-side exception or description about the issue. Based on *Extra information*, the following tables provide the recommended steps to troubleshoot the coverage issues for your EKS clusters. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/guardduty/latest/ug/eks-runtime-monitoring-coverage.html) **Troubleshooting steps for Addon creation/updation error with Addon issue code** [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/guardduty/latest/ug/eks-runtime-monitoring-coverage.html)