# Logging and monitoring
To detect incidents, receive alerts when incidents occur, and respond to them, use these options with Amazon EMR on EKS:
+ Monitor Amazon EMR on EKS with AWS CloudTrail ‐ [AWS CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/) provides a record of actions taken by a user, role, or an AWS service in Amazon EMR on EKS. It captures calls from the Amazon EMR console and code calls to the Amazon EMR on EKS API operations as events. This allows you to determine the request that was made to Amazon EMR on EKS, the IP address from which the request was made, who made the request, when it was made, and additional details. For more information, see [Logging Amazon EMR on EKS API calls using AWS CloudTrail](logging-using-cloudtrail.md).
+ Use CloudWatch Events with Amazon EMR on EKS ‐ CloudWatch Events delivers a near real-time stream of system events that describe changes in AWS resources. CloudWatch Events becomes aware of operational changes as they occur, responds to them, and takes corrective action as necessary, by sending messages to respond to the environment, activating functions, making changes, and capturing state information. To use CloudWatch Events with Amazon EMR on EKS, create a rule that triggers on an Amazon EMR on EKS API call via CloudTrail. For more information, see [Monitor jobs with Amazon CloudWatch Events](monitoring.md#monitoring-cloudwatch-events).
# Encrypting Amazon EMR on EKS logs with managed storage
The sections that follow show you how to configure encryption for logs.
## Enable encryption
To encrypt logs in managed storage with your own KMS key, use the following configuration when you submit a job run.
```
"monitoringConfiguration": {
"managedLogs": {
"allowAWSToRetainLogs":"ENABLED",
"encryptionKeyArn":"KMS key arn"
},
"persistentAppUI": "ENABLED"
}
```
The `allowAWSToRetainLogs` configuration allows AWS to retain system namespace logs when running a job using Native FGAC. The `persistentAppUI` configuration allows AWS to save event logs which are used to generate the Spark UI. The `encryptionKeyArn` is used to specify the KMS key ARN you want to use to encrypt the logs stored by AWS.
## Required permissions for log encryption
The user who submits the job or views the Spark UI must be allowed the actions `kms:DescribeKey`, `kms:GenerateDataKey`, and `kms:Decrypt` for the encryption key. These permissions are used to verify the validity of the key and check that the user has the necessary permissions to read and write logs encrypted with the KMS key. If the user who submits the job lacks the necessary key permissions, Amazon EMR on EKS rejects the job run submission.
**Example IAM Policy for Role Used to Call StartJobRun**
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"emr-containers:StartJobRun"
],
"Resource": [
"*"
],
"Effect": "Allow",
"Sid": "AllowEMRCONTAINERSStartjobrun"
},
{
"Action": [
"kms:DescribeKey",
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": [
"arn:aws:kms:*:*:key/key-id"
],
"Effect": "Allow",
"Sid": "AllowKMSDescribekey"
}
]
}
```
------
You must also configure the KMS key to allow the `persistentappui.elasticmapreduce.amazonaws.com` and `elasticmapreduce.amazonaws.com` Service Principals to `kms:GenerateDataKey` and `kms:Decrypt`. This allows EMR to read and write logs encrypted with the KMS key to managed storage.
**Example KMS Key Policy**
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:DescribeKey"
],
"Resource": [
"*"
],
"Condition": {
"StringLike": {
"kms:viaService": "emr-containers.*.amazonaws.com"
}
},
"Sid": "AllowKMSDescribekey"
},
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": [
"*"
],
"Condition": {
"StringLike": {
"kms:viaService": "emr-containers.*.amazonaws.com",
"kms:EncryptionContext:aws:emr-containers:virtualClusterId": "virtual cluster id"
}
},
"Sid": "AllowKMSDecryptGenerate"
},
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": [
"*"
],
"Condition": {
"StringLike": {
"kms:EncryptionContext:aws:emr-containers:virtualClusterId": "virtual cluster id"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:emr-containers:*:*:/virtualclusters/virtual_cluster_id"
}
},
"Sid": "AllowKMSDecryptService"
}
]
}
```
------
As a security best practice, we recommend that you add the `kms:viaService`, `kms:EncryptionContext`, and `aws:SourceArn` conditions. These conditions help ensure the key is only used by Amazon EMR on EKS and only used for logs generated from jobs running in a specific virtual cluster.
# Logging Amazon EMR on EKS API calls using AWS CloudTrail
Amazon EMR on EKS is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or an AWS service in Amazon EMR on EKS. CloudTrail captures all API calls for Amazon EMR on EKS as events. The calls captured include calls from the Amazon EMR on EKS console and code calls to the Amazon EMR on EKS API operations. If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for Amazon EMR on EKS. If you don't configure a trail, you can still view the most recent events in the CloudTrail console in **Event history**. Using the information collected by CloudTrail, you can determine the request that was made to Amazon EMR on EKS, the IP address from which the request was made, who made the request, when it was made, and additional details.
To learn more about CloudTrail, see the [AWS CloudTrail User Guide](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-user-guide.html).
## Amazon EMR on EKS information in CloudTrail
CloudTrail is enabled on your AWS account when you create the account. When activity occurs in Amazon EMR on EKS, that activity is recorded in a CloudTrail event along with other AWS service events in **Event history**. You can view, search, and download recent events in your AWS account. For more information, see [Viewing events with CloudTrail Event history](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events.html).
For an ongoing record of events in your AWS account, including events for Amazon EMR on EKS, create a trail. A *trail* enables CloudTrail to deliver log files to an Amazon S3 bucket. By default, when you create a trail in the console, the trail applies to all AWS Regions. The trail logs events from all Regions in the AWS partition and delivers the log files to the Amazon S3 bucket that you specify. Additionally, you can configure other AWS services to further analyze and act upon the event data collected in CloudTrail logs. For more information, see the following:
+ [Overview for creating a trail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html)
+ [CloudTrail supported services and integrations](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-aws-service-specific-topics.html)
+ [Configuring Amazon SNS notifications for CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/configure-sns-notifications-for-cloudtrail.html)
+ [Receiving CloudTrail log files from multiple regions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/receive-cloudtrail-log-files-from-multiple-regions.html) and [Receiving CloudTrail log files from multiple accounts](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-receive-logs-from-multiple-accounts.html)
All Amazon EMR on EKS actions are logged by CloudTrail and are documented in [Amazon EMR on EKS API documentation](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/). For example, calls to the `CreateVirtualCluster`, `StartJobRun` and `ListJobRuns` actions generate entries in the CloudTrail log files.
Every event or log entry contains information about who generated the request. The identity information helps you determine the following:
+ Whether the request was made with root or AWS Identity and Access Management (IAM) user credentials.
+ Whether the request was made with temporary security credentials for a role or federated user.
+ Whether the request was made by another AWS service.
For more information, see the [CloudTrail user Identity element](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-user-identity.html).
## Understanding Amazon EMR on EKS log file entries
A trail is a configuration that enables delivery of events as log files to an Amazon S3 bucket that you specify. CloudTrail log files contain one or more log entries. An event represents a single request from any source and includes information about the requested action, the date and time of the action, request parameters, and so on. CloudTrail log files aren't an ordered stack trace of the public API calls, so they don't appear in any specific order.
The following example shows a CloudTrail log entry that demonstrates the [https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_ListJobRuns.html](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_ListJobRuns.html) action.
```
{
"eventVersion": "1.05",
"userIdentity": {
"type": "AssumedRole",
"principalId": "AIDACKCEVSQ6C2EXAMPLE:admin",
"arn": "arn:aws:sts::012345678910:assumed-role/Admin/admin",
"accountId": "012345678910",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "AIDACKCEVSQ6C2EXAMPLE",
"arn": "arn:aws:iam::012345678910:role/Admin",
"accountId": "012345678910",
"userName": "Admin"
},
"webIdFederationData": {},
"attributes": {
"mfaAuthenticated": "false",
"creationDate": "2020-11-04T21:49:36Z"
}
}
},
"eventTime": "2020-11-04T21:52:58Z",
"eventSource": "emr-containers.amazonaws.com",
"eventName": "ListJobRuns",
"awsRegion": "us-east-1",
"sourceIPAddress": "203.0.113.1",
"userAgent": "aws-cli/1.11.167 Python/2.7.10 Darwin/16.7.0 botocore/1.7.25",
"requestParameters": {
"virtualClusterId": "1K48XXXXXXHCB"
},
"responseElements": null,
"requestID": "890b8639-e51f-11e7-b038-EXAMPLE",
"eventID": "874f89fa-70fc-4798-bc00-EXAMPLE",
"readOnly": true,
"eventType": "AwsApiCall",
"recipientAccountId": "012345678910"
}
```