# Security in Amazon EMR on EKS
Cloud security at AWS is the highest priority. As an AWS customer, you benefit from data centers and network architectures that are built to meet the requirements of the most security-sensitive organizations.
Security is a shared responsibility between AWS and you. The [shared responsibility model](https://aws.amazon.com/compliance/shared-responsibility-model/) describes this as security *of* the cloud and security *in* the cloud:
+ **Security of the cloud** – AWS is responsible for protecting the infrastructure that runs AWS services in the AWS Cloud. AWS also provides you with services that you can use securely. Third-party auditors regularly test and verify the effectiveness of our security as part of the [AWS Compliance Programs](https://aws.amazon.com/compliance/programs/). To learn about the compliance programs that apply to Amazon EMR, see [AWS Services in Scope by Compliance Program](https://aws.amazon.com/compliance/services-in-scope/).
+ **Security in the cloud** – Your responsibility is determined by the AWS service that you use. You are also responsible for other factors including the sensitivity of your data, your company’s requirements, and applicable laws and regulations.
This documentation helps you understand how to apply the shared responsibility model when using Amazon EMR on EKS. The following topics show you how to configure Amazon EMR on EKS to meet your security and compliance objectives. You also learn how to use other AWS services that help you to monitor and secure your Amazon EMR on EKS resources.
**Topics**
+ [
# Amazon EMR on EKS security best practices
](security-best-practices.md)
+ [
# Data protection
](data-protection.md)
+ [
# Identity and Access Management
](security-iam.md)
+ [
# Using Amazon EMR on EKS with AWS Lake Formation for fine-grained access control
](security_iam_fgac-lf.md)
+ [
# Logging and monitoring
](security-logging-monitoring.md)
+ [
# Using Amazon S3 Access Grants with Amazon EMR on EKS
](access-grants.md)
+ [
# Compliance validation for Amazon EMR on EKS
](compliance.md)
+ [
# Resilience in Amazon EMR on EKS
](disaster-recovery-resiliency.md)
+ [
# Infrastructure security in Amazon EMR on EKS
](infrastructure-security.md)
+ [
# Configuration and vulnerability analysis
](configuration-vulnerability.md)
+ [
# Connect to Amazon EMR on EKS Using an interface VPC endpoint
](security-vpc.md)
+ [
# Set up cross-account access for Amazon EMR on EKS
](security-cross-account.md)
# Amazon EMR on EKS security best practices
Amazon EMR on EKS provides a number of security features to consider as you develop and implement your own security policies. The following best practices are general guidelines and don’t represent a complete security solution. Because these best practices might not be appropriate or sufficient for your environment, treat them as helpful considerations rather than prescriptions.
**Note**
For more security best practices, see [Amazon EMR on EKS security best practices](#security-best-practices).
## Apply principle of least privilege
Amazon EMR on EKS provides a granular access policy for applications using IAM roles, such as execution roles. These execution roles are mapped to Kubernetes service accounts through the IAM role’s trust policy. Amazon EMR on EKS creates pods within a registered Amazon EKS namespace that execute user-provided application code. The job pods running the application code assume the execution role when connecting to other AWS services. We recommend that execution roles be granted only the minimum set of privileges required by the job, such as covering your application and access to log destination. We also recommend auditing the jobs for permissions on a regular basis and upon any change to application code.
## Access control list for endpoints
Managed endpoints can be created only for those EKS clusters that have been configured to use at least one private subnet in your VPC. This configuration restricts access to the load balancers created by managed endpoints so that they can only be accessed from your VPC. To further enhance security, we recommend that you configure security groups with these load balancers so that they can restrict incoming traffic to a selected set of IP addresses.
## Get the latest security updates for custom images
To use custom images with Amazon EMR on EKS, you can install any binaries and libraries on the image. You are responsible for the security patching of the binaries you add to the image. Amazon EMR on EKS images are regularly patched with latest security patches. To get the latest image, you must rebuild the custom images whenever there is a new base image version of the Amazon EMR release. For more information, see [Amazon EMR on EKS releases](emr-eks-releases.md) and [Details for selecting a base image URI](docker-custom-images-tag.md).
## Limit pod credential access
Kubernetes supports several methods of assigning credentials to a pod. Provisioning multiple credentials providers can increase the complexity of your security model. Amazon EMR on EKS has adopted the use of [IAM roles for services accounts (IRSA)](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) as a standard credential provider within a registered EKS namespace. Other methods are not supported, including [kube2iam](https://github.com/jtblin/kube2iam), [kiam](https://github.com/uswitch/kiam) and using an EC2 instance profile of the instance running on the cluster.
## Isolate untrusted application code
Amazon EMR on EKS does not inspect the integrity of the application code submitted by users of the system. If you are running a multi-tenanted virtual cluster that is configured using multiple execution roles that can be used to submit jobs by untrusted tenants running arbitrary code, there is a risk of a malicious application escalating its privileges. In this situation, consider isolating execution roles with similar privileges into a different virtual cluster.
## Role-based access control (RBAC) permissions
Administrators should strictly control Role-based access control (RBAC) permissions for Amazon EMR on EKS managed namespaces. At a minimum, the following permissions should not be granted to job submitters in Amazon EMR on EKS managed namespaces.
+ Kubernetes RBAC permissions to modify configmap ‐ because Amazon EMR on EKS uses Kubernetes configmaps to generate managed pod templates that have the managed service account name. This attribute should not be mutated.
+ Kubernetes RBAC permissions to exec into Amazon EMR on EKS pods ‐ to avoid giving access to managed pod templates that have the managed SA name. This attribute should not be mutated. This permission can also give access to the JWT token mounted into the pod which can then be used to retrieve the execution role credentials.
+ Kubernetes RBAC permissions to create pods ‐ to prevent users from creating pods using a Kubernetes ServiceAccount which may be mapped to an IAM role with more AWS privileges than the user.
+ Kubernetes RBAC permissions to deploy mutating webhook ‐ to prevent users from using the mutating webhook to mutate Kubernetes ServiceAccount name for pods created by Amazon EMR on EKS.
+ Kubernetes RBAC permissions to read Kubernetes secrets ‐ to prevent users from reading confidential data stored in these secrets.
## Restrict access to nodegroup IAM role or instance profile credentials
+ We recommend that you assign minimum AWS permissions to nodegroup’s IAM role(s). This helps to avoid privilege escalation by code that may run using instance profile credentials of EKS worker nodes.
+ To completely block access to instance profile credentials to all pods that runs in Amazon EMR on EKS managed namespaces, we recommend that you run `iptables` commands on EKS nodes. For more information, see [Restricting access to Amazon EC2 instance profile credentials](https://docs.aws.amazon.com/eks/latest/userguide/restrict-ec2-credential-access.html). However, it is important to properly scope your service account IAM roles so that your pods have all of the necessary permissions. For example, the node IAM role is assigned permissions to pull container images from Amazon ECR. If a pod isn't assigned those permissions, the pod can't pull container images from Amazon ECR. The VPC CNI plugin also needs to be updated. For more information, see [Walkthrough: Updating the VPC CNI plugin to use IAM roles for service accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-cni-walkthrough.html).
# Data protection
The AWS [shared responsibility model](https://aws.amazon.com/compliance/shared-responsibility-model/) applies to data protection in Amazon EMR on EKS. As described in this model, AWS is responsible for protecting the global infrastructure that runs all of the AWS Cloud. You are responsible for maintaining control over your content that is hosted on this infrastructure. This content includes the security configuration and management tasks for the AWS services that you use. For more information about data privacy, see the [Data Privacy FAQ ](https://aws.amazon.com/compliance/data-privacy-faq/). For information about data protection in Europe, see [the AWS Shared Responsibility Model and GDPR](http://aws.amazon.com/blogs/security/the-aws-shared-responsibility-model-and-gdpr/) blog post on the AWS Security Blog.
For data protection purposes, we recommend that you protect AWS account credentials and set up individual accounts with AWS Identity and Access Management (IAM). That way each user is given only the permissions necessary to fulfill their job duties. We also recommend that you secure your data in the following ways:
+ Use multi-factor authentication (MFA) with each account.
+ Use SSL/TLS to communicate with AWS resources. We recommend TLS 1.2 or later.
+ Set up API and user activity logging with AWS CloudTrail.
+ Use AWS encryption solutions, along with all default security controls within AWS services.
+ Use advanced managed security services such as Amazon Macie, which assists in discovering and securing personal data that is stored in Amazon S3.
+ Use Amazon EMR on EKS encryption options to encrypt data at rest and in transit.
+ If you require FIPS 140-2 validated cryptographic modules when accessing AWS through a command line interface or an API, use a FIPS endpoint. For more information about the available FIPS endpoints, see [Federal Information Processing Standard (FIPS) 140-2](https://aws.amazon.com/compliance/fips/).
We strongly recommend that you never put sensitive identifying information, such as your customers' account numbers, into free-form fields such as a **Name** field. This includes when you work with Amazon EMR on EKS or other AWS services using the console, API, AWS CLI, or AWS SDKs. Any data that you enter into Amazon EMR on EKS or other services might get picked up for inclusion in diagnostic logs. When you provide a URL to an external server, don't include credentials information in the URL to validate your request to that server.
## Encryption at rest
Data encryption helps prevent unauthorized users from reading data on a cluster and associated data storage systems. This includes data saved to persistent media, known as data at rest, and data that may be intercepted as it travels the network, known as data in transit.
Data encryption requires keys and certificates. You can choose from several options, including keys managed by AWS Key Management Service, keys managed by Amazon S3, and keys and certificates from custom providers that you supply. When using AWS KMS as your key provider, charges apply for the storage and use of encryption keys. For more information, see [AWS KMS Pricing](https://aws.amazon.com/kms/pricing/).
Before you specify encryption options, decide on the key and certificate management systems you want to use. Then create the keys and certificates for the custom providers that you specify as part of encryption settings.
### Encryption at rest for EMRFS data in Amazon S3
Amazon S3 encryption works with EMR File System (EMRFS) objects read from and written to Amazon S3. You specify Amazon S3 server-side encryption (SSE) or client-side encryption (CSE) as the **Default encryption mode** when you enable encryption at rest. Optionally, you can specify different encryption methods for individual buckets using **Per bucket encryption overrides**. Regardless of whether Amazon S3 encryption is enabled, Transport Layer Security (TLS) encrypts the EMRFS objects in transit between EMR cluster nodes and Amazon S3. For in-depth information about Amazon S3 encryption, see [Protecting Data Using Encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingEncryption.html) in the Amazon Simple Storage Service Developer Guide.
**Note**
When you use AWS KMS, charges apply for the storage and use of encryption keys. For more information, see [AWS KMS Pricing](https://aws.amazon.com/kms/pricing/).
### Amazon S3 server-side encryption
When you set up Amazon S3 server-side encryption, Amazon S3 encrypts data at the object level as it writes the data to disk and decrypts the data when it is accessed. For more information about SSE, see [Protecting Data Using Server-Side Encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html) in the Amazon Simple Storage Service Developer Guide.
You can choose between two different key management systems when you specify SSE in Amazon EMR on EKS:
+ **SSE-S3** ‐ Amazon S3 manages keys for you.
+ **SSE-KMS** ‐ You use an AWS KMS key to set up with policies suitable for Amazon EMR on EKS.
SSE with customer-provided keys (SSE-C) is not available for use with Amazon EMR on EKS.
### Amazon S3 client-side encryption
With Amazon S3 client-side encryption, the Amazon S3 encryption and decryption takes place in the EMRFS client on your cluster. Objects are encrypted before being uploaded to Amazon S3 and decrypted after they are downloaded. The provider you specify supplies the encryption key that the client uses. The client can use keys provided by AWS KMS (CSE-KMS) or a custom Java class that provides the client-side root key (CSE-C). The encryption specifics are slightly different between CSE-KMS and CSE-C, depending on the specified provider and the metadata of the object being decrypted or encrypted. For more information about these differences, see [Protecting Data Using Client-Side Encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html) in the Amazon Simple Storage Service Developer Guide.
**Note**
Amazon S3 CSE only ensures that EMRFS data exchanged with Amazon S3 is encrypted; not all data on cluster instance volumes is encrypted. Furthermore, because Hue does not use EMRFS, objects that the Hue S3 File Browser writes to Amazon S3 are not encrypted.
### Local disk encryption
Apache Spark supports encrypting temporary data written to local disks. This covers shuffle files, shuffle spills, and data blocks stored on disk for both caching and broadcast variables. It does not cover encrypting output data generated by applications with APIs such as `saveAsHadoopFile` or `saveAsTable`. It also may not cover temporary files created explicitly by the user. For more information, see [Local Storage Encryption](https://spark.apache.org/docs/latest/security.html#local-storage-encryption) in the Spark documentation. Spark does not support encrypted data on local disk, such as intermediate data written to a local disk by an executor process when the data does not fit in memory. Data that is persisted to disk is scoped to the job runtime, and the key that is used to encrypt the data is generated dynamically by Spark for every job run. Once the Spark job terminates, no other process can decrypt the data.
For driver and executor pod, you encrypt data at rest that is persisted to the mounted volume. There are three different AWS native storage options you can use with Kubernetes: [EBS](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html), [EFS](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEFS.html), and [FSx for Lustre](https://docs.aws.amazon.com/fsx/latest/LustreGuide/what-is.html). All three offer encryption at rest using a service managed key or an AWS KMS key. For more information see the [EKS Best Practices Guide](https://aws.github.io/aws-eks-best-practices/security/docs/data). With this approach, all data persisted to the mounted volume is encrypted.
### Key management
You can configure KMS to automatically rotate your KMS keys. This rotates your keys once a year while saving old keys indefinitely so that your data can still be decrypted. For additional information, see [Rotating AWS KMS keys](https://docs.aws.amazon.com/kms/latest/developerguide/rotate-keys.html).
## Encryption in transit
Several encryption mechanisms are enabled with in-transit encryption. These are open-source features, are application-specific, and may vary by Amazon EMR on EKS release. The following application-specific encryption features can be enabled with Amazon EMR on EKS:
+ Spark
+ Internal RPC communication between Spark components, such as the block transfer service and the external shuffle service, is encrypted using the AES-256 cipher in Amazon EMR versions 5.9.0 and later. In earlier releases, internal RPC communication is encrypted using SASL with DIGEST-MD5 as the cipher.
+ HTTP protocol communication with user interfaces such as Spark History Server and HTTPS-enabled file servers is encrypted using Spark's SSL configuration. For more information, see [SSL Configuration](https://spark.apache.org/docs/latest/security.html#ssl-configuration) in Spark documentation.
For more information, see [Spark security settings](http://spark.apache.org/docs/latest/security.html).
+ You should allow only encrypted connections over HTTPS (TLS) using [the aws:SecureTransport condition](https://docs.aws.amazon.com//IAM/latest/UserGuide/reference_policies_elements_condition_operators.html#Conditions_Boolean) on Amazon S3 bucket IAM policies.
+ Query results that stream to JDBC or ODBC clients are encrypted using TLS.
# Identity and Access Management
AWS Identity and Access Management (IAM) is an AWS service that helps an administrator securely control access to AWS resources. IAM administrators control who can be *authenticated* (signed in) and *authorized* (have permissions) to use Amazon EMR on EKS resources. IAM is an AWS service that you can use with no additional charge.
**Topics**
+ [
## Audience
](#security_iam_audience)
+ [
## Authenticating with identities
](#security_iam_authentication)
+ [
## Managing access using policies
](#security_iam_access-manage)
+ [
# How Amazon EMR on EKS works with IAM
](security_iam_service-with-iam.md)
+ [
# Using service-linked roles for Amazon EMR on EKS
](using-service-linked-roles.md)
+ [
# Managed policies for Amazon EMR on EKS
](emr-eks-managed-polices.md)
+ [
# Using job execution roles with Amazon EMR on EKS
](iam-execution-role.md)
+ [
# Identity-based policy examples for Amazon EMR on EKS
](security_iam_id-based-policy-examples.md)
+ [
# Policies for tag-based access control
](security_iam_TBAC.md)
+ [
# Troubleshooting Amazon EMR on EKS identity and access
](security_iam_troubleshoot.md)
## Audience
How you use AWS Identity and Access Management (IAM) differs based on your role:
+ **Service user** - request permissions from your administrator if you cannot access features (see [Troubleshooting Amazon EMR on EKS identity and access](security_iam_troubleshoot.md))
+ **Service administrator** - determine user access and submit permission requests (see [How Amazon EMR on EKS works with IAM](security_iam_service-with-iam.md))
+ **IAM administrator** - write policies to manage access (see [Identity-based policy examples for Amazon EMR on EKS](security_iam_id-based-policy-examples.md))
## Authenticating with identities
Authentication is how you sign in to AWS using your identity credentials. You must be authenticated as the AWS account root user, an IAM user, or by assuming an IAM role.
You can sign in as a federated identity using credentials from an identity source like AWS IAM Identity Center (IAM Identity Center), single sign-on authentication, or Google/Facebook credentials. For more information about signing in, see [How to sign in to your AWS account](https://docs.aws.amazon.com/signin/latest/userguide/how-to-sign-in.html) in the *AWS Sign-In User Guide*.
For programmatic access, AWS provides an SDK and CLI to cryptographically sign requests. For more information, see [AWS Signature Version 4 for API requests](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv.html) in the *IAM User Guide*.
### AWS account root user
When you create an AWS account, you begin with one sign-in identity called the AWS account *root user* that has complete access to all AWS services and resources. We strongly recommend that you don't use the root user for everyday tasks. For tasks that require root user credentials, see [Tasks that require root user credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_root-user.html#root-user-tasks) in the *IAM User Guide*.
### Federated identity
As a best practice, require human users to use federation with an identity provider to access AWS services using temporary credentials.
A *federated identity* is a user from your enterprise directory, web identity provider, or Directory Service that accesses AWS services using credentials from an identity source. Federated identities assume roles that provide temporary credentials.
For centralized access management, we recommend AWS IAM Identity Center. For more information, see [What is IAM Identity Center?](https://docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html) in the *AWS IAM Identity Center User Guide*.
### IAM users and groups
An *[IAM user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users.html)* is an identity with specific permissions for a single person or application. We recommend using temporary credentials instead of IAM users with long-term credentials. For more information, see [Require human users to use federation with an identity provider to access AWS using temporary credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#bp-users-federation-idp) in the *IAM User Guide*.
An [https://docs.aws.amazon.com/IAM/latest/UserGuide/id_groups.html](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_groups.html) specifies a collection of IAM users and makes permissions easier to manage for large sets of users. For more information, see [Use cases for IAM users](https://docs.aws.amazon.com/IAM/latest/UserGuide/gs-identities-iam-users.html) in the *IAM User Guide*.
### IAM roles
An *[IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html)* is an identity with specific permissions that provides temporary credentials. You can assume a role by [switching from a user to an IAM role (console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-console.html) or by calling an AWS CLI or AWS API operation. For more information, see [Methods to assume a role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_manage-assume.html) in the *IAM User Guide*.
IAM roles are useful for federated user access, temporary IAM user permissions, cross-account access, cross-service access, and applications running on Amazon EC2. For more information, see [Cross account resource access in IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies-cross-account-resource-access.html) in the *IAM User Guide*.
## Managing access using policies
You control access in AWS by creating policies and attaching them to AWS identities or resources. A policy defines permissions when associated with an identity or resource. AWS evaluates these policies when a principal makes a request. Most policies are stored in AWS as JSON documents. For more information about JSON policy documents, see [Overview of JSON policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html#access_policies-json) in the *IAM User Guide*.
Using policies, administrators specify who has access to what by defining which **principal** can perform **actions** on what **resources**, and under what **conditions**.
By default, users and roles have no permissions. An IAM administrator creates IAM policies and adds them to roles, which users can then assume. IAM policies define permissions regardless of the method used to perform the operation.
### Identity-based policies
Identity-based policies are JSON permissions policy documents that you attach to an identity (user, group, or role). These policies control what actions identities can perform, on which resources, and under what conditions. To learn how to create an identity-based policy, see [Define custom IAM permissions with customer managed policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html) in the *IAM User Guide*.
Identity-based policies can be *inline policies* (embedded directly into a single identity) or *managed policies* (standalone policies attached to multiple identities). To learn how to choose between managed and inline policies, see [Choose between managed policies and inline policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies-choosing-managed-or-inline.html) in the *IAM User Guide*.
### Resource-based policies
Resource-based policies are JSON policy documents that you attach to a resource. Examples include IAM *role trust policies* and Amazon S3 *bucket policies*. In services that support resource-based policies, service administrators can use them to control access to a specific resource. You must [specify a principal](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html) in a resource-based policy.
Resource-based policies are inline policies that are located in that service. You can't use AWS managed policies from IAM in a resource-based policy.
### Other policy types
AWS supports additional policy types that can set the maximum permissions granted by more common policy types:
+ **Permissions boundaries** – Set the maximum permissions that an identity-based policy can grant to an IAM entity. For more information, see [Permissions boundaries for IAM entities](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html) in the *IAM User Guide*.
+ **Service control policies (SCPs)** – Specify the maximum permissions for an organization or organizational unit in AWS Organizations. For more information, see [Service control policies](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html) in the *AWS Organizations User Guide*.
+ **Resource control policies (RCPs)** – Set the maximum available permissions for resources in your accounts. For more information, see [Resource control policies (RCPs)](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_rcps.html) in the *AWS Organizations User Guide*.
+ **Session policies** – Advanced policies passed as a parameter when creating a temporary session for a role or federated user. For more information, see [Session policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html#policies_session) in the *IAM User Guide*.
### Multiple policy types
When multiple types of policies apply to a request, the resulting permissions are more complicated to understand. To learn how AWS determines whether to allow a request when multiple policy types are involved, see [Policy evaluation logic](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic.html) in the *IAM User Guide*.
# How Amazon EMR on EKS works with IAM
Before you use IAM to manage access to Amazon EMR on EKS, learn what IAM features are available to use with Amazon EMR on EKS.
**IAM features you can use with Amazon EMR on EKS**
| IAM feature | Amazon EMR on EKS support |
| --- | --- |
| [Identity-based policies](#security_iam_service-with-iam-id-based-policies) | Yes |
| [Resource-based policies](#security_iam_service-with-iam-resource-based-policies) | No |
| [Policy actions](#security_iam_service-with-iam-id-based-policies-actions) | Yes |
| [Policy resources](#security_iam_service-with-iam-id-based-policies-resources) | Yes |
| [Policy condition keys](#security_iam_service-with-iam-id-based-policies-conditionkeys) | Yes |
| [ACLs](#security_iam_service-with-iam-acls) | No |
| [ABAC (tags in policies)](#security_iam_service-with-iam-tags) | Yes |
| [Temporary credentials](#security_iam_service-with-iam-roles-tempcreds) | Yes |
| [Principal permissions](#security_iam_service-with-iam-principal-permissions) | Yes |
| [Service roles](#security_iam_service-with-iam-roles-service) | No |
| [Service-linked roles](#security_iam_service-with-iam-roles-service-linked) | Yes |
To get a high-level view of how Amazon EMR on EKS and other AWS services work with most IAM features, see [AWS services that work with IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-services-that-work-with-iam.html) in the *IAM User Guide*.
## Identity-based policies for Amazon EMR on EKS
**Supports identity-based policies:** Yes
Identity-based policies are JSON permissions policy documents that you can attach to an identity, such as an IAM user, group of users, or role. These policies control what actions users and roles can perform, on which resources, and under what conditions. To learn how to create an identity-based policy, see [Define custom IAM permissions with customer managed policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html) in the *IAM User Guide*.
With IAM identity-based policies, you can specify allowed or denied actions and resources as well as the conditions under which actions are allowed or denied. To learn about all of the elements that you can use in a JSON policy, see [IAM JSON policy elements reference](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements.html) in the *IAM User Guide*.
### Identity-based policy examples for Amazon EMR on EKS
To view examples of Amazon EMR on EKS identity-based policies, see [Identity-based policy examples for Amazon EMR on EKS](security_iam_id-based-policy-examples.md).
## Resource-based policies within Amazon EMR on EKS
**Supports resource-based policies:** No
Resource-based policies are JSON policy documents that you attach to a resource. Examples of resource-based policies are IAM *role trust policies* and Amazon S3 *bucket policies*. In services that support resource-based policies, service administrators can use them to control access to a specific resource. For the resource where the policy is attached, the policy defines what actions a specified principal can perform on that resource and under what conditions. You must [specify a principal](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html) in a resource-based policy. Principals can include accounts, users, roles, federated users, or AWS services.
To enable cross-account access, you can specify an entire account or IAM entities in another account as the principal in a resource-based policy. For more information, see [Cross account resource access in IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies-cross-account-resource-access.html) in the *IAM User Guide*.
## Policy actions for Amazon EMR on EKS
**Supports policy actions:** Yes
Administrators can use AWS JSON policies to specify who has access to what. That is, which **principal** can perform **actions** on what **resources**, and under what **conditions**.
The `Action` element of a JSON policy describes the actions that you can use to allow or deny access in a policy. Include actions in a policy to grant permissions to perform the associated operation.
To see a list of Amazon EMR on EKS actions, see [Actions, resources, and condition keys for Amazon EMR on EKS](https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazonemroneksemrcontainers.html) in the *Service Authorization Reference*.
Policy actions in Amazon EMR on EKS use the following prefix before the action:
```
emr-containers
```
To specify multiple actions in a single statement, separate them with commas.
```
"Action": [
"emr-containers:action1",
"emr-containers:action2"
]
```
To view examples of Amazon EMR on EKS identity-based policies, see [Identity-based policy examples for Amazon EMR on EKS](security_iam_id-based-policy-examples.md).
## Policy resources for Amazon EMR on EKS
**Supports policy resources:** Yes
Administrators can use AWS JSON policies to specify who has access to what. That is, which **principal** can perform **actions** on what **resources**, and under what **conditions**.
The `Resource` JSON policy element specifies the object or objects to which the action applies. As a best practice, specify a resource using its [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html). For actions that don't support resource-level permissions, use a wildcard (\$1) to indicate that the statement applies to all resources.
```
"Resource": "*"
```
To see a list of Amazon EMR on EKS resource types and their ARNs, see [Resources defined by Amazon EMR on EKS](https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazonemroneksemrcontainers.html#amazonemroneksemrcontainers-resources-for-iam-policies) in the *Service Authorization Reference*. To learn which actions you can specify the ARN of each resource, see [Actions, resources, and condition keys for Amazon EMR on EKS](https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazonemroneksemrcontainers.html).
To view examples of Amazon EMR on EKS identity-based policies, see [Identity-based policy examples for Amazon EMR on EKS](security_iam_id-based-policy-examples.md).
## Policy condition keys for Amazon EMR on EKS
**Supports service-specific policy condition keys:** Yes
Administrators can use AWS JSON policies to specify who has access to what. That is, which **principal** can perform **actions** on what **resources**, and under what **conditions**.
The `Condition` element specifies when statements execute based on defined criteria. You can create conditional expressions that use [condition operators](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition_operators.html), such as equals or less than, to match the condition in the policy with values in the request. To see all AWS global condition keys, see [AWS global condition context keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html) in the *IAM User Guide*.
To see a list of Amazon EMR on EKS condition keys and to learn which actions and resources you can use a condition key, see [Actions, resources, and condition keys for Amazon EMR on EKS](https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazonemroneksemrcontainers.html) in the *Service Authorization Reference*.
To view examples of Amazon EMR on EKS identity-based policies, see [Identity-based policy examples for Amazon EMR on EKS](security_iam_id-based-policy-examples.md).
## Access control lists (ACLs) in Amazon EMR on EKS
**Supports ACLs:** No
Access control lists (ACLs) control which principals (account members, users, or roles) have permissions to access a resource. ACLs are similar to resource-based policies, although they do not use the JSON policy document format.
## Attribute-based access control (ABAC) with Amazon EMR on EKS
| | |
| --- |--- |
| Supports ABAC (tags in policies) | Yes |
Attribute-based access control (ABAC) is an authorization strategy that defines permissions based on attributes called tags. You can attach tags to IAM entities and AWS resources, then design ABAC policies to allow operations when the principal's tag matches the tag on the resource.
To control access based on tags, you provide tag information in the [condition element](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition.html) of a policy using the `aws:ResourceTag/key-name`, `aws:RequestTag/key-name`, or `aws:TagKeys` condition keys.
If a service supports all three condition keys for every resource type, then the value is **Yes** for the service. If a service supports all three condition keys for only some resource types, then the value is **Partial**.
For more information about ABAC, see [Define permissions with ABAC authorization](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html) in the *IAM User Guide*. To view a tutorial with steps for setting up ABAC, see [Use attribute-based access control (ABAC)](https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_attribute-based-access-control.html) in the *IAM User Guide*.
## Using Temporary credentials with Amazon EMR on EKS
**Supports temporary credentials:** Yes
Temporary credentials provide short-term access to AWS resources and are automatically created when you use federation or switch roles. AWS recommends that you dynamically generate temporary credentials instead of using long-term access keys. For more information, see [Temporary security credentials in IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html) and [AWS services that work with IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-services-that-work-with-iam.html) in the *IAM User Guide*.
## Cross-service principal permissions for Amazon EMR on EKS
**Supports forward access sessions (FAS):** Yes
Forward access sessions (FAS) use the permissions of the principal calling an AWS service, combined with the requesting AWS service to make requests to downstream services. For policy details when making FAS requests, see [Forward access sessions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_forward_access_sessions.html).
## Service roles for Amazon EMR on EKS
| | |
| --- |--- |
| Supports service roles | No |
## Service-linked roles for Amazon EMR on EKS
| | |
| --- |--- |
| Supports service-linked roles | Yes |
For details about creating or managing service-linked roles, see [AWS services that work with IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-services-that-work-with-iam.html). Find a service in the table that includes a `Yes` in the **Service-linked role** column. Choose the **Yes** link to view the service-linked role documentation for that service.
# Using service-linked roles for Amazon EMR on EKS
Amazon EMR on EKS uses AWS Identity and Access Management (IAM)[ service-linked roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html#iam-term-service-linked-role). A service-linked role is a unique type of IAM role that is linked directly to Amazon EMR on EKS. Service-linked roles are predefined by Amazon EMR on EKS and include all the permissions that the service requires to call other AWS services on your behalf.
A service-linked role makes setting up Amazon EMR on EKS easier because you don’t have to manually add the necessary permissions. Amazon EMR on EKS defines the permissions of its service-linked roles, and unless defined otherwise, only Amazon EMR on EKS can assume its roles. The defined permissions include the trust policy and the permissions policy, and that permissions policy cannot be attached to any other IAM entity.
You can delete a service-linked role only after first deleting their related resources. This protects your Amazon EMR on EKS resources because you can't inadvertently remove permission to access the resources.
For information about other services that support service-linked roles, see [AWS Services That Work with IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-services-that-work-with-iam.html) and look for the services that have **Yes **in the **Service-Linked Role** column. Choose a **Yes** with a link to view the service-linked role documentation for that service.
## Service-linked role permissions for Amazon EMR on EKS
Amazon EMR on EKS uses the service-linked role named **AWSServiceRoleForAmazonEMRContainers**.
The `AWSServiceRoleForAmazonEMRContainers` service-linked role trusts the following services to assume the role:
+ `emr-containers.amazonaws.com`
The role permissions policy `AmazonEMRContainersServiceRolePolicy` allows Amazon EMR on EKS to complete a set of actions on the specified resources, as the following policy statement demonstrates.
**Note**
Managed policy contents change, so the policy shown here may be out-of-date. View the most up-to-date policy documentation at [AmazonEMRContainersServiceRolePolicy](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonEMRContainersServiceRolePolicy.html) in the *AWS Managed Policy Reference Guide*.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"eks:DescribeCluster",
"eks:ListNodeGroups",
"eks:DescribeNodeGroup",
"ec2:DescribeRouteTables",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups",
"elasticloadbalancing:DescribeInstanceHealth",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DescribeTargetHealth",
"eks:ListPodIdentityAssociations",
"eks:DescribePodIdentityAssociation"
],
"Resource": [
"*"
],
"Sid": "AllowEKSDescribecluster"
},
{
"Effect": "Allow",
"Action": [
"acm:ImportCertificate",
"acm:AddTagsToCertificate"
],
"Resource": [
"*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/emr-container:endpoint:managed-certificate": "true"
}
},
"Sid": "AllowACMImportcertificate"
},
{
"Effect": "Allow",
"Action": [
"acm:DeleteCertificate"
],
"Resource": [
"*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/emr-container:endpoint:managed-certificate": "true"
}
},
"Sid": "AllowACMDeletecertificate"
}
]
}
```
------
You must configure permissions to allow an IAM entity (such as a user, group, or role) to create, edit, or delete a service-linked role. For more information, see [Service-Linked Role Permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#service-linked-role-permissions) in the *IAM User Guide*.
## Creating a service-linked role for Amazon EMR on EKS
You don't need to manually create a service-linked role. When you create a virtual cluster, Amazon EMR on EKS creates the service-linked role for you.
If you delete this service-linked role, and then need to create it again, you can use the same process to recreate the role in your account. When you create a virtual cluster, Amazon EMR on EKS creates the service-linked role for you again.
You can also use the IAM console to create a service-linked role with the **Amazon EMR on EKS** use case. In the AWS CLI or the AWS API, create a service-linked role with the `emr-containers.amazonaws.com` service name. For more information, see [Creating a Service-Linked Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#create-service-linked-role) in the *IAM User Guide*. If you delete this service-linked role, you can use this same process to create the role again.
## Editing a service-linked role for Amazon EMR on EKS
Amazon EMR on EKS does not allow you to edit the `AWSServiceRoleForAmazonEMRContainers` service-linked role. After you create a service-linked role, you cannot change the name of the role because various entities might reference the role. However, you can edit the description of the role using IAM. For more information, see [Editing a Service-Linked Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#edit-service-linked-role) in the *IAM User Guide*.
## Deleting a service-linked role for Amazon EMR on EKS
If you no longer need to use a feature or service that requires a service-linked role, we recommend that you delete that role. That way you don’t have an unused entity that is not actively monitored or maintained. However, you must clean up the resources for your service-linked role before you can manually delete it.
**Note**
If the Amazon EMR on EKS service is using the role when you try to delete the resources, then the deletion might fail. If that happens, wait for a few minutes and try the operation again.
**To delete Amazon EMR on EKS resources used by the `AWSServiceRoleForAmazonEMRContainers`**
1. Open the Amazon EMR console.
1. Choose a virtual cluster.
1. On the `Virtual Cluster` page choose **Delete**.
1. Repeat this procedure for any other virtual clusters in your account.
**To manually delete the service-linked role using IAM**
Use the IAM console, the AWS CLI, or the AWS API to delete the `AWSServiceRoleForAmazonEMRContainers` service-linked role. For more information, see [Deleting a Service-Linked Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html#delete-service-linked-role) in the *IAM User Guide*.
## Supported Regions for Amazon EMR on EKS service-linked roles
Amazon EMR on EKS supports using service-linked roles in all of the Regions where the service is available. For more information, see [Amazon EMR on EKS service endpoints and quotas](service-quotas.md).
# Managed policies for Amazon EMR on EKS
View details about updates to AWS managed policies for Amazon EMR on EKS since March 1, 2021.
| Change | Description | Date |
| --- | --- | --- |
| AmazonEMRContainersServiceRolePolicy ‐ Added read permissions to list EKS pod identity associations in a cluster, and another read permission to return descriptive information about pod identity associations in a cluster. For more information, see [AmazonEMRContainersServiceRolePolicy](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonEMRContainersServiceRolePolicy.html). | The following permissions are added to the policy: `eks:ListPodIdentityAssociations`, `eks:DescribePodIdentityAssociation`. | February 3, 2023 |
| AmazonEMRContainersServiceRolePolicy ‐ Added permissions to describe and list Amazon EKS nodegroups, describe load balancer target groups, and describe load balancer target health. | The following permissions are added to the policy: `eks:ListNodeGroups`, `eks:DescribeNodeGroup`, `elasticloadbalancing:DescribeTargetGroups`, `elasticloadbalancing:DescribeTargetHealth`. | March 13, 2023 |
| AmazonEMRContainersServiceRolePolicy ‐ Added permissions to import and delete certificates in AWS Certificate Manager. | The following permissions are added to the policy: `acm:ImportCertificate`, `acm:AddTagsToCertificate`, `acm:DeleteCertificate`. | Dec 3, 2021 |
| Amazon EMR on EKS started tracking changes | Amazon EMR on EKS started tracking changes for its AWS managed policies. | March 1, 2021 |
# Using job execution roles with Amazon EMR on EKS
To use the `StartJobRun` command to submit a job run on an EKS cluster, you must first onboard a job execution role to be used with a virtual cluster. For more information, see [Create a job execution role](creating-job-execution-role.md) in [Setting up Amazon EMR on EKS](setting-up.md). You can also follow the instructions in the [Create IAM Role for job execution](https://www.eksworkshop.com/advanced/430_emr_on_eks/prereqs/#create-iam-role-for-job-execution) section of the Amazon EMR on EKS Workshop.
The following permissions must be included in the trust policy for the job execution role.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRoleWithWebIdentity"
],
"Resource": "*",
"Condition": {
"StringLike": {
"aws:userid": "system:serviceaccount:NAMESPACE:emr-containers-sa-*-*-AWS_ACCOUNT_ID-BASE36_ENCODED_ROLE_NAME"
}
},
"Sid": "AllowSTSAssumerolewithwebidentity"
}
]
}
```
------
The trust policy in the preceding example grants permissions only to an Amazon EMR managed Kubernetes service account with a name that matches the `emr-containers-sa-*-*-AWS_ACCOUNT_ID-BASE36_ENCODED_ROLE_NAME` pattern. Service accounts with this pattern will be automatically created at job submission, and scoped to the namespace where you submit the job. This trust policy allows these service accounts to assume the execution role and get the temporary credentials of the execution role. Service accounts from a different Amazon EKS cluster or from a different namespace within the same EKS cluster are restricted from assuming the execution role.
You can run the following command to automatically update the trust policy in the format given above.
```
aws emr-containers update-role-trust-policy \
--cluster-name cluster \
--namespace namespace \
--role-name iam_role_name_for_job_execution
```
**Controlling access to the execution role**
An administrator for your Amazon EKS cluster can create a multi-tenant Amazon EMR on EKS virtual cluster to which an IAM administrator can add multiple execution roles. Because untrusted tenants can use these execution roles to submit jobs that run arbitrary code, you might want to restrict those tenants so that they can't run code that gains the permissions assigned to one or more of these execution roles. To restrict the IAM policy attached to an IAM identity, the IAM administrator can use the optional Amazon Resource Name (ARN) condition key `emr-containers:ExecutionRoleArn`. This condition accepts a list of execution role ARNs that have permissions to the virtual cluster, as the following permissions policy demonstrates.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"emr-containers:StartJobRun"
],
"Resource": [
"arn:aws:emr-containers:*:*:/virtualclusters/VIRTUAL_CLUSTER_ID"
],
"Condition": {
"ArnEquals": {
"emr-containers:ExecutionRoleArn": [
"arn:aws:iam::*:role/execution_role_name_1",
"arn:aws:iam::*:role/execution_role_name_2"
]
}
},
"Sid": "AllowEMRCONTAINERSStartjobrun"
}
]
}
```
------
If you want to allow all execution roles that begin with a particular prefix, such as `MyRole`, you can replace the condition operator `ArnEquals` with the `ArnLike` operator, and you can replace the `execution_role_arn` value in the condition with a wildcard `*` character. For example, `arn:aws:iam::AWS_ACCOUNT_ID:role/MyRole*`. All other [ARN condition keys ](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition_operators.html#Conditions_ARN) are also supported.
**Note**
With Amazon EMR on EKS, you can't grant permissions to execution roles based on tags or attributes. Amazon EMR on EKS doesn't support tag-based access control (TBAC) or attribute-based access control (ABAC) for execution roles.
# Identity-based policy examples for Amazon EMR on EKS
By default, users and roles don't have permission to create or modify Amazon EMR on EKS resources. To grant users permission to perform actions on the resources that they need, an IAM administrator can create IAM policies.
To learn how to create an IAM identity-based policy by using these example JSON policy documents, see [Create IAM policies (console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create-console.html) in the *IAM User Guide*.
For details about actions and resource types defined by Amazon EMR on EKS, including the format of the ARNs for each of the resource types, see [Actions, resources, and condition keys for Amazon EMR on EKS](https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazonemroneksemrcontainers.html) in the *Service Authorization Reference*.
**Topics**
+ [
## Policy best practices
](#security_iam_service-with-iam-policy-best-practices)
+ [
## Using the Amazon EMR on EKS console
](#security_iam_id-based-policy-examples-console)
+ [
## Allow users to view their own permissions
](#security_iam_id-based-policy-examples-view-own-permissions)
## Policy best practices
Identity-based policies determine whether someone can create, access, or delete Amazon EMR on EKS resources in your account. These actions can incur costs for your AWS account. When you create or edit identity-based policies, follow these guidelines and recommendations:
+ **Get started with AWS managed policies and move toward least-privilege permissions** – To get started granting permissions to your users and workloads, use the *AWS managed policies* that grant permissions for many common use cases. They are available in your AWS account. We recommend that you reduce permissions further by defining AWS customer managed policies that are specific to your use cases. For more information, see [AWS managed policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html#aws-managed-policies) or [AWS managed policies for job functions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_job-functions.html) in the *IAM User Guide*.
+ **Apply least-privilege permissions** – When you set permissions with IAM policies, grant only the permissions required to perform a task. You do this by defining the actions that can be taken on specific resources under specific conditions, also known as *least-privilege permissions*. For more information about using IAM to apply permissions, see [ Policies and permissions in IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html) in the *IAM User Guide*.
+ **Use conditions in IAM policies to further restrict access** – You can add a condition to your policies to limit access to actions and resources. For example, you can write a policy condition to specify that all requests must be sent using SSL. You can also use conditions to grant access to service actions if they are used through a specific AWS service, such as CloudFormation. For more information, see [ IAM JSON policy elements: Condition](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition.html) in the *IAM User Guide*.
+ **Use IAM Access Analyzer to validate your IAM policies to ensure secure and functional permissions** – IAM Access Analyzer validates new and existing policies so that the policies adhere to the IAM policy language (JSON) and IAM best practices. IAM Access Analyzer provides more than 100 policy checks and actionable recommendations to help you author secure and functional policies. For more information, see [Validate policies with IAM Access Analyzer](https://docs.aws.amazon.com/IAM/latest/UserGuide/access-analyzer-policy-validation.html) in the *IAM User Guide*.
+ **Require multi-factor authentication (MFA)** – If you have a scenario that requires IAM users or a root user in your AWS account, turn on MFA for additional security. To require MFA when API operations are called, add MFA conditions to your policies. For more information, see [ Secure API access with MFA](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_configure-api-require.html) in the *IAM User Guide*.
For more information about best practices in IAM, see [Security best practices in IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html) in the *IAM User Guide*.
## Using the Amazon EMR on EKS console
To access the Amazon EMR on EKS console, you must have a minimum set of permissions. These permissions must allow you to list and view details about the Amazon EMR on EKS resources in your AWS account. If you create an identity-based policy that is more restrictive than the minimum required permissions, the console won't function as intended for entities (users or roles) with that policy.
You don't need to allow minimum console permissions for users that are making calls only to the AWS CLI or the AWS API. Instead, allow access to only the actions that match the API operation that they're trying to perform.
To ensure that users and roles can still use the Amazon EMR on EKS console, also attach the Amazon EMR on EKS `ConsoleAccess` or `ReadOnly` AWS managed policy to the entities. For more information, see [Adding permissions to a user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_change-permissions.html#users_change_permissions-add-console) in the *IAM User Guide*.
## Allow users to view their own permissions
This example shows how you might create a policy that allows IAM users to view the inline and managed policies that are attached to their user identity. This policy includes permissions to complete this action on the console or programmatically using the AWS CLI or AWS API.
```
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ViewOwnUserInfo",
"Effect": "Allow",
"Action": [
"iam:GetUserPolicy",
"iam:ListGroupsForUser",
"iam:ListAttachedUserPolicies",
"iam:ListUserPolicies",
"iam:GetUser"
],
"Resource": ["arn:aws:iam::*:user/${aws:username}"]
},
{
"Sid": "NavigateInConsole",
"Effect": "Allow",
"Action": [
"iam:GetGroupPolicy",
"iam:GetPolicyVersion",
"iam:GetPolicy",
"iam:ListAttachedGroupPolicies",
"iam:ListGroupPolicies",
"iam:ListPolicyVersions",
"iam:ListPolicies",
"iam:ListUsers"
],
"Resource": "*"
}
]
}
```
# Policies for tag-based access control
You can use conditions in your identity-based policy to control access to virtual clusters and job runs based on tags. For more information about tagging, see [Tagging your Amazon EMR on EKS resources](tag-resources.md).
The following examples demonstrate different scenarios and ways to use condition operators with Amazon EMR on EKS condition keys. These IAM policy statements are intended for demonstration purposes only and should not be used in production environments. There are multiple ways to combine policy statements to grant and deny permissions according to your requirements. For more information about planning and testing IAM policies, see the [IAM user Guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/).
**Important**
Explicitly denying permission for tagging actions is an important consideration. This prevents users from tagging a resource and thereby granting themselves permissions that you did not intend to grant. If tagging actions for a resource are not denied, a user can modify tags and circumvent the intention of the tag-based policies. For an example of a policy that denies tagging actions, see [Deny access to add and remove tags](#security_iam_TBAC_deny).
The examples below demonstrate identity-based permissions policies that are used to control the actions that are allowed with Amazon EMR on EKS virtual clusters.
## Allow actions only on resources with specific tag values
In the following policy example, the StringEquals condition operator tries to match dev with the value for the tag department. If the tag department hasn't been added to the virtual cluster, or doesn't contain the value dev, the policy doesn't apply, and the actions aren't allowed by this policy. If no other policy statements allow the actions, the user can only work with virtual clusters that have this tag with this value.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"emr-containers:DescribeVirtualCluster"
],
"Resource": [
"*"
],
"Condition": {
"StringEquals": {
"aws:ResourceTag/department": "dev"
}
},
"Sid": "AllowEMRCONTAINERDescribevirtualcluster"
}
]
}
```
------
You can also specify multiple tag values using a condition operator. For example, to allow actions on virtual clusters where the `department` tag contains the value `dev` or `test`, you could replace the condition block in the earlier example with the following.
```
"Condition": {
"StringEquals": {
"aws:ResourceTag/department": ["dev", "test"]
}
}
```
## Require tagging when a resource is created
In the example below, the tag needs to be applied when creating the virtual cluster.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"emr-containers:CreateVirtualCluster"
],
"Resource": [
"*"
],
"Condition": {
"StringEquals": {
"aws:RequestTag/department": "dev"
}
},
"Sid": "AllowEMRCONTAINERSCreatevirtualcluster"
}
]
}
```
------
The following policy statement allows a user to create a virtual cluster only if the cluster has a `department` tag, which can contain any value.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"emr-containers:CreateVirtualCluster"
],
"Resource": [
"*"
],
"Condition": {
"Null": {
"aws:RequestTag/department": "false"
}
},
"Sid": "AllowEMRCONTAINERSCreatevirtualcluster"
}
]
}
```
------
## Deny access to add and remove tags
The effect of this policy is to deny a user the permission to add or remove any tags on virtual clusters that are tagged with a `department` tag that contains the `dev` value.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Action": [
"emr-containers:TagResource",
"emr-containers:UntagResource"
],
"Resource": [
"*"
],
"Condition": {
"StringNotEquals": {
"aws:ResourceTag/department": "dev"
}
},
"Sid": "AllowEMRCONTAINERSTagresource"
}
]
}
```
------
# Troubleshooting Amazon EMR on EKS identity and access
Use the following information to help you diagnose and fix common issues that you might encounter when working with Amazon EMR on EKS and IAM.
**Topics**
+ [
## I am not authorized to perform an action in Amazon EMR on EKS
](#security_iam_troubleshoot-no-permissions)
+ [
## I am not authorized to perform iam:PassRole
](#security_iam_troubleshoot-passrole)
+ [
## I want to allow people outside of my AWS account to access my Amazon EMR on EKS resources
](#security_iam_troubleshoot-cross-account-access)
## I am not authorized to perform an action in Amazon EMR on EKS
If the AWS Management Console tells you that you're not authorized to perform an action, then you must contact your administrator for assistance. Your administrator is the person that provided you with your user name and password.
The following example error occurs when the `mateojackson` user tries to use the console to view details about a fictional `my-example-widget` resource but does not have the fictional `emr-containers:GetWidget` permissions.
```
User: arn:aws:iam::123456789012:user/mateojackson is not authorized to perform: emr-containers:GetWidget on resource: my-example-widget
```
In this case, Mateo asks his administrator to update his policies to allow him to access the `my-example-widget` resource using the `emr-containers:GetWidget` action.
## I am not authorized to perform iam:PassRole
If you receive an error that you're not authorized to perform the `iam:PassRole` action, your policies must be updated to allow you to pass a role to Amazon EMR on EKS.
Some AWS services allow you to pass an existing role to that service instead of creating a new service role or service-linked role. To do this, you must have permissions to pass the role to the service.
The following example error occurs when an IAM user named `marymajor` tries to use the console to perform an action in Amazon EMR on EKS. However, the action requires the service to have permissions that are granted by a service role. Mary does not have permissions to pass the role to the service.
```
User: arn:aws:iam::123456789012:user/marymajor is not authorized to perform: iam:PassRole
```
In this case, Mary's policies must be updated to allow her to perform the `iam:PassRole` action.
If you need help, contact your AWS administrator. Your administrator is the person who provided you with your sign-in credentials.
## I want to allow people outside of my AWS account to access my Amazon EMR on EKS resources
You can create a role that users in other accounts or people outside of your organization can use to access your resources. You can specify who is trusted to assume the role. For services that support resource-based policies or access control lists (ACLs), you can use those policies to grant people access to your resources.
To learn more, consult the following:
+ To learn whether Amazon EMR on EKS supports these features, see [How Amazon EMR on EKS works with IAM](security_iam_service-with-iam.md).
+ To learn how to provide access to your resources across AWS accounts that you own, see [Providing access to an IAM user in another AWS account that you own](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_common-scenarios_aws-accounts.html) in the *IAM User Guide*.
+ To learn how to provide access to your resources to third-party AWS accounts, see [Providing access to AWS accounts owned by third parties](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_common-scenarios_third-party.html) in the *IAM User Guide*.
+ To learn how to provide access through identity federation, see [Providing access to externally authenticated users (identity federation)](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_common-scenarios_federated-users.html) in the *IAM User Guide*.
+ To learn the difference between using roles and resource-based policies for cross-account access, see [Cross account resource access in IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies-cross-account-resource-access.html) in the *IAM User Guide*.
# Using Amazon EMR on EKS with AWS Lake Formation for fine-grained access control
With Amazon EMR release 7.7 and higher, you can leverage AWS Lake Formation to apply fine-grained access controls on AWS Glue Data Catalog tables that are backed by Amazon S3 buckets. This capability lets you configure table, row, column, and cell-level access controls for read queries within your Amazon EMR on EKS Spark Jobs.
**Topics**
+ [
# How Amazon EMR on EKS works with AWS Lake Formation
](security_iam_fgac-lf-works.md)
+ [
# Enable Lake Formation with Amazon EMR on EKS
](security_iam_fgac-lf-enable.md)
+ [
# Considerations and limitations
](security_iam_fgac-considerations.md)
+ [
# Troubleshooting
](security_iam_fgac-troubleshooting.md)
# How Amazon EMR on EKS works with AWS Lake Formation
Using Amazon EMR on EKS with Lake Formation lets you enforce a layer of permissions on each Spark Job to apply Lake Formation permission control when Amazon EMR on EKS executes jobs. Amazon EMR on EKS uses [Spark resource profiles](https://spark.apache.org/docs/latest/api/java/org/apache/spark/resource/ResourceProfile.html) to create two profiles to effectively execute jobs. The User Profile executes user-supplied code, while the system profile enforces Lake Formation policies. Each Lake Formation enabled Job utilizes two Spark drivers, one for the User profile, and another for the System profile. For more information, see What is [AWS Lake Formation](https://docs.aws.amazon.com/lake-formation/latest/dg/what-is-lake-formation.html).
The following is a high-level overview of how Amazon EMR on EKS gets access to data protected by Lake Formation security policies.
![\[Job security by means of Lake Formation\]](http://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/images/fgac_diagram_eks_spark.png)
The following steps describe this process:
1. A user submits a Spark Job to an AWS Lake Formation-enabled Amazon EMR on EKS virtual cluster.
1. The Amazon EMR on EKS service sets up the User Driver and runs the job in the User Profile. The User Driver runs a lean version of Spark that has no ability to launch tasks, requests executors, access Amazon S3 or the Glue Data Catalog. It only builds a Job plan.
1. The Amazon EMR on EKS service sets up a second driver called a System Driver and runs it in the System Profile (with a privileged identity). Amazon EKS sets up an encrypted TLS channel between the two drivers for communication. The User Driver uses the channel to send the job plans to the System Driver. The System Driver does not run user-submitted code. It runs full Spark and communicates with Amazon S3 and the Data Catalog for data access. It requests executors and compiles the Job Plan into a sequence of execution stages.
1. Amazon EMR on EKS service then runs the stages on executors. User Code in any stage is run exclusively on User profile executors.
1. Stages that read data from Data Catalog tables protected by Lake Formation or those that apply security filters are delegated to System executors.
# Enable Lake Formation with Amazon EMR on EKS
With Amazon EMR release 7.7 and higher, you can leverage AWS Lake Formation to apply fine-grained access controls on Data Catalog tables that are backed by Amazon S3. This capability lets you configure table, row, column, and cell level access controls for read queries within your Amazon EMR on EKS Spark Jobs.
This section covers how to create a security configuration and set up Lake Formation to work with Amazon EMR. It also describes how to create a virtual cluster with the Security Configuration that you created for Lake Formation. These sections are meant to be completed in sequence.
## Step 1: Set up Lake Formation-based column, row, or cell-level permissions
First, to apply row and column level permissions with Lake Formation, the data lake administrator for Lake Formation must set the **LakeFormationAuthorizedCaller** Session Tag. Lake Formation uses this session tag to authorize callers and provide access to the data lake.
Navigate to the AWS Lake Formation console and select the **Application integration settings** option from the **Administration** section in the sidebar. Then, check the box **Allow external engines to filter data in Amazon S3 locations registered with Lake Formation**. Add the **AWS Account IDs ** where the Spark Jobs would be running, and the **Session tag Values**.
![\[Application integration settings\]](http://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/images/application_integration_settings_fgac.png)
Note that the **LakeFormationAuthorizedCaller** Session Tag passed here is passed in the **SecurityConfiguration** later when you set up IAM roles, in section 3.
## Step 2: Setup EKS RBAC permissions
Second, you set up permissions for role-based access control.
### Provide EKS Cluster Permissions to the Amazon EMR on EKS service
The Amazon EMR on EKS Service must have EKS Cluster Role permissions so that it can create cross namespace permissions for the System Driver to spin off User executors in the User namespace.
**Create Cluster Role**
This sample defines permissions for a collection of resources.
```
vim emr-containers-cluster-role.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: emr-containers
rules:
- apiGroups: [""]
resources: ["namespaces"]
verbs: ["get"]
- apiGroups: [""]
resources: ["serviceaccounts", "services", "configmaps", "events", "pods", "pods/log"]
verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create", "patch", "delete", "watch"]
- apiGroups: ["apps"]
resources: ["statefulsets", "deployments"]
verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
- apiGroups: ["extensions", "networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
- apiGroups: ["rbac.authorization.k8s.io"]
resources: ["clusterroles","clusterrolebindings","roles", "rolebindings"]
verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"]
- apiGroups: ["kyverno.io"]
resources: ["clusterpolicies"]
verbs: ["create", "delete"]
---
```
```
kubectl apply -f emr-containers-cluster-role.yaml
```
**Create Cluster Role Bindings**
```
vim emr-containers-cluster-role-binding.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: emr-containers
subjects:
- kind: User
name: emr-containers
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: emr-containers
apiGroup: rbac.authorization.k8s.io
---
```
```
kubectl apply -f emr-containers-cluster-role-binding.yaml
```
### Provide Namespace access to the Amazon EMR on EKS service
Create two Kubernetes namespaces, one for User driver and executors, and another for System driver & executors, and enable Amazon EMR on EKS service access to submit Jobs in both User and System Namespaces. Follow the existing guide to provide access for each namespace, which is available at [Enable cluster access using `aws-auth`](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-cluster-access.html#setting-up-cluster-access-aws-auth).
## Step 3: Setup IAM Roles for user and system profile components
Third, you set up roles for specific components. A Lake Formation-enabled Spark Job has two components, User and System. The User driver and executors run in User namespace, and are tied to the JobExecutionRole that is passed in the StartJobRun API. The System driver and executors run in the System namespace, and are tied to the **QueryEngine** role.
### Configure Query Engine role
The QueryEngine role is tied to the System Space Components, and would have permissions to assume the **JobExecutionRole** with **LakeFormationAuthorizedCaller** Session tag. The IAM Permissions Policy of Query Engine role is the following:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AssumeJobRoleWithSessionTagAccessForSystemDriver",
"Effect": "Allow",
"Action": [
"sts:AssumeRole",
"sts:TagSession"
],
"Resource": [
"arn:aws:iam::*:role/JobExecutionRole"
],
"Condition": {
"StringLike": {
"aws:RequestTag/LakeFormationAuthorizedCaller": "EMR on EKS Engine"
}
}
},
{
"Sid": "AssumeJobRoleWithSessionTagAccessForSystemExecutor",
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": [
"arn:aws:iam::*:role/JobExecutionRole"
]
},
{
"Sid": "CreateCertificateAccessForTLS",
"Effect": "Allow",
"Action": [
"emr-containers:CreateCertificate"
],
"Resource": [
"*"
]
}
]
}
```
------
Configure the Trust policy of Query Engine role to trust the Kubernetes System namespace.
```
aws emr-containers update-role-trust-policy \
--cluster-name eks cluster \
--namespace eks system namespace \
--role-name query_engine_iam_role_name
```
For more information, see [Updating the role trust policy](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-trust-policy.html).
### Configure the Job Execution Role
Lake Formation permissions control access to AWS Glue Data Catalog resources, Amazon S3 locations, and the underlying data at those locations. IAM permissions control access to the Lake Formation and AWS Glue APIs and resources. Although you might have the Lake Formation permission to access a table in the Data Catalog (SELECT), your operation fails if you don’t have the IAM permission on the `glue:Get*` API operations.
IAM Permissions Policy of **JobExecutionRole**: The **JobExecution** Role should have the Policy Statements in its Permissions Policy.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "GlueCatalogAccess",
"Effect": "Allow",
"Action": [
"glue:Get*",
"glue:Create*",
"glue:Update*"
],
"Resource": [
"*"
]
},
{
"Sid": "LakeFormationAccess",
"Effect": "Allow",
"Action": [
"lakeformation:GetDataAccess"
],
"Resource": [
"*"
]
},
{
"Sid": "CreateCertificateAccessForTLS",
"Effect": "Allow",
"Action": [
"emr-containers:CreateCertificate"
],
"Resource": [
"*"
]
}
]
}
```
------
IAM Trust Policy for **JobExecutionRole**:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "TrustQueryEngineRoleForSystemDriver",
"Effect": "Allow",
"Action": [
"sts:AssumeRole",
"sts:TagSession"
],
"Resource": [
"arn:aws:iam::*:role/QueryExecutionRole"
],
"Condition": {
"StringLike": {
"aws:RequestTag/LakeFormationAuthorizedCaller": "EMR on EKS Engine"
}
}
},
{
"Sid": "TrustQueryEngineRoleForSystemExecutor",
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": [
"arn:aws:iam::*:role/QueryEngineRole"
]
}
]
}
```
------
Configure the Trust Policy of Job execution Role to trust the Kubernetes user namespace:
```
aws emr-containers update-role-trust-policy \
--cluster-name eks cluster \
--namespace eks User namespace \
--role-name job_execution_role_name
```
For more information, see [Update the trust policy of the job execution role](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-trust-policy.html).
## Step 4: Setup security configuration
To run a Lake Formation-enabled job, you must create a security configuration.
```
aws emr-containers create-security-configuration \
--name 'security-configuration-name' \
--security-configuration '{
"authorizationConfiguration": {
"lakeFormationConfiguration": {
"authorizedSessionTagValue": "SessionTag configured in LakeFormation",
"secureNamespaceInfo": {
"clusterId": "eks-cluster-name",
"namespace": "system-namespace-name"
},
"queryEngineRoleArn": "query-engine-IAM-role-ARN"
}
}
}'
```
Ensure that the Session Tag passed in the field **authorizedSessionTagValue** can authorize Lake Formation. Set the value to the one configured in Lake Formation, in [Step 1: Set up Lake Formation-based column, row, or cell-level permissions](#security_iam_fgac-lf-enable-permissions).
## Step 5: Create a virtual cluster
Create a Amazon EMR on EKS virtual cluster with a security configuration.
```
aws emr-containers create-virtual-cluster \
--name my-lf-enabled-vc \
--container-provider '{
"id": "eks-cluster",
"type": "EKS",
"info": {
"eksInfo": {
"namespace": "user-namespace"
}
}
}' \
--security-configuration-id SecurityConfiguraionId
```
Ensure the **SecurityConfiguration** Id from the previous step is passed, so that the Lake Formation authorization configuration is applied to all Jobs running on the virtual cluster. For more information, see [Register the Amazon EKS cluster with Amazon EMR]().
## Step 6: Submit a Job in the FGAC Enabled VirtualCluster
The Process for Job Submission is same for both non Lake Formation and Lake Formation jobs. For more information, see [Submit a job run with `StartJobRun`](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/emr-eks-jobs-submit.html).
The Spark Driver, Executor and Event Logs of the System Driver are stored in AWS Service Account’s S3 Bucket for debugging. We recommend configuring a customer-managed KMS Key in the Job Run to encrypt all logs stored in the AWS service bucket. For more information about enabling log encryption, see [Encrypting Amazon EMR on EKS logs](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/security_iam_fgac-logging-kms.html).
# Considerations and limitations
Note the following considerations and limitations when you use Lake Formation with Amazon EMR on EKS:
+ Amazon EMR on EKS supports fine-grained access control via Lake Formation only for Apache Hive, Apache Iceberg, Apache Hudi, and Delta table Formats. Apache Hive formats include Parquet, ORC, and xSV.
+ `DynamicResourceAllocation` is enabled by default, and you can't turn off `DynamicResourceAllocation` for Lake Formation jobs. As DRA `spark.dynamicAllocation.maxExecutors` configuration's default value is infinity, please configure an appropriate value based on your workload.
+ By default, `spark.dynamicAllocation.preallocateExecutors` is enabled in Amazon EMR Spark, which can cause excessive container churn when `spark.dynamicAllocation.initialExecutors` and `spark.dynamicAllocation.minExecutors` are not set. For recommended configurations to manage executor preallocation, see the [Performance](best-practices.md#performance) section in [Links to Amazon EMR on EKS best practices guides on GitHubRunning interactive workloads on Amazon EMR on EKS](best-practices.md).
+ Lake Formation-enabled jobs don’t support usage of customized EMR on EKS Images in System Driver and System Executors.
+ You can only use Lake Formation with Spark jobs.
+ EMR on EKS with Lake Formation only supports a single Spark session throughout a job.
+ EMR on EKS with Lake Formation only supports cross-account table queries shared through resource links.
+ The following aren't supported:
+ Resilient distributed datasets (RDD)
+ Spark streaming
+ Write with Lake Formation granted permissions
+ Access control for nested columns
+ EMR on EKS blocks functionalities that might undermine the complete isolation of system driver, including the following:
+ UDTs, HiveUDFs, and any user-defined function that involves custom classes
+ Custom data sources
+ Supply of additional jars for Spark extension, connector, or metastore `ANALYZE TABLE` command
+ To enforce access controls, `EXPLAIN PLAN` and DDL operations such as `DESCRIBE TABLE` don't expose restricted information.
+ Amazon EMR on EKS restricts access to system driver Spark logs on Lake Formation-enabled jobs. Since the system driver runs with more access, events and logs that the system driver generates can include sensitive information. To prevent unauthorized users or code from accessing this sensitive data, EMR on EKS disabled access to system driver logs. For troubleshooting, contact AWS support.
+ If you registered a table location with Lake Formation, the data access path goes through the Lake Formation stored credentials, regardless of the IAM permission for the EMR on EKS job execution role. If you misconfigure the role registered with the table location, jobs submitted that use the role with S3 IAM permission to the table location will fail.
+ Writing to a Lake Formation table uses IAM permission rather than Lake Formation granted permissions. If your job execution role has the necessary S3 permissions, you can use it to run write operations.
The following are considerations and limitations when using Apache Iceberg:
+ You can only use Apache Iceberg with session catalog and not arbitrarily named catalogs.
+ Iceberg tables that are registered in Lake Formation only support the metadata tables `history`, `metadata_log_entries`, `snapshots`, `files`, `manifests`, and `refs`. Amazon EMR hides the columns that might have sensitive data, such as `partitions`, `path`, and `summaries`. This limitation doesn't apply to Iceberg tables that aren't registered in Lake Formation.
+ Tables that you don't register in Lake Formation support all Iceberg stored procedures. The `register_table` and `migrate` procedures aren't supported for any tables.
+ We recommend that you use Iceberg DataFrameWriterV2 instead of V1.
For more information, see [Understanding Amazon EMR on EKS concepts and terminology](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/emr-eks-concepts.html) and [Enable cluster access for Amazon EMR on EKS](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-cluster-access.html).
## Disclaimer for data administrators
**Note**
When you grant access to Lake Formation resources to an IAM role for EMR on EKS, you must ensure the EMR cluster administrator or operator is a trusted administrator. This is particularly relevant for Lake Formation resources that are shared across multiple organizations and AWS accounts.
## Responsibilities for EKS administrators
+ The `System` namespace should be protected. No user or resource or entity or tooling would be allowed to have any Kubernetes RBAC permissions on the Kubernetes resources in the `System` namespace.
+ No user or resource or entity except the EMR on EKS service should have access to `CREATE` access to POD, CONFIG\$1MAP and SECRET in the `User` namespace.
+ `System` drivers and `System` executors contain sensitive data. So, Spark events, Spark driver logs, and Spark executor logs in the `System` namespace should not be forwarded to external log storage systems.
# Troubleshooting
## Logging
EMR on EKS uses Spark resources profiles to split job execution. Amazon EMR on EKS uses the user profile to run the code you supplied, while the system profile enforces Lake Formation policies. You can access the logs for the containers ran as the user profile by configuring the StartJobRun request with [MonitoringConfiguration](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/emr-eks-jobs-s3.html).
## Spark History Server
The Spark History Server have all Spark events generated from the user profile and redacted events generated from the system driver. You can see all of the containers from both the user and system drivers in the **Executors** tab. However, log links are available only for the user profile.
## Job failed with insufficient Lake Formation permissions
Make sure that your job execution role has the permissions to run `SELECT` and `DESCRIBE` on the table that you are accessing.
## Job with RDD execution failed
EMR on EKS currently doesn't support resilient distributed dataset (RDD) operations on Lake Formation-enabled jobs.
## Unable to access data files in Amazon S3
Make sure you have registered the location of the data lake in Lake Formation.
## Security validation exception
EMR on EKS detected a security validation error. Contact AWS support for assistance.
## Sharing AWS Glue Data Catalog and tables across accounts
You can share databases and tables across accounts and still use Lake Formation. For more information, see [Cross-account data sharing in Lake Formation](https://docs.aws.amazon.com/lake-formation/latest/dg/cross-account-permissions.html) and [How do I share AWS Glue Data Catalog and tables cross-account using AWS Lake Formation?](https://repost.aws/knowledge-center/glue-lake-formation-cross-account).
## Iceberg Job throwing initialization error not setting the AWS region
Message is the following:
```
25/02/25 13:33:19 ERROR SparkFGACExceptionSanitizer: Client received error with id = b921f9e6-f655-491f-b8bd-b2842cdc20c7,
reason = IllegalArgumentException, message = Cannot initialize
LakeFormationAwsClientFactory, please set client.region to a valid aws region
```
Make sure the Spark configuration `spark.sql.catalog.catalog_name.client.region` is set to a valid region.
## Iceberg Job throwing SparkUnsupportedOperationException
Message is the following:
```
25/02/25 13:53:15 ERROR SparkFGACExceptionSanitizer: Client received error with id = 921fef42-0800-448b-bef5-d283d1278ce0,
reason = SparkUnsupportedOperationException, message = Either glue.id or glue.account-id is set with non-default account.
Cross account access with fine-grained access control is only supported with AWS Resource Access Manager.
```
Make sure the Spark Configuration `spark.sql.catalog.catalog_name.glue.account-id` is set to a valid account id.
## Iceberg Job fail with "403 Access Denied" during MERGE operation
Message is the following:
```
software.amazon.awssdk.services.s3.model.S3Exception: Access Denied (Service: S3, Status Code: 403,
...
at software.amazon.awssdk.services.s3.DefaultS3Client.deleteObject(DefaultS3Client.java:3365)
at org.apache.iceberg.aws.s3.S3FileIO.deleteFile(S3FileIO.java:162)
at org.apache.iceberg.io.FileIO.deleteFile(FileIO.java:86)
at org.apache.iceberg.io.RollingFileWriter.closeCurrentWriter(RollingFileWriter.java:129)
```
Disable S3 Delete operations in Spark by adding the following property. `--conf spark.sql.catalog.s3-table-name.s3.delete-enabled=false`.
# Logging and monitoring
To detect incidents, receive alerts when incidents occur, and respond to them, use these options with Amazon EMR on EKS:
+ Monitor Amazon EMR on EKS with AWS CloudTrail ‐ [AWS CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/) provides a record of actions taken by a user, role, or an AWS service in Amazon EMR on EKS. It captures calls from the Amazon EMR console and code calls to the Amazon EMR on EKS API operations as events. This allows you to determine the request that was made to Amazon EMR on EKS, the IP address from which the request was made, who made the request, when it was made, and additional details. For more information, see [Logging Amazon EMR on EKS API calls using AWS CloudTrail](logging-using-cloudtrail.md).
+ Use CloudWatch Events with Amazon EMR on EKS ‐ CloudWatch Events delivers a near real-time stream of system events that describe changes in AWS resources. CloudWatch Events becomes aware of operational changes as they occur, responds to them, and takes corrective action as necessary, by sending messages to respond to the environment, activating functions, making changes, and capturing state information. To use CloudWatch Events with Amazon EMR on EKS, create a rule that triggers on an Amazon EMR on EKS API call via CloudTrail. For more information, see [Monitor jobs with Amazon CloudWatch Events](monitoring.md#monitoring-cloudwatch-events).
# Encrypting Amazon EMR on EKS logs with managed storage
The sections that follow show you how to configure encryption for logs.
## Enable encryption
To encrypt logs in managed storage with your own KMS key, use the following configuration when you submit a job run.
```
"monitoringConfiguration": {
"managedLogs": {
"allowAWSToRetainLogs":"ENABLED",
"encryptionKeyArn":"KMS key arn"
},
"persistentAppUI": "ENABLED"
}
```
The `allowAWSToRetainLogs` configuration allows AWS to retain system namespace logs when running a job using Native FGAC. The `persistentAppUI` configuration allows AWS to save event logs which are used to generate the Spark UI. The `encryptionKeyArn` is used to specify the KMS key ARN you want to use to encrypt the logs stored by AWS.
## Required permissions for log encryption
The user who submits the job or views the Spark UI must be allowed the actions `kms:DescribeKey`, `kms:GenerateDataKey`, and `kms:Decrypt` for the encryption key. These permissions are used to verify the validity of the key and check that the user has the necessary permissions to read and write logs encrypted with the KMS key. If the user who submits the job lacks the necessary key permissions, Amazon EMR on EKS rejects the job run submission.
**Example IAM Policy for Role Used to Call StartJobRun**
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"emr-containers:StartJobRun"
],
"Resource": [
"*"
],
"Effect": "Allow",
"Sid": "AllowEMRCONTAINERSStartjobrun"
},
{
"Action": [
"kms:DescribeKey",
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": [
"arn:aws:kms:*:*:key/key-id"
],
"Effect": "Allow",
"Sid": "AllowKMSDescribekey"
}
]
}
```
------
You must also configure the KMS key to allow the `persistentappui.elasticmapreduce.amazonaws.com` and `elasticmapreduce.amazonaws.com` Service Principals to `kms:GenerateDataKey` and `kms:Decrypt`. This allows EMR to read and write logs encrypted with the KMS key to managed storage.
**Example KMS Key Policy**
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:DescribeKey"
],
"Resource": [
"*"
],
"Condition": {
"StringLike": {
"kms:viaService": "emr-containers.*.amazonaws.com"
}
},
"Sid": "AllowKMSDescribekey"
},
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": [
"*"
],
"Condition": {
"StringLike": {
"kms:viaService": "emr-containers.*.amazonaws.com",
"kms:EncryptionContext:aws:emr-containers:virtualClusterId": "virtual cluster id"
}
},
"Sid": "AllowKMSDecryptGenerate"
},
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": [
"*"
],
"Condition": {
"StringLike": {
"kms:EncryptionContext:aws:emr-containers:virtualClusterId": "virtual cluster id"
},
"ArnLike": {
"aws:SourceArn": "arn:aws:emr-containers:*:*:/virtualclusters/virtual_cluster_id"
}
},
"Sid": "AllowKMSDecryptService"
}
]
}
```
------
As a security best practice, we recommend that you add the `kms:viaService`, `kms:EncryptionContext`, and `aws:SourceArn` conditions. These conditions help ensure the key is only used by Amazon EMR on EKS and only used for logs generated from jobs running in a specific virtual cluster.
# Logging Amazon EMR on EKS API calls using AWS CloudTrail
Amazon EMR on EKS is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or an AWS service in Amazon EMR on EKS. CloudTrail captures all API calls for Amazon EMR on EKS as events. The calls captured include calls from the Amazon EMR on EKS console and code calls to the Amazon EMR on EKS API operations. If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for Amazon EMR on EKS. If you don't configure a trail, you can still view the most recent events in the CloudTrail console in **Event history**. Using the information collected by CloudTrail, you can determine the request that was made to Amazon EMR on EKS, the IP address from which the request was made, who made the request, when it was made, and additional details.
To learn more about CloudTrail, see the [AWS CloudTrail User Guide](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-user-guide.html).
## Amazon EMR on EKS information in CloudTrail
CloudTrail is enabled on your AWS account when you create the account. When activity occurs in Amazon EMR on EKS, that activity is recorded in a CloudTrail event along with other AWS service events in **Event history**. You can view, search, and download recent events in your AWS account. For more information, see [Viewing events with CloudTrail Event history](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events.html).
For an ongoing record of events in your AWS account, including events for Amazon EMR on EKS, create a trail. A *trail* enables CloudTrail to deliver log files to an Amazon S3 bucket. By default, when you create a trail in the console, the trail applies to all AWS Regions. The trail logs events from all Regions in the AWS partition and delivers the log files to the Amazon S3 bucket that you specify. Additionally, you can configure other AWS services to further analyze and act upon the event data collected in CloudTrail logs. For more information, see the following:
+ [Overview for creating a trail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html)
+ [CloudTrail supported services and integrations](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-aws-service-specific-topics.html)
+ [Configuring Amazon SNS notifications for CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/configure-sns-notifications-for-cloudtrail.html)
+ [Receiving CloudTrail log files from multiple regions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/receive-cloudtrail-log-files-from-multiple-regions.html) and [Receiving CloudTrail log files from multiple accounts](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-receive-logs-from-multiple-accounts.html)
All Amazon EMR on EKS actions are logged by CloudTrail and are documented in [Amazon EMR on EKS API documentation](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/). For example, calls to the `CreateVirtualCluster`, `StartJobRun` and `ListJobRuns` actions generate entries in the CloudTrail log files.
Every event or log entry contains information about who generated the request. The identity information helps you determine the following:
+ Whether the request was made with root or AWS Identity and Access Management (IAM) user credentials.
+ Whether the request was made with temporary security credentials for a role or federated user.
+ Whether the request was made by another AWS service.
For more information, see the [CloudTrail user Identity element](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-user-identity.html).
## Understanding Amazon EMR on EKS log file entries
A trail is a configuration that enables delivery of events as log files to an Amazon S3 bucket that you specify. CloudTrail log files contain one or more log entries. An event represents a single request from any source and includes information about the requested action, the date and time of the action, request parameters, and so on. CloudTrail log files aren't an ordered stack trace of the public API calls, so they don't appear in any specific order.
The following example shows a CloudTrail log entry that demonstrates the [https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_ListJobRuns.html](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_ListJobRuns.html) action.
```
{
"eventVersion": "1.05",
"userIdentity": {
"type": "AssumedRole",
"principalId": "AIDACKCEVSQ6C2EXAMPLE:admin",
"arn": "arn:aws:sts::012345678910:assumed-role/Admin/admin",
"accountId": "012345678910",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "AIDACKCEVSQ6C2EXAMPLE",
"arn": "arn:aws:iam::012345678910:role/Admin",
"accountId": "012345678910",
"userName": "Admin"
},
"webIdFederationData": {},
"attributes": {
"mfaAuthenticated": "false",
"creationDate": "2020-11-04T21:49:36Z"
}
}
},
"eventTime": "2020-11-04T21:52:58Z",
"eventSource": "emr-containers.amazonaws.com",
"eventName": "ListJobRuns",
"awsRegion": "us-east-1",
"sourceIPAddress": "203.0.113.1",
"userAgent": "aws-cli/1.11.167 Python/2.7.10 Darwin/16.7.0 botocore/1.7.25",
"requestParameters": {
"virtualClusterId": "1K48XXXXXXHCB"
},
"responseElements": null,
"requestID": "890b8639-e51f-11e7-b038-EXAMPLE",
"eventID": "874f89fa-70fc-4798-bc00-EXAMPLE",
"readOnly": true,
"eventType": "AwsApiCall",
"recipientAccountId": "012345678910"
}
```
# Using Amazon S3 Access Grants with Amazon EMR on EKS
## S3 Access Grants overview for Amazon EMR on EKS
With Amazon EMR releases 6.15.0 and higher, Amazon S3 Access Grants provide a scalable access control solution that you can use to augment access to your Amazon S3 data from Amazon EMR on EKS. If you have a complex or large permission configuration for your S3 data, you can use Access Grants to scale S3 data permissions for users, roles, and applications.
Use S3 Access Grants to augment access to Amazon S3 data beyond the permissions granted by the runtime role or the IAM roles that are attached to the identities with access to your Amazon EMR on EKS cluster.
For more information, see [Managing access with S3 Access Grants for Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-access-grants.html) in the *Amazon EMR Management Guide* and [Managing access with S3 Access Grants](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-grants.html) in the *Amazon Simple Storage Service User Guide*.
This page describes the requirements to run a Spark job in Amazon EMR on EKS with S3 Access Grants integration. With Amazon EMR on EKS, S3 Access Grants requires an additional IAM policy statement in the execution role for your job, and an additional override configuration for the `StartJobRun` API. For steps to set up S3 Access Grants with other Amazon EMR deployments, see the following documentation:
+ [Using S3 Access Grants with Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-access-grants.html)
+ [Using S3 Access Grants with EMR Serverless](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/access-grants.html)
## Launch an Amazon EMR on EKS cluster with S3 Access Grants for data management
You can enable S3 Access Grants on Amazon EMR on EKS and launch a Spark job. When your application makes a request for S3 data, Amazon S3 provides temporary credentials that are scoped to the specific bucket, prefix, or object.
1. Set up a job execution role for your Amazon EMR on EKS cluster. Include the required IAM permissions that you need to run Spark jobs, `s3:GetDataAccess` and `s3:GetAccessGrantsInstanceForPrefix`:
```
{
"Effect": "Allow",
"Action": [
"s3:GetDataAccess",
"s3:GetAccessGrantsInstanceForPrefix"
],
"Resource": [ //LIST ALL INSTANCE ARNS THAT THE ROLE IS ALLOWED TO QUERY
"arn:aws_partition:s3:Region:account-id1:access-grants/default",
"arn:aws_partition:s3:Region:account-id2:access-grants/default"
]
}
```
**Note**
If you specify IAM roles that for job execution that have any additional permissions to access S3 directly, then users might be able to access data regardless of the permissions that you define in S3 Access Grants
1. Submit a job to your Amazon EMR on EKS cluster with an Amazon EMR release label of 6.15 or higher and the `emrfs-site` classification, as the following example shows. Replace the values in `red text` with the appropriate values for your usage scenario.
```
{
"name": "myjob",
"virtualClusterId": "123456",
"executionRoleArn": "iam_role_name_for_job_execution",
"releaseLabel": "emr-7.12.0-latest",
"jobDriver": {
"sparkSubmitJobDriver": {
"entryPoint": "entryPoint_location",
"entryPointArguments": ["argument1", "argument2"],
"sparkSubmitParameters": "--class main_class"
}
},
"configurationOverrides": {
"applicationConfiguration": [
{
"classification": "emrfs-site",
"properties": {
"fs.s3.s3AccessGrants.enabled": "true",
"fs.s3.s3AccessGrants.fallbackToIAM": "false"
}
}
],
}
}
```
## S3 Access Grants considerations with Amazon EMR on EKS
For important support, compatibility, and behavioral information when you use Amazon S3 Access Grants with Amazon EMR on EKS, see [S3 Access Grants considerations with Amazon EMR](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-access-grants-considerations.html) in the *Amazon EMR Management Guide*.
# Compliance validation for Amazon EMR on EKS
Third-party auditors assess the security and compliance of Amazon EMR on EKS as part of multiple AWS compliance programs. These include SOC, PCI, FedRAMP, HIPAA, and others.
# Resilience in Amazon EMR on EKS
The AWS global infrastructure is built around AWS Regions and Availability Zones. AWS Regions provide multiple physically separated and isolated Availability Zones, which are connected with low-latency, high-throughput, and highly redundant networking. With Availability Zones, you can design and operate applications and databases that automatically fail over between zones without interruption. Availability Zones are more highly available, fault tolerant, and scalable than traditional single or multiple data center infrastructures.
For more information about AWS Regions and Availability Zones, see [AWS Global Infrastructure](https://aws.amazon.com/about-aws/global-infrastructure/).
In addition to the AWS global infrastructure, Amazon EMR on EKS offers integration with Amazon S3 through EMRFS to help support your data resiliency and backup needs.
# Infrastructure security in Amazon EMR on EKS
As a managed service, Amazon EMR is protected by AWS global network security. For information about AWS security services and how AWS protects infrastructure, see [AWS Cloud Security](https://aws.amazon.com/security/). To design your AWS environment using the best practices for infrastructure security, see [Infrastructure Protection](https://docs.aws.amazon.com/wellarchitected/latest/security-pillar/infrastructure-protection.html) in *Security Pillar AWS Well‐Architected Framework*.
You use AWS published API calls to access Amazon EMR through the network. Clients must support the following:
+ Transport Layer Security (TLS). We require TLS 1.2 and recommend TLS 1.3.
+ Cipher suites with perfect forward secrecy (PFS) such as DHE (Ephemeral Diffie-Hellman) or ECDHE (Elliptic Curve Ephemeral Diffie-Hellman). Most modern systems such as Java 7 and later support these modes.
# Configuration and vulnerability analysis
AWS handles basic security tasks like guest operating system (OS) and database patching, firewall configuration, and disaster recovery. These procedures have been reviewed and certified by the appropriate third parties. For more details, see the following resources:
+ [Compliance validation for Amazon EMR on EKS](compliance.md)
+ [Shared Responsibility Model](https://aws.amazon.com/compliance/shared-responsibility-model/)
+ [Amazon Web Services: Overview of Security Processes](https://d0.awsstatic.com/whitepapers/Security/AWS_Security_Whitepaper.pdf) (whitepaper)
# Connect to Amazon EMR on EKS Using an interface VPC endpoint
You can connect directly to Amazon EMR on EKS using [Interface VPC endpoints (AWS PrivateLink)](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html) in your Virtual Private Cloud (VPC) instead of connecting over the internet. When you use an interface VPC endpoint, communication between your VPC and Amazon EMR on EKS is conducted entirely within the AWS network. Each VPC endpoint is represented by one or more [Elastic network interfaces (ENIs)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html) with private IP addresses in your VPC subnets.
The interface VPC endpoint connects your VPC directly to Amazon EMR on EKS without an internet gateway, NAT device, VPN connection, or AWS Direct Connect connection. The instances in your VPC don't need public IP addresses to communicate with the Amazon EMR on EKS API.
You can create an interface VPC endpoint to connect to Amazon EMR on EKS using the AWS Management Console or AWS Command Line Interface (AWS CLI) commands. For more information, see [Creating an Interface Endpoint](https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpce-interface.html#create-interface-endpoint).
After you create an interface VPC endpoint, if you enable private DNS hostnames for the endpoint, the default Amazon EMR on EKS endpoint resolves to your VPC endpoint. The default service name endpoint for Amazon EMR on EKS is in the following format.
```
emr-containers.Region.amazonaws.com
```
If you do not enable private DNS hostnames, Amazon VPC provides a DNS endpoint name that you can use in the following format.
```
VPC_Endpoint_ID.emr-containers.Region.vpce.amazonaws.com
```
For more information, see [Interface VPC Endpoints (AWS PrivateLink)](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html) in the Amazon VPC User Guide. Amazon EMR on EKS supports making calls to all of its [API Actions](https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_Operations.html) inside your VPC.
You can attach VPC endpoint policies to a VPC endpoint to control access for IAM principals. You can also associate security groups with a VPC endpoint to control inbound and outbound access based on the origin and destination of network traffic, such as a range of IP addresses. For more information, see [Controlling Access to Services with VPC Endpoints](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints-access.html).
## Create a VPC Endpoint Policy for Amazon EMR on EKS
You can create a policy for Amazon VPC endpoints for Amazon EMR on EKS to specify the following:
+ The principal that can or cannot perform actions
+ The actions that can be performed
+ The resources on which actions can be performed
For more information, see [Controlling Access to Services with VPC Endpoints](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints-access.html) in the Amazon VPC User Guide.
**Example VPC Endpoint Policy to Deny All Access From a Specified AWS Account**
The following VPC endpoint policy denies AWS account *123456789012* all access to resources using the endpoint.
```
{
"Statement": [
{
"Action": "*",
"Effect": "Allow",
"Resource": "*",
"Principal": "*"
},
{
"Action": "*",
"Effect": "Deny",
"Resource": "*",
"Principal": {
"AWS": [
"123456789012"
]
}
}
]
}
```
**Example VPC Endpoint Policy to Allow VPC Access Only to a Specified IAM Principal (User)**
The following VPC endpoint policy allows full access only to the IAM user *lijuan* in AWS account *123456789012*. All other IAM principals are denied access using the endpoint.
```
{
"Statement": [
{
"Action": "*",
"Effect": "Allow",
"Resource": "*",
"Principal": {
"AWS": [
"arn:aws:iam::123456789012:user/lijuan"
]
}
}
]
}
```
**Example VPC Endpoint Policy to Allow Read-Only Amazon EMR on EKS Operations**
The following VPC endpoint policy allows only AWS account *123456789012* to perform the specified Amazon EMR on EKS actions.
The actions specified provide the equivalent of read-only access for Amazon EMR on EKS. All other actions on the VPC are denied for the specified account. All other accounts are denied any access. For a list of Amazon EMR on EKS actions, see [Actions, Resources, and Condition Keys for Amazon EMR on EKS](https://docs.aws.amazon.com/IAM/latest/UserGuide/list_amazonemroneksemrcontainers.html).
```
{
"Statement": [
{
"Action": [
"emr-containers:DescribeJobRun",
"emr-containers:DescribeVirtualCluster",
"emr-containers:ListJobRuns",
"emr-containers:ListTagsForResource",
"emr-containers:ListVirtualClusters"
],
"Effect": "Allow",
"Resource": "*",
"Principal": {
"AWS": [
"123456789012"
]
}
}
]
}
```
**Example VPC Endpoint Policy Denying Access to a Specified Virtual Cluster**
The following VPC endpoint policy allows full access for all accounts and principals, but denies any access for AWS account *123456789012* to actions performed on the virtual cluster with cluster ID *A1B2CD34EF5G*. Other Amazon EMR on EKS actions that don't support resource-level permissions for virtual clusters are still allowed. For a list of Amazon EMR on EKS actions and their corresponding resource type, see [Actions, Resources, and Condition Keys for Amazon EMR on EKS](https://docs.aws.amazon.com/IAM/latest/UserGuide/list_amazonemroneksemrcontainers.html)- in the *AWS Identity and Access Management User Guide*.
```
{
"Statement": [
{
"Action": "*",
"Effect": "Allow",
"Resource": "*",
"Principal": "*"
},
{
"Action": "*",
"Effect": "Deny",
"Resource": "arn:aws:emr-containers:us-west-2:123456789012:/virtualclusters/A1B2CD34EF5G",
"Principal": {
"AWS": [
"123456789012"
]
}
}
]
}
```
# Set up cross-account access for Amazon EMR on EKS
You can set up cross-account access for Amazon EMR on EKS. Cross-account access enables users from one AWS account to run Amazon EMR on EKS jobs and access the underlying data that belongs to another AWS account.
## Prerequisites
To set up cross-account access for Amazon EMR on EKS, you’ll complete tasks while signed in to the following AWS accounts:
+ `AccountA` ‐ An AWS account where you have created an Amazon EMR on EKS virtual cluster by registering Amazon EMR with a namespace on an EKS cluster.
+ `AccountB` ‐ An AWS account that contains an Amazon S3 bucket or a DynamoDB table that you want your Amazon EMR on EKS jobs to access.
You must have the following ready in your AWS accounts before setting up cross-account access:
+ An Amazon EMR on EKS virtual cluster in `AccountA` where you want to run jobs.
+ A job execution role in `AccountA` that has the required permissions to run jobs in the virtual cluster. For more information, see [Create a job execution role](creating-job-execution-role.md) and [Using job execution roles with Amazon EMR on EKS](iam-execution-role.md).
## How to access a cross-account Amazon S3 bucket or DynamoDB table
To set up cross-account access for Amazon EMR on EKS, complete the following steps.
1. Create an Amazon S3 bucket, `cross-account-bucket`, in `AccountB`. For more information, see [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/gsg/CreatingABucket.html). If you want to have cross-account access to DynamoDB, you can also create a DynamoDB table in `AccountB`. For more information, see [Creating a DynamoDB table](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/getting-started-step-1.html).
1. Create a `Cross-Account-Role-B` IAM role in `AccountB` that can access the `cross-account-bucket`.
1. Sign in to the IAM console.
1. Choose **Roles** and create a new role: `Cross-Account-Role-B`. For more information about how to create IAM roles, see [Creating IAM roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create.html) in the IAM user Guide.
1. Create an IAM policy that specifies the permissions for `Cross-Account-Role-B` to access the `cross-account-bucket` S3 bucket, as the following policy statement demonstrates. Then attach the IAM policy to `Cross-Account-Role-B`. For more information, see [Creating a New Policy](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html) in the IAM user Guide.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::cross-account-bucket",
"arn:aws:s3:::cross-account-bucket/*"
],
"Sid": "AllowS3"
}
]
}
```
------
If DynamoDB access is required, create an IAM policy that specifies permissions to access the cross-account DynamoDB table. Then attach the IAM policy to `Cross-Account-Role-B`. For more information, see [Create a DynamoDB table](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_examples_dynamodb_specific-table.html) in the IAM user guide.
Following is a policy to access a DynamoDB table, `CrossAccountTable`.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:*"
],
"Resource": [
"arn:aws:dynamodb:us-east-1:*:table/CrossAccountTable"
],
"Sid": "AllowDYNAMODB"
}
]
}
```
------
1. Edit the trust relationship for the `Cross-Account-Role-B` role.
1. To configure the trust relationship for the role, choose the **Trust Relationships** tab in the IAM console for the role created in Step 2: `Cross-Account-Role-B`.
1. Select **Edit Trust Relationship**.
1. Add the following policy document, which allows `Job-Execution-Role-A` in `AccountA` to assume this `Cross-Account-Role-B` role.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": [
"arn:aws:iam::*:role/Job-Execution-Role-A"
],
"Sid": "AllowSTSAssumerole"
}
]
}
```
------
1. Grant `Job-Execution-Role-A` in `AccountA` with - STS Assume role permission to assume `Cross-Account-Role-B`.
1. In the IAM console for AWS account `AccountA`, select `Job-Execution-Role-A`.
1. Add the following policy statement to the `Job-Execution-Role-A` to allow the `AssumeRole` action on the `Cross-Account-Role-B` role.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": [
"arn:aws:iam::*:role/Cross-Account-Role-B"
],
"Sid": "AllowSTSAssumerole"
}
]
}
```
------
1. For Amazon S3 access, set the following `spark-submit` parameters (`spark conf`) while submitting the job to Amazon EMR on EKS.
**Note**
By default, EMRFS uses the job execution role to access the S3 bucket from the job. But when `customAWSCredentialsProvider` is set to `AssumeRoleAWSCredentialsProvider`, EMRFS uses the corresponding role that you specify with `ASSUME_ROLE_CREDENTIALS_ROLE_ARN` instead of the `Job-Execution-Role-A` for Amazon S3 access.
+ `--conf spark.hadoop.fs.s3.customAWSCredentialsProvider=com.amazonaws.emr.AssumeRoleAWSCredentialsProvider`
+ `--conf spark.kubernetes.driverEnv.ASSUME_ROLE_CREDENTIALS_ROLE_ARN=arn:aws:iam::AccountB:role/Cross-Account-Role-B \`
+ `--conf spark.executorEnv.ASSUME_ROLE_CREDENTIALS_ROLE_ARN=arn:aws:iam::AccountB:role/Cross-Account-Role-B \`
**Note**
You must set `ASSUME_ROLE_CREDENTIALS_ROLE_ARN` for both executor and driver `env` in the job spark configuration.
For DynamoDB cross-account access, you must set `--conf spark.dynamodb.customAWSCredentialsProvider=com.amazonaws.emr.AssumeRoleAWSCredentialsProvider`.
1. Run the Amazon EMR on EKS job with cross-account access, as the following example demonstrates.
```
aws emr-containers start-job-run \
--virtual-cluster-id 123456 \
--name myjob \
--execution-role-arn execution-role-arn \
--release-label emr-6.2.0-latest \
--job-driver '{"sparkSubmitJobDriver": {"entryPoint": "entryPoint_location", "entryPointArguments": ["arguments_list"], "sparkSubmitParameters": "--class --conf spark.executor.instances=2 --conf spark.executor.memory=2G --conf spark.executor.cores=2 --conf spark.driver.cores=1 --conf spark.hadoop.fs.s3.customAWSCredentialsProvider=com.amazonaws.emr.AssumeRoleAWSCredentialsProvider --conf spark.kubernetes.driverEnv.ASSUME_ROLE_CREDENTIALS_ROLE_ARN=arn:aws:iam::AccountB:role/Cross-Account-Role-B --conf spark.executorEnv.ASSUME_ROLE_CREDENTIALS_ROLE_ARN=arn:aws:iam::AccountB:role/Cross-Account-Role-B"}} ' \
--configuration-overrides '{"applicationConfiguration": [{"classification": "spark-defaults", "properties": {"spark.driver.memory": "2G"}}], "monitoringConfiguration": {"cloudWatchMonitoringConfiguration": {"logGroupName": "log_group_name", "logStreamNamePrefix": "log_stream_prefix"}, "persistentAppUI":"ENABLED", "s3MonitoringConfiguration": {"logUri": "s3://my_s3_log_location" }}}'
```