# Overview of Lake Formation permissions There are two main types of permissions in AWS Lake Formation: + Metadata access – Permissions on Data Catalog resources (*Data Catalog permissions*). These permissions enable principals to create, read, update, and delete metadata databases and tables in the Data Catalog. + Underlying data access – Permissions on locations in Amazon Simple Storage Service (Amazon S3) (*data access permissions* and *data location permissions*). + Data lake permissions enable principals to read and write data to *underlying* Amazon S3 locations—data pointed to by Data Catalog resources. + Data location permissions enable principals to create and alter metadata databases and tables that point to specific Amazon S3 locations. For both areas, Lake Formation uses a combination of Lake Formation permissions and AWS Identity and Access Management (IAM) permissions. The IAM permissions model consists of IAM policies. The Lake Formation permissions model is implemented as DBMS-style GRANT/REVOKE commands, such as `Grant SELECT on tableName to userName`. When a principal makes a request to access Data Catalog resources or underlying data, for the request to succeed, it must pass permission checks by both IAM and Lake Formation. ![\[A requestor's request must pass through two "doors" to get to resources: Lake Formation permissions and IAM permissions.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/permissions_doors.png) Lake Formation permissions control access to Data Catalog resources, Amazon S3 locations, and the underlying data at those locations. IAM permissions control access to the Lake Formation and AWS Glue APIs and resources. So although you might have the Lake Formation permission to create a metadata table in the Data Catalog (`CREATE_TABLE`), your operation fails if you don't have the IAM permission on the `glue:CreateTable` API. (Why a `glue:` permission? Because Lake Formation uses the AWS Glue Data Catalog.) **Note** Lake Formation permissions apply only in the Region in which they were granted. AWS Lake Formation requires that each principal (user or role) be authorized to perform actions on Lake Formation–managed resources. A principal is granted the necessary authorizations by the data lake administrator or another principal with the permissions to grant Lake Formation permissions. When you grant a Lake Formation permission to a principal, you can optionally grant the ability to pass that permission to another principal. You can use the Lake Formation API, the AWS Command Line Interface (AWS CLI), or the **Data permissions** and **Data locations** pages of the Lake Formation console to grant and revoke Lake Formation permissions. # Methods for fine-grained access control With a data lake, the goal is to have fine-grained access control to data. In Lake Formation, this means fine-grained access control to Data Catalog resources and Amazon S3 locations. You can achieve fine-grained access control with one of the following methods. | Method | Lake Formation Permissions | IAM Permissions | Comments | | --- | --- | --- | --- | | Method 1 | Open | Fine-grained | **This is the default method** for backward compatibility with AWS Glue. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/lake-formation/latest/dg/access-control-fine-grained.html) On the Lake Formation console, this method appears as **Use only IAM access control**. | | Method 2 | Fine-grained | Coarse-grained | **This is the recommended method.** [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/lake-formation/latest/dg/access-control-fine-grained.html) | **Important** Be aware of the following: By default, Lake Formation has the **Use only IAM access control** settings enabled for compatibility with existing AWS Glue Data Catalog behavior. We recommend that you disable these settings after you transition to using Lake Formation permissions. For more information, see [Changing the default settings for your data lake](change-settings.md). Data lake administrators and database creators have implicit Lake Formation permissions that you must understand. For more information, see [Implicit Lake Formation permissions](implicit-permissions.md). # Metadata access control For access control for Data Catalog resources, the following discussion assumes fine-grained access control with Lake Formation permissions and coarse-grained access control with IAM policies. There are two distinct methods for granting Lake Formation permissions on Data Catalog resources: + **Named resource access control** – With this method, you grant permissions on specific databases or tables by specifying database or table names. The grants have this form: Grant *permissions* to *principals* on *resources* [with grant option]. With the grant option, you can allow the grantee to grant the permissions to other principals. + **Tag-based access control** – With this method, you assign one or more LF-Tags to Data Catalog databases, tables, and columns, and grant permissions on one or more LF-Tags to principals. Each LF-Tag is a key-value pair, such as `department=sales`. A principal that has LF-Tags that match the LF-Tags on a Data Catalog resource can access that resource. This method is recommended for data lakes with a large number of databases and tables. It's explained in detail in [Lake Formation tag-based access control](tag-based-access-control.md). The permissions that a principal has on a resource is the union of the permissions granted by both the methods. The following table summarizes the available Lake Formation permissions on Data Catalog resources. The column headings indicate the resource on which the permission is granted. | Catalog | Database | Table | | --- | --- | --- | | CREATE\$1DATABASE | CREATE\$1TABLE | ALTER | | | ALTER | DROP | | | DROP | DESCRIBE | | | DESCRIBE | SELECT\$1 | | | | INSERT\$1 | | | | DELETE\$1 | For example, the `CREATE_TABLE` permission is granted on a database. This means that the principal is allowed to create tables in that database. The permissions with an asterisk (\$1) are granted on Data Catalog resources, but they apply to the underlying data. For example, the `DROP` permission on a metadata table enables you to drop the table from the Data Catalog. However, the `DELETE` permission granted on the same table enables you to delete the table's underlying data in Amazon S3, using, for example, a SQL `DELETE` statement. With these permissions, you also can view the table on the Lake Formation console and retrieve information about the table with the AWS Glue API. Thus, `SELECT`, `INSERT`, and `DELETE` are both Data Catalog permissions and data access permissions. When granting `SELECT` on a table, you can add a filter that includes or excludes one or more columns. This permits fine-grained access control on metadata table columns, limiting the columns that users of integrated services can see when running queries. This capability is not available using just IAM policies. There is also a special permission named `Super`. The `Super` permission enables a principal to perform every supported Lake Formation operation on the database or table on which it is granted. This permission can coexist with the other Lake Formation permissions. For example, you can grant `Super`, `SELECT`, and `INSERT` on a metadata table. The principal can perform all supported actions on the table, and when you revoke `Super`, the `SELECT` and `INSERT` permissions remain. For details on each permission, see [Lake Formation permissions reference](lf-permissions-reference.md). **Important** To be able to see a Data Catalog table created by another user, you must be granted at least one Lake Formation permission on the table. If you are granted at least one permission on the table, you can also see the table's containing database. You can grant or revoke Data Catalog permissions using the Lake Formation console, the API, or the AWS Command Line Interface (AWS CLI). The following is an example of an AWS CLI command that grants the user `datalake_user1` permission to create tables in the `retail` database. ``` aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "CREATE_TABLE" --resource '{ "Database": {"Name":"retail"}}' ``` The following is an example of a coarse-grained access control IAM policy that complements fine-grained access control with Lake Formation permissions. It permits all operations on any metadata database or table. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "glue:*Database*", "glue:*Table*", "glue:*Partition*" ], "Resource": "*" } ] } ``` ------ The next example is also coarse-grained but somewhat more restrictive. It permits read-only operations on all metadata databases and tables in the Data Catalog in the designated account and Region. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "glue:GetTables", "glue:SearchTables", "glue:GetTable", "glue:GetDatabase", "glue:GetDatabases" ], "Resource": "arn:aws:glue:us-east-1:111122223333:*" } ] } ``` ------ Compare these policies to the following policy, which implements IAM-based fine-grained access control. It grants permissions only on a subset of tables in the customer relationship management (CRM) metadata database in the designated account and Region. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "glue:GetTables", "glue:SearchTables", "glue:GetTable", "glue:GetDatabase", "glue:GetDatabases" ], "Resource": [ "arn:aws:glue:us-east-1:111122223333:catalog", "arn:aws:glue:us-east-1:111122223333:database/CRM", "arn:aws:glue:us-east-1:111122223333:table/CRM/P*" ] } ] } ``` ------ For more examples of coarse-grained access control policies, see [Lake Formation personas and IAM permissions reference](permissions-reference.md). # Underlying data access control When an integrated AWS service requests access to data in an Amazon S3 location that is access-controlled by AWS Lake Formation, Lake Formation supplies temporary credentials to access the data. To enable Lake Formation to control access to underlying data at an Amazon S3 location, you *register* that location with Lake Formation. After you register an Amazon S3 location, you can start granting the following Lake Formation permissions: + Data access permissions (`SELECT`, `INSERT`, and `DELETE)` on Data Catalog tables that point to that location. + Data location permissions on that location. Lake Formation data location permissions control the ability to create Data Catalog resources that point to particular Amazon S3 locations. Data location permissions provide an extra layer of security to locations within the data lake. When you grant the `CREATE_TABLE` or `ALTER` permission to a principal, you also grant data location permissions to limit the locations for which the principal can create or alter metadata tables. Amazon S3 locations are buckets or prefixes under a bucket, but not individual Amazon S3 objects. You can grant data location permissions to a principal by using the Lake Formation console, the API, or the AWS CLI. The general form of a grant is as follows: ``` grant DATA_LOCATION_ACCESS to principal on S3 location [with grant option] ``` If you include `with grant option`, the grantee can grant the permissions to other principals. Recall that Lake Formation permissions always work in combination with AWS Identity and Access Management (IAM) permissions for fine-grained access control. For read/write permissions on underlying Amazon S3 data, IAM permissions are granted as follows: When you register a location, you specify an IAM role that grants read/write permissions on that location. Lake Formation assumes that role when supplying temporary credentials to integrated AWS services. A typical role might have the following policy attached, where the registered location is the bucket `awsexamplebucket`. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject" ], "Resource": [ "arn:aws:s3:::amzn-s3-demo-bucket/*" ] }, { "Effect": "Allow", "Action": [ "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::amzn-s3-demo-bucket" ] } ] } ``` ------ Lake Formation provides a service-linked role that you can use during registration to automatically create policies like this. For more information, see [Using service-linked roles for Lake Formation](service-linked-roles.md). Therefore, registering an Amazon S3 location grants the required IAM `s3:` permissions on that location, where the permissions are specified by the role used to register the location. **Important** Avoid registering an Amazon S3 bucket that has **Requester pays** enabled. For buckets registered with Lake Formation, the role used to register the bucket is always viewed as the requester. If the bucket is accessed by another AWS account, the bucket owner is charged for data access if the role belongs to the same account as the bucket owner. For read/write access to underlying data, in addition to Lake Formation permissions, principals also need the `lakeformation:GetDataAccess` IAM permission. With this permission, Lake Formation grants the request for temporary credentials to access the data. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "lakeformation:GetDataAccess", "Resource": "*" } ] } ``` ------ In the above policy, you must set the Resource parameter to '\$1' (all. Specifying any other resource for this permission is not supported. This configuration ensures that Lake Formation can manage data access across your entire data lake environment efficiently. **Note** Amazon Athena requires the user to have the `lakeformation:GetDataAccess` permission. Other integrated services require their underlying execution role to have the `lakeformation:GetDataAccess` permission. This permission is included in the suggested policies in the [Lake Formation personas and IAM permissions reference](permissions-reference.md). To summarize, to enable Lake Formation principals to read and write underlying data with access controlled by Lake Formation permissions: + Register the Amazon S3 locations that contain the data with Lake Formation. + Principals who create Data Catalog tables that point to underlying data locations must have data location permissions. + Principals who read and write underlying data must have Lake Formation data access permissions on the Data Catalog tables that point to the underlying data locations. + Principals who read and write underlying data must have the `lakeformation:GetDataAccess` IAM permission when the underlying data location is registered with Lake Formation. **Note** The Lake Formation permissions model doesn't prevent access to Amazon S3 locations through the Amazon S3 API or console if you have access to them through IAM or Amazon S3 policies. You can attach IAM policies to principals to block this access. **More on data location permissions** Data location permissions govern the outcome of create and update operations on Data Catalog databases and tables. The rules are as follows: + A principal must have explicit or implicit data location permissions on an Amazon S3 location to create or update a database or table that specifies that location. + The explicit permission `DATA_LOCATION_ACCESS` is granted using the console, API, or AWS CLI. + Implicit permissions are granted when a database has a location property that points to a registered location, the principal has the `CREATE_TABLE` permission on the database, and the principal tries to create a table at that location or a child location. + If a principal is granted data location permissions on a location, the principal has data location permissions on all child locations. + A principal does not need data location permissions to perform read/write operations on the underlying data. It is sufficient to have the `SELECT` or `INSERT` data access permissions. Data location permissions apply only to creating Data Catalog resources that point to the location. Consider the scenario shown in the following diagram. ![\[Folder hierarchy and two databases, database A and B, with database B pointing to the Customer service folder.\]](http://docs.aws.amazon.com/lake-formation/latest/dg/images/location-permissions-example.png) In this diagram: + The Amazon S3 buckets `Products`, `Finance`, and `Customer Service` are registered with Lake Formation. + `Database A` has no location property, and `Database B` has a location property that points to the `Customer Service` bucket. + User `datalake_user` has `CREATE_TABLE` on both databases. + User `datalake_user` has been granted data location permissions only on the `Products` bucket. The following are the results when user `datalake_user` tries to create a catalog table in a particular database at a particular location. **Location where `datalake_user` tries to create a table** | Database and Location | Succeeds or Fails | Reason | | --- | --- | --- | | Database A at Finance/Sales | Fails | No data location permission | | Database A at Products | Succeeds | Has data location permission | | Database A at HR/Plans | Succeeds | Location is not registered | | Database B at Customer Service/Incidents | Succeeds | Database has location property at Customer Service | For more information, see the following: + [Adding an Amazon S3 location to your data lake](register-data-lake.md) + [Lake Formation permissions reference](lf-permissions-reference.md) + [Lake Formation personas and IAM permissions reference](permissions-reference.md)