# Connecting to Slack in AWS Glue Studio
Slack is an enterprise communications app that lets users send messages and attachments through various public and private channels. If you're a Slack user, you can connect AWS Glue to your Slack account. Then, you can use Slack as a data source in your ETL Jobs. Run these jobs to transfer data between Slack and AWS services or other supported applications.
**Topics**
+ [AWS Glue support for Slack](slack-support.md)
+ [Policies containing the API operations for creating and using connections](slack-configuring-iam-permissions.md)
+ [Configuring Slack](slack-configuring.md)
+ [Configuring Slack connections](slack-configuring-connections.md)
+ [Reading from Slack entities](slack-reading-from-entities.md)
+ [Slack connection options](slack-connection-options.md)
+ [Limitations](slack-limitations.md)
+ [Creating a new Slack account and configuring the client app](slack-new-account-creation.md)
# AWS Glue support for Slack
AWS Glue supports Slack as follows:
**Supported as a source?**
Yes. You can use AWS Glue ETL jobs to query data from Slack.
**Supported as a target?**
No.
**Supported Slack API versions**
Slack API v2.
# Policies containing the API operations for creating and using connections
The following sample policy describes the required IAM permissions for creating and using connections. If you are creating a new role, create a policy that contains the following:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"glue:ListConnectionTypes",
"glue:DescribeConnectionType",
"glue:RefreshOAuth2Tokens",
"glue:ListEntities",
"glue:DescribeEntity"
],
"Resource": "*"
}
]
}
```
------
You can also use the following managed IAM policies to allow access:
+ [ AWSGlueServiceRole ](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole) – Grants access to resources that various AWS Glue processes require to run on your behalf. These resources include AWS Glue, Amazon S3, IAM, Amazon CloudWatch Logs, and Amazon EC2. If you follow the naming convention for resources specified in this policy, AWS Glue processes have the required permissions. This policy is typically attached to roles specified when defining crawlers, jobs, and development endpoints.
+ [ AWSGlueConsoleFullAccess ](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/AWSGlueConsoleFullAccess) – Grants full access to AWS Glue resources when an identity that the policy is attached to uses the AWS Management Console. If you follow the naming convention for resources specified in this policy, users have full console capabilities. This policy is typically attached to users of the AWS Glue console.
# Configuring Slack
Before you can use AWS Glue to transfer data to or from Slack, you must meet these requirements:
## Minimum requirements
+ You must have a Slack account. For more information, see [Creating a new Slack account and configuring the client app](slack-new-account-creation.md).
If you meet these requirements, you’re ready to connect AWS Glue to your Slack account.
# Configuring Slack connections
Slack supports the `AUTHORIZATION_CODE` grant type for OAuth 2.
This grant type is considered “three-legged” OAuth as it relies on redirecting users to the third party authorization server to authenticate the user. It is used when creating connections via the AWS Glue Console. The AWS Glue Console will redirect the user to Slack where the user must login and allow AWS Glue the requested permissions to access their Slack instance.
Users may still opt to create their own connected app in Slack and provide their own client id and client secret when creating connections through the AWS Glue Console. In this scenario, they will still be redirected to Slack to login and authorize AWS Glue to access their resources.
This grant type results in a refresh token and access token. The access token expires after 1 hour of creation. A new access token can be fetched using the refresh token.
For more information on creating a connected app for Authorization Code OAuth flow, see [ Slack API ](https://api.slack.com/quickstart).
To configure an Slack connection:
1. In AWS Secrets Manager, create a secret with the following details. You must create a secret for the connection in AWS Glue.
1. For customer managed connected app – Secret should contain the connected app Consumer Secret with `USER_MANAGED_CLIENT_APPLICATION_CLIENT_SECRET` as key.
1. In AWS Glue Glue Studio, create a connection under **Data Connections** by following the steps below:
1. When selecting a **Connection type**, select Slack.
1. Provide the Slack environment.
1. Select the IAM role which AWS Glue can assume and has permissions for following actions:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:PutSecretValue",
"ec2:CreateNetworkInterface",
"ec2:DescribeNetworkInterfaces",
"ec2:DeleteNetworkInterface"
],
"Resource": "*"
}
]
}
```
------
1. Select the `secretName` which you want to use for this connection in AWS Glue to put the tokens.
1. Select the network options if you want to use your network.
1. Grant the IAM role associated with your AWS Glue job permission to read `secretName`.
# Reading from Slack entities
**Prerequisites**
+ A Slack object you would like to read from.
**Supported entities**
| Entity | Can be Filtered | Supports Limit | Supports Order By | Supports Select \$1 | Supports Partitioning |
| --- | --- | --- | --- | --- | --- |
| conversations | Yes | Yes | No | Yes | Yes |
**Example**
```
slack_read = glueContext.create_dynamic_frame.from_options(
connection_type="slack",
connection_options={
"connectionName": "connectionName",
"ENTITY_NAME": "conversations/C058W38R5J8"
}
)
```
**Slack entity and field details**
| Entity | Field | Data Type | Supported Operators |
| --- | --- | --- | --- |
| conversations | attachments | List | NA |
| conversations | bot\$1id | String | NA |
| conversations | blocks | List | NA |
| conversations | client\$1msg\$1id | String | NA |
| conversations | is\$1starred | Boolean | NA |
| conversations | last\$1read | String | NA |
| conversations | latest\$1reply | String | NA |
| conversations | reactions | List | NA |
| conversations | replies | List | NA |
| conversations | reply\$1count | Integer | NA |
| conversations | reply\$1users | List | NA |
| conversations | reply\$1users\$1count | Integer | NA |
| conversations | subscribed | Boolean | NA |
| conversations | subtype | String | NA |
| conversations | text | String | NA |
| conversations | team | String | NA |
| conversations | thread\$1ts | String | NA |
| conversations | ts | String | EQUAL\$1TO, BETWEEN, LESS\$1THAN, LESS\$1THAN\$1OR\$1EQUAL\$1TO, GREATER\$1THAN, GREATER\$1THAN\$1OR\$1EQUAL\$1TO |
| conversations | type | String | NA |
| conversations | user | String | NA |
| conversations | inviter | String | NA |
| conversations | root | Struct | NA |
| conversations | is\$1locked | Boolean | NA |
| conversations | files | List | NA |
| conversations | room | Struct | NA |
| conversations | upload | Boolean | NA |
| conversations | display\$1as\$1bot | Boolean | NA |
| conversations | channel | String | NA |
| conversations | no\$1notifications | Boolean | NA |
| conversations | permalink | String | NA |
| conversations | pinned\$1to | List | NA |
| conversations | pinned\$1info | Struct | NA |
| conversations | edited | Struct | NA |
| conversations | app\$1id | String | NA |
| conversations | bot\$1profile | Struct | NA |
| conversations | metadata | Struct | NA |
**Partitioning queries**
Additional spark options `PARTITION_FIELD`, `LOWER_BOUND`, `UPPER_BOUND`, `NUM_PARTITIONS` can be provided if you want to utilize concurrency in Spark. With these parameters, the original query would be split into `NUM_PARTITIONS` number of sub-queries that can be executed by spark tasks concurrently.
+ `PARTITION_FIELD`: the name of the field to be used to partition query.
+ `LOWER_BOUND`: an inclusive lower bound value of the chosen partition field.
For date, we accept the Spark date format used in Spark SQL queries. Example of valid value: `"2024-07-01T00:00:00.000Z"`.
+ `UPPER_BOUND`: an exclusive upper bound value of the chosen partition field.
+ `NUM_PARTITIONS`: number of partitions.
Entity wise partitioning field support details are captured in below table.
| Entity Name | Partitioning Field | Data Type |
| --- | --- | --- |
| conversations | ts | String |
**Example**
```
slack_read = glueContext.create_dynamic_frame.from_options(
connection_type="slack",
connection_options={
"connectionName": "connectionName",
"ENTITY_NAME": "conversations/C058W38R5J8",
"PARTITION_FIELD": "ts"
"LOWER_BOUND": "2022-12-01T00:00:00.000Z"
"UPPER_BOUND": "2024-09-23T15:00:00.000Z"
"NUM_PARTITIONS": "2"
}
)
```
# Slack connection options
The following are connection options for Slack:
+ `ENTITY_NAME`(String) - (Required) Used for Read. Supported entity name. Example: `conversations/C058W38R5J8`.
+ `SELECTED_FIELDS`(List) - Default: empty(SELECT \$1). Used for Read. Fields you want to select for the object.
+ `FILTER_PREDICATE`(String) - Default: empty. Used for Read. It should be in the Spark SQL format.
+ `QUERY`(String) - Default: empty. Used for Read. Full Spark SQL query.
+ `NUM_PARTITIONS`(Integer) - Default: 1. Used for Read. Number of partitions for read.
# Limitations
The following are limitations for the Slack connector:
+ Record based partitioning is not supported as connector does not provide any means to fetch the total number of records (messages) available in a given conversation.
# Creating a new Slack account and configuring the client app
**Creating a Slack account**
1. Open the [Slack home page](https://slack.com/intl/en-in/) to sign-up for an account.
1. Choose **SIGN UP WITH EMAIL ADDRESS**. Enter your email ID and choose **Continue**.
1. Enter the 6-character code sent to your email address, it will redirect you to create a workspace or to join an existing workspace.
1. Choose **Create a workspace** to create a new workspace. It will redirect you to answer a few questions as a part of the set-up process.
+ Name of company
+ Your name
+ To add colleagues by email
+ What's your team working on? (This will be the channel name)
1. Fill in the input fields for these questions and continue. Your account is now ready to be used.
**Creating a Slack developer app**
1. Log in to your Slack account and sign into your Slack workspace.
1. From the workspace menu, select **Tools and settings** and then select **Manage apps**.
1. From the Slack App Directory menu, select **Build**.
1. On the **Your Apps** page, select **Create an App**.
1. On the **Create an app** page, select **From scratch**.
1. In the **Name app & choose workspace** dialog box that opens, add an App name and **Pick a workspace to deploy your app in**. Then select **Create App**.
1. Note down your Client Id and Secret displayed in App Credentials
1. On the OAuth & Permissions sidebar, go to Scopes and choose **Add an OAuth Scope**. You can add the redirect URLs to your app for configuration to automatically generate the 'Add to Slack' button or to distribute your app. Scroll up to the Redirect URLs section and choose **Add New Redirect URL** and save.
1. Then, scroll to OAuth Tokens for Your Workspace section, and choose **Install to Workspace**.
1. On the dialog box that opens up informing you that the app that you created is requesting permission to access the Slack workspace you wanted to connect it to, select **Allow**.
1. On successful completion, the console will display a OAuth Tokens for Your Workspace screen.
1. From the OAuth Tokens for Your Workspace screen, copy and save the OAuth token you will use to connect to AWS Glue
1. Next, you retrieve your Slack team ID. From the Slack workspace menu, select **Tools and settings** and then select **Manage apps**. You'll find your team ID in the URL of the page that opens.
1. To publicly distribute your app, you can activate by heading over to the **Manage Distribution** button on the sidebar. Scroll down to the Share Your App with Other Workspaces section and choose **Remove Hard Coded Information**. Provide consent and choose **Active Public Distribution**.
1. Your app is now publicly distributed. To access the entity APIs, the app needs to be added to every workspace channel the user wants to access from.
1. Sign into your slack account and open the workspace whose channel needs to be accessed.
1. In the workspace, open the channel for which the app wants to access and choose the channel title. Select the **Integrations** tab from the pop-up and add the app. In this way, the app is integrated with the channel to have access to its API.
The OAuth 2.0 client ID must have one or more authorized redirect URLs. Redirect URLs have the following format:
**Note**
Appflow redirect URLs are subject to change post redirect URLs for AWS Glue platform are available. Client ID and Client Secret are from the settings for your OAuth 2.0 client ID.
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/glue/latest/dg/slack-new-account-creation.html)