# Connecting to ServiceNow ServiceNow is a cloud-based SaaS platform for automating IT management workflows. The ServiceNow platform easily integrates with other tools, letting users manage projects, teams and customer interactions using a variety of apps and plugins. As a ServiceNow user you can connect AWS Glue to your ServiceNow account. Then, you can use ServiceNow as a data source in your ETL jobs. Run these jobs to transfer data between ServiceNow and AWS services or other supported applications. **Topics** + [AWS Glue support for ServiceNow](servicenow-support.md) + [Policies containing the API operations for creating and using connections](servicenow-configuring-iam-permissions.md) + [Configuring ServiceNow](servicenow-configuring.md) + [Configuring ServiceNow connections](servicenow-configuring-connections.md) + [Reading from ServiceNow entities](servicenow-reading-from-entities.md) + [ServiceNow connection options](servicenow-connection-options.md) + [Limitations and notes for ServiceNow connector](servicenow-connector-limitations.md) # AWS Glue support for ServiceNow AWS Glue supports ServiceNow as follows: **Supported as a source?** Yes. You can use AWS Glue ETL jobs to query data from ServiceNow. **Supported as a target?** No. **Supported ServiceNow API versions** The following ServiceNow API versions are supported: + v2 For entity support per version specific, see Supported entities for Source. # Policies containing the API operations for creating and using connections The following sample policy describes the required AWS IAM permissions for creating and using connections. If you are creating a new role, create a policy that contains the following: ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "glue:ListConnectionTypes", "glue:DescribeConnectionType", "glue:RefreshOAuth2Tokens", "glue:ListEntities", "glue:DescribeEntity" ], "Resource": "*" } ] } ``` ------ If you don't want to use the above method, alternatively use the following managed IAM policies: + [AWSGlueServiceRole](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole) – Grants access to resources that various AWS Glue processes require to run on your behalf. These resources include AWS Glue, Amazon S3, IAM, CloudWatch Logs, and Amazon EC2. If you follow the naming convention for resources specified in this policy, AWS Glue processes have the required permissions. This policy is typically attached to roles specified when defining crawlers, jobs, and development endpoints. + [AWSGlueConsoleFullAccess](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/AWSGlueConsoleFullAccess) – Grants full access to AWS Glue resources when an identity that the policy is attached to uses the AWS Management Console. If you follow the naming convention for resources specified in this policy, users have full console capabilities. This policy is typically attached to users of the AWS Glue console. # Configuring ServiceNow Before you can use AWS Glue to transfer data from ServiceNow, you must meet these requirements: ## Minimum requirements The following are minimum requirements: + You have a ServiceNow account with email and password. For more information, see [Creating a ServiceNow account](#servicenow-configuring-creating-servicenow-account). + Your ServiceNow account is enabled for API access. All use of the ServiceNow API is available at no additional cost. If you meet these requirements, you’re ready to connect AWS Glue to your ServiceNow account. ## Creating a ServiceNow account To create a ServiceNow account: 1. Navigate to the sign up page on servicenow.com, enter your details, and click **Continue**. 1. When you receive a verification code in your registered mail, enter that code and choose **Verify**. 1. Set up multi-factor authentication or skip doing so. Your account is created and ServiceNow displays your profile. ## Creating a ServiceNow developer instance Request a developer instance after logging in to ServiceNow. 1. At the [ServiceNow login page](https://signon.service-now.com/x_snc_sso_auth.do?pageId=username), enter your account credentials. 1. Choose the **ServiceNow Developer Program**. ![\[\]](http://docs.aws.amazon.com/glue/latest/dg/images/servicenow-dev-program.png) 1. Choose **Request Instance** in the top right. 1. Enter your job responsibilities. Indicate your agreement to the terms of use, and choose **Finish setup**. 1. Once the instance is created, note your instance URL and credentials. ## Retrieving BasicAuth credentials To retrieve Basic Auth credentials for a free account: 1. At the [ServiceNow login page](https://signon.service-now.com/x_snc_sso_auth.do?pageId=username), enter your account credentials. 1. On the home page choose the edit profile section (top right corner) and choose **Manage Instance Password**. 1. Retrieve the login credentials such as username, password, and instance URL. **Note** If MFA is enabled for the account, append the MFA token to the end of the user's password in the basic auth: : For more information, see [Building applications](https://docs.servicenow.com/bundle/xanadu-application-development/page/build/custom-application/concept/build-applications.html) in the ServiceNow documentation. ## Creating OAuth 2.0 credentials To use OAuth2.0 in ServiceNow connector, you need to create an inbound client) to generate the Client ID and Client Secret: 1. At the [ServiceNow login page](https://signon.service-now.com/x_snc_sso_auth.do?pageId=username), enter your account credentials. 1. On the home page choose **Start Building**. 1. On the App Engine Studio page, search for **Application Registry**. 1. Choose **New** in the top right. 1. Choose the **Create an OAuth API endpoint for external clients** option. 1. Make any required changes to the OAuth configuration and choose **Update**. Example for Redirect URL: https://us-east-1.console.aws.amazon.com/gluestudio/oauth 1. Select the newly created OAuth client app to retrieve the Client ID and Client Secret. 1. Store the Client ID and Client Secret for further processing. To configure OAuth in a non-production developer account: 1. Create an authentication profile using the [Create an authentication profile](https://docs.servicenow.com/bundle/washingtondc-platform-security/page/integrate/authentication/task/create-an-authentication-profile.html) topic in the ServiceNow documentation. 1. In the Authentication Profile for OAuth, select **Type** as OAuth and select the above-created inbound client to set the **OAuth Entity**. 1. If there are multiple clients, then you need to create multiple authentication profiles to set the required OAuth entity in the authentication profile. 1. If not configured, create a REST API access policy, to give access to the TABLE API. See [Create REST API access policy](https://docs.servicenow.com/bundle/washingtondc-platform-security/page/integrate/authentication/task/create-api-access-policy.html). # Configuring ServiceNow connections The grant type determines how AWS Glue communicates with ServiceNow to request access to your data. Your choice affects the requirements that you must meet before you create the connection. ServiceNow supports only the AUTHORIZATION\$1CODE grant type for OAuth 2.0. + This grant type is considered "three-legged" OAuth as it relies on redirecting users to a third-party authorization server to authenticate the user. It is used when creating connections via the AWS Glue console. The AWS Glue console will redirect the user to ServiceNow where the user must login and allow AWS Glue the requested permissions to access their ServiceNow instance. + Users may still opt to create their own connected app in ServiceNow and provide their own client id and client secret when creating connections through the AWS Glue console. In this scenario, they will still be redirected to ServiceNow to login and authorize AWS Glue to access their resources. + This grant type results in a refresh token and access token. The access token is short lived, and may be refreshed automatically without user interaction using the refresh token. + For public ServiceNow documentation on creating a connected app for Authorization Code OAuth flow, see [Set up OAuth](https://docs.servicenow.com/bundle/vancouver-platform-security/page/administer/security/task/t_SettingUpOAuth.html). To configure a ServiceNow connection: 1. In AWS Secrets Manager, create a secret with the following details: 1. For basic authentication, the Secret should contain the connected app Consumer Secret with `USERNAME` and `PASSWORD` as key. 1. For an authorization code grant type, the Secret should contain the connected app Consumer Secret with `USER_MANAGED_CLIENT_APPLICATION_CLIENT_SECRET` as key. 1. Note: You must create a secret per connection in AWS Glue. 1. In AWS Glue Glue Studio, create a connection under **Data Connections** by following the steps below: 1. When selecting a **Connection type**, select ServiceNow. 1. Provide the INSTANCE\$1URL of the ServiceNow instance you want to connect to. 1. Select the AWS IAM role which AWS Glue can assume and has permissions for following actions: ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "secretsmanager:DescribeSecret", "secretsmanager:GetSecretValue", "secretsmanager:PutSecretValue", "ec2:CreateNetworkInterface", "ec2:DescribeNetworkInterfaces", "ec2:DeleteNetworkInterface" ], "Resource": "*" } ] } ``` ------ 1. Select the **Authentication Type** you want to use for this connection in AWS Glue. 1. Basic Auth: this auth type works well for automation scenarios as it allows to use username and password up front with the permissions of a particular user in the ServiceNow instance. AWS Glue is able to use the username and password to authenticate ServiceNow APIs. Enter the following inputs only in case of Basic Auth: `Username` and `Password`. 1. OAuth2: enter the following inputs only in case of OAuth2: `ClientId`, `ClientSecret`, `Authorization URL`, `Authorization Token URL`. 1. Select the `secretName` which you want to use for this connection in AWS Glue to put the tokens. 1. Select the network options if you want to use your network. 1. Grant the IAM role associated with your AWS Glue job permission to read `secretName`. # Reading from ServiceNow entities **Prerequisite** A ServiceNow Tables object you would like to read from. You will need the object name such as pa\$1bucket or incident. **Example**: ``` servicenow_read = glueContext.create_dynamic_frame.from_options( connection_type="servicenow", connection_options={ "connectionName": "connectionName", "ENTITY_NAME": "pa_buckets", "API_VERSION": "v2" "instanceUrl": "https://.service-now.com" } ) ``` **ServiceNow entity and field details**: For the following entities, ServiceNow provides endpoints to fetch metadata dynamically, so that operator support is captured at the datatype level for each entity. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/glue/latest/dg/servicenow-reading-from-entities.html) **Note** The Struct data type is converted to a String data type in the response of the connector. **Note** `DML_STATUS` is an additional user-defined attribute used for tracking CREATED/UPDATED records. ## Partitioning queries **Field base partitioning**: You can provide the additional Spark options `PARTITION_FIELD`, `LOWER_BOUND`, `UPPER_BOUND`, and `NUM_PARTITIONS` if you want to utilize concurrency in Spark. With these parameters, the original query would be split into `NUM_PARTITIONS` number of sub-queries that can be executed by Spark tasks concurrently. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/glue/latest/dg/servicenow-reading-from-entities.html) + `PARTITION_FIELD`: the name of the field to be used to partition the query. + `LOWER_BOUND`: an **inclusive** lower bound value of the chosen partition field. For the Datetime field, we accept the Spark timestamp format used in SPark SQL queries. Examples of valid value: ``` "2024-01-30T06:47:51.000Z" ``` + `UPPER_BOUND`: an **exclusive** upper bound value of the chosen partition field. + `NUM_PARTITIONS`: the number of partitions. The following table describes the entity partitioning field support details: Example: ``` servicenow_read = glueContext.create_dynamic_frame.from_options( connection_type="servicenow", connection_options={ "connectionName": "connectionName", "ENTITY_NAME": "pa_buckets", "API_VERSION": "v2", "instanceUrl": "https://.service-now.com" "PARTITION_FIELD": "sys_created_on" "LOWER_BOUND": "2024-01-30T06:47:51.000Z" "UPPER_BOUND": "2024-06-30T06:47:51.000Z" "NUM_PARTITIONS": "10" } ``` **Record-based partitioning**: You can provide the additional Spark option `NUM_PARTITIONS` if you want to utilize concurrency in Spark. With this parameter, the original query is split into `NUM_PARTITIONS` number of sub-queries that can be executed by Spark tasks concurrently. In record-based partitioning, the total number of records present is queried from the ServiceNow API, and it is divided by `NUM_PARTITIONS` number provided. The resulting number of records are then concurrently fetched by each sub-query. + `NUM_PARTITIONS`: the number of partitions. Example: ``` servicenow_read = glueContext.create_dynamic_frame.from_options( connection_type="servicenow", connection_options={ "connectionName": "connectionName", "ENTITY_NAME": "pa_buckets", "API_VERSION": "v2", "instanceUrl": "https://.service-now.com" "NUM_PARTITIONS": "2" } ``` # ServiceNow connection options The following are connection options for ServiceNow: + `ENTITY_NAME`(String) - (Required) Used for Read. The name of your object in ServiceNow. + `API_VERSION`(String) - (Required) Used for Read. ServiceNow Rest API version you want to use. For example: v1,v2,v3,v4. + `SELECTED_FIELDS`(List) - Default: empty(SELECT \$1). Used for Read. Columns you want to select for the object. + `FILTER_PREDICATE`(String) - Default: empty. Used for Read. It should be in the Spark SQL format. + `QUERY`(String) - Default: empty. Used for Read. Full Spark SQL query. + `PARTITION_FIELD`(String) - Used for Read. Field to be used to partition query. + `LOWER_BOUND`(String)- Used for Read. An inclusive lower bound value of the chosen partition field. For example: 2024-01-30T06:47:51.000Z. + `UPPER_BOUND`(String) - Used for Read. An exclusive upper bound value of the chosen partition field. For example: 2024-06-30T06:47:51.000Z. + `NUM_PARTITIONS`(Integer) - Default: 1. Used for Read. Number of partitions for read. For example: 10. + `INSTANCE_URL`(String) - (Required) A valid ServiceNow instance URL with format https://.service-now.com. + `PAGE_SIZE`(Integer) - Defines the page size for querying the records. The default page size is 1,000. When a page size is specified, ServiceNow returns only the defined number of records per API call, rather than the entire dataset. The connector will still provide the total number of records and handle pagination using your specified page size. If you require a larger page size, you can choose any value up to 10,000, which is the maximum allowed. Any specified page size exceeding 10,000 will be ignored. Instead, the system will use the maximum allowed page size. You can specify the page size in the AWS Glue Studio UI by adding a connection option `PAGE_SIZE` with your desired value. For example: 5000. # Limitations and notes for ServiceNow connector The following are limitations or notes for the ServiceNow connector: + As per [SaaS documentation](https://www.servicenow.com/docs/bundle/washingtondc-application-development/page/build/applications/reference/r_GlobalDefaultFields.html), `sys_created_on`, `sys_updated_on`, and `sys_mod_count` are system generated fields. The connector relies on SaaS APIs to provide these fields in the response body. + If SaaS doesn't generate these fields for any entity, filter based partitioning cannot be supported. + If SaaS APIs don't return `sys_created_on` and `sys_updated_on` fields in the response, `DML_STATUS` cannot be calculated. + Enhance read performance and efficiency + The ServiceNow connector now automatically sorts the records in ascending order by the `sys_id` field (must be present in metadata) when no ORDER BY clause is specified by the user. In this case, records will be paginated by the new optimized keyset based pagination. + If the ORDER BY clause is specified, the new optimization will not be used and the records will be fetched using the existing (user defined Order By and Offset-Limit based pagination) method.