# AWS Glue support for SAP OData AWS Glue supports SAP OData as follows: **Supported as a source?** Yes. You can use AWS Glue ETL jobs to query data from SAP OData. **Supported as a target?** Yes. You can use AWS Glue ETL jobs to write records into SAP OData. **Supported SAP OData API versions** The following SAP OData API versions are supported: + 2.0 **Supported sources** The following sources are supported: + ODP (Operational Data Provisioning) Sources: + BW Extractors (DataSources) + CDS Views + SLT + Non-ODP Sources, for example: + CDS View Services + RFC-based Services + Custom ABAP Services **Supported SAP Components** The following are minimum requirements: + You must enable catalog service for service discovery. + Configure operational data provisioning (ODP) data sources for extraction in the SAP Gateway of your SAP system. + **OData V2.0**: Enable the OData V2.0 catalog service(s) in your SAP Gateway via transaction `/IWFND/MAINT_SERVICE`. + Enable OData V2.0 services in your SAP Gateway via transaction `/IWFND/MAINT_SERVICE`. + Your SAP OData service must support client side pagination/query options such as `$top` and `$skip`. It must also support system query option `$count`. + You must provide the required authorization for the user in SAP to discover the services and extract data using SAP OData services. Refer to the security documentation provided by SAP. + If you want to use OAuth 2.0 as an authorization mechanism, you must enable OAuth 2.0 for the OData service and register the OAuth client per SAP documentation. + To generate an OData service based on ODP data sources, SAP Gateway Foundation must be installed locally in your ERP/BW stack or in a hub configuration. + For your ERP/BW applications, the SAP NetWeaver AS ABAP stack must be at 7.50 SP02 or above. + For the hub system (SAP Gateway), the SAP NetWeaver AS ABAP of the hub system must be 7.50 SP01 or above for remote hub setup. + For non-ODP sources, your SAP NetWeaver stack version must be 7.40 SP02 or above. **Supported Authentication Methods** The following authentication methods are supported: + Basic Authentication + OAuth 2.0 # Prerequisites Prior to initiating an AWS Glue job for data extraction from SAP OData using the SAP OData connection, complete the following prerequisites: + The relevant SAP OData Service must be activated in the SAP system, ensuring the data source is available for consumption. If the OData service is not activated, the Glue job will not be able to access or extract data from SAP. + Appropriate authentication mechanisms such as basic (custom) authentication or OAuth 2.0 must be configured in SAP to ensure that the AWS Glue job can successfully establish a connection with the SAP OData service. + Configure IAM policies to grant the AWS Glue job appropriate permissions for accessing SAP, Secrets Manager, and other AWS resources involved in the process. + If the SAP system is hosted within a private network, VPC connectivity must be configured to ensure that the AWS Glue job can securely communicate with SAP without exposing sensitive data over public internet. AWS Secrets Manager can be used to securely store sensitive information such as SAP credentials, which the AWS Glue job can dynamically retrieve at runtime. This approach eliminates the need to hard-code credentials, enhancing security and flexibility. The following prerequisites provide step-by-step guidance on how to set up each component for a smooth integration between AWS Glue and SAP OData. **Topics** + [ # SAP OData activation ](sap-odata-activation.md) + [ # IAM policies ](sap-odata-configuring-iam-permissions.md) + [ # Connectivity / VPC Connection ](sap-odata-connectivity-vpc-connection.md) + [ # SAP Authentication ](sap-odata-authentication.md) + [ # AWS Secrets Manager to store your Auth secret ](sap-odata-aws-secret-manager-auth-secret.md) # SAP OData activation Complete the following steps for SAP OData connection: ## ODP Sources Before you can transfer data from an ODP provider, you must meet the following requirements: + You have an SAP NetWeaver AS ABAP instance. + Your SAP NetWeaver instance contains an ODP provider that you want to transfer data from. ODP providers include: + SAP DataSources (Transaction code RSO2) + SAP Core Data Services ABAP CDS Views + SAP BW or SAP BW/4HANA systems (InfoObject, DataStore Object) + Real-time replication of Tables and DB-Views from SAP Source System via SAP Landscape Replication Server (SAP SLT) + SAP HANA Information Views in SAP ABAP based Sources + Your SAP NetWeaver instance has the SAP Gateway Foundation component. + You have created an OData service that extracts data from your ODP provider. To create the OData service, you use the SAP Gateway Service Builder. To access your ODP data, Amazon AppFlow calls this service by using the OData API. For more information, see [Generating a Service for Extracting ODP Data via OData](https://help.sap.com/docs/SAP_BPC_VERSION_BW4HANA/dd104a87ab9249968e6279e61378ff66/69b481859ef34bab9cc7d449e6fff7b6.html?version=11.0) in the SAP BW/4HANA documentation. + To generate an OData service based on ODP data sources, SAP Gateway Foundation must be installed locally in your ERP/BW stack or in a hub configuration. + For your ERP/BW applications, the SAP NetWeaver AS ABAP stack must be at 7.50 SP02 or above. + For the hub system (SAP Gateway), the SAP NetWeaver AS ABAP of the hub system must be 7.50 SP01 or above for remote hub setup. ## Non-ODP Sources + Your SAP NetWeaver stack version must be 7.40 SP02 or above. + You must enable catalog service for service discovery. + **OData V2.0**: The OData V2.0 catalog service(s) can be enabled in your SAP Gateway via transaction `/IWFND/MAINT_SERVICE` + Your SAP OData service must support client side pagination/query options such as `$top` and `$skip`. It must also support system query option `$count`. + For OAuth 2.0, you must enable OAuth 2.0 for the OData service and register the OAuth client per SAP documentation and set the authorized redirect URL as follows: + `https://.console.aws.amazon.com/gluestudio/oauth`, replacing `` with the region where AWS Glue is running, example: us-east-1. + You must enable secure setup for connecting over HTTPS. + You must provide required authorization for the user in SAP to discover the services and extract data using SAP OData services. Please refer to the security documentation provided by SAP. # IAM policies IAM policies ## Policies containing the API operations for creating and using connections The following sample policy describes the required AWS IAM permissions for creating and using connections. If you are creating a new role, create a policy that contains the following: ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "glue:ListConnectionTypes", "glue:DescribeConnectionType", "glue:RefreshOAuth2Tokens", "glue:ListEntities", "glue:DescribeEntity" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "secretsmanager:DescribeSecret", "secretsmanager:GetSecretValue", "secretsmanager:PutSecretValue" ], "Resource": "*" } ] } ``` ------ The role must grant access to all the resources used by the job, for example Amazon S3. If you don’t want to use the above method, alternatively use the following managed IAM policies. + [AWSGlueServiceRole](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole) – Grants access to resources that various AWS Glue processes require to run on your behalf. These resources include AWS Glue, Amazon S3, IAM, CloudWatch Logs, and Amazon EC2. If you follow the naming convention for resources specified in this policy, AWS Glue processes have the required permissions. This policy is typically attached to roles specified when defining crawlers, jobs, and development endpoints. + [AWSGlueConsoleFullAccess](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/AWSGlueConsoleFullAccess) – Grants full access to AWS Glue resources when an identity that the policy is attached to uses the AWS Management Console. If you follow the naming convention for resources specified in this policy, users have full console capabilities. This policy is typically attached to users of the AWS Glue console. + [SecretsManagerReadWrite](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/SecretsManagerReadWrite) – Provides read/write access to AWS Secrets Manager via the AWS Management Console. Note: this excludes IAM actions, so combine with `IAMFullAccess` if rotation configuration is required. **IAM Policies/Permissions needed to configure VPC** The following IAM permissions are required while using VPC connection for creating AWS Glue Connection. For more details, refer to [create an IAM policy for AWS Glue](https://docs.aws.amazon.com/glue/latest/dg/create-service-policy.html). ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ec2:CreateNetworkInterface", "ec2:DeleteNetworkInterface", "ec2:DescribeNetworkInterfaces" ], "Resource": [ "*" ] } ] } ``` ------ # Connectivity / VPC Connection Steps for VPC Connection: 1. Use existing VPC connection or create a new connection by following the [Amazon VPC documentation](https://docs.aws.amazon.com/vpc/latest/userguide/create-vpc.html). 1. Make sure you have NAT Gateway which routes the traffic to internet. 1. Choose VPC endpoint as Amazon S3 Gateway to create connection. 1. Enable DNS resolution and DNS hostname to use AWS provided DNS Services. 1. Go to created VPC and add necessary endpoints for different services like STS, AWS Glue, Secret Managers. 1. Choose Create Endpoint. 1. For Service Category, choose AWS Services. 1. For Service Name, choose the service that you are connecting to. 1. Choose VPC and Enable DNS Name. 1. VCP Endpoints required for VPC connection: 1. [STS](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sts_vpc_endpoint_create.html) 1. [AWS Glue](https://docs.aws.amazon.com/glue/latest/dg/vpc-interface-endpoints.html) 1. [Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/vpc-endpoint-overview.html) ## Security Group Configuration Security group must allow traffic to its listening port from AWS Glue VPC for AWS Glue to be able to connect to it. It is a good practice to restrict the range of source IP addresses as much as possible. AWS Glue requires special security group that allows all inbound traffic from itself. You can create a self-referencing rule that allows all traffic originating from the security group. You can modify an existing security group and specify the security group as source. Open the communication from the HTTPS ports of the URL endpoint (either NLB or SAP instance). ## Connectivity options + HTTPS connection with internal and external NLB, SSL certificate from certificate authority (CA), not self-signed SSL certificate + HTTPS connection with SAP instance SSL certificate from certificate authority (CA), not self-signed SSL certificate # SAP Authentication The SAP connector supports both CUSTOM (this is SAP BASIC authentication) and OAUTH authentication methods. ## Custom Authentication AWS Glue supports Custom (Basic Authentication) as a method for establishing connections to your SAP systems, allowing the use of a username and password for secure access. This auth type works well for automation scenarios as it allows using username and password up front with the permissions of a particular user in the SAP OData instance. AWS Glue is able to use the username and password to authenticate SAP OData APIs. In AWS Glue, basic authorization is implemented as custom authorization. For public SAP OData documentation for Basic Auth flow, see [HTTP Basic Authentication](https://help.sap.com/docs/SAP_SUCCESSFACTORS_PLATFORM/d599f15995d348a1b45ba5603e2aba9b/5c8bca0af1654b05a83193b2922dcee2.html). ## OAuth 2.0 Authentication AWS Glue also supports OAuth 2.0 as a secure authentication mechanism for establishing connections to your SAP systems. This enables seamless integration while ensuring compliance with modern authentication standards and enhancing the security of data access. ## AUTHORIZATION\$1CODE Grant Type The grant type determines how AWS Glue communicates with SAP OData to request access to your data. SAP OData supports only the `AUTHORIZATION_CODE` grant type. This grant type is considered "three-legged" OAuth as it relies on redirecting users to the third-party authorization server to authenticate the user. It is used when creating connections via the AWS Glue console. Users may still opt to create their own connected app in SAP OData and provide their own client ID and client secret when creating connections through the AWS Glue console. In this scenario, they will still be redirected to SAP OData to login and authorize AWS Glue to access their resources. This grant type results in a refresh token and access token. The access token is short lived, and may be refreshed automatically without user interaction using the refresh token. For public SAP OData documentation on creating a connected app for Authorization Code OAuth flow, see [Authentication Using OAuth 2.0](https://help.sap.com/docs/ABAP_PLATFORM_NEW/e815bb97839a4d83be6c4fca48ee5777/2e5104fd87ff452b9acb247bd02b9f9e.html). # AWS Secrets Manager to store your Auth secret AWS Secrets Manager You will need to store the SAP OData connection secrets in AWS Secrets Manager, configure the necessary permissions for retrieval as specified in the [IAM policies](sap-odata-configuring-iam-permissions.md) section, and use it while creating a connection. Use the AWS Management Console for AWS Secrets Manager to create a secret for your SAP source. For more information, see [Create an AWS Secrets Manager secret](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html). Details in AWS Secrets Manager should include the elements in the following code. ## Custom Authentication Secret You will need to enter your SAP system username in place of ** and its password in place of ** and True or False. In this context, setting `basicAuthDisableSSO` to `true` disables Single Sign-On (SSO) for Basic Authentication requests, requiring explicit user credentials for each request. Conversely, setting it to `false` allows the use of existing SSO sessions if available. ``` { "basicAuthUsername": "", "basicAuthPassword": "", "basicAuthDisableSSO": "", "customAuthenticationType": "CustomBasicAuth" } ``` ## OAuth 2.0 Secret In case you are using OAuth 2.0 as your authentication mechanism, the secret in the AWS Secrets Manager should have the **User Managed Client Application ClientId** in the following format. You will need to enter your SAP client secret in place of . ``` {"USER_MANAGED_CLIENT_APPLICATION_CLIENT_SECRET": "" } ```