# Connecting to Domo
Domo is a cloud-based dash-boarding tool. With Domo’s enterprise application platform, the foundation you need to extend Domo is in place, so you can build your custom solutions faster.
**Topics**
+ [AWS Glue support for Domo](domo-support.md)
+ [Policies containing the API operations for creating and using connections](domo-configuring-iam-permissions.md)
+ [Configuring Domo](domo-configuring.md)
+ [Configuring Domo connections](domo-configuring-connections.md)
+ [Reading from Domo entities](domo-reading-from-entities.md)
+ [Domo connection options](domo-connection-options.md)
+ [Domo limitations](domo-connection-limitations.md)
# AWS Glue support for Domo
AWS Glue supports Domo as follows:
**Supported as a source?**
Yes. You can use AWS Glue ETL jobs to query data from Domo.
**Supported as a target?**
No.
**Supported Domo API versions**
The following Domo API versions are supported:
+ v1
# Policies containing the API operations for creating and using connections
The following sample policy describes the required AWS IAM permissions for creating and using connections. If you are creating a new role, create a policy that contains the following:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"glue:ListConnectionTypes",
"glue:DescribeConnectionType",
"glue:RefreshOAuth2Tokens",
"glue:ListEntities",
"glue:DescribeEntity"
],
"Resource": "*"
}
]
}
```
------
If you don't want to use the above method, alternatively use the following managed IAM policies:
+ [AWSGlueServiceRole](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole) – Grants access to resources that various AWS Glue processes require to run on your behalf. These resources include AWS Glue, Amazon S3, IAM, CloudWatch Logs, and Amazon EC2. If you follow the naming convention for resources specified in this policy, AWS Glue processes have the required permissions. This policy is typically attached to roles specified when defining crawlers, jobs, and development endpoints.
+ [AWSGlueConsoleFullAccess](https://console.aws.amazon.com/iam/home#policies/arn:aws:iam::aws:policy/AWSGlueConsoleFullAccess) – Grants full access to AWS Glue resources when an identity that the policy is attached to uses the AWS Management Console. If you follow the naming convention for resources specified in this policy, users have full console capabilities. This policy is typically attached to users of the AWS Glue console.
# Configuring Domo
Before you can use AWS Glue to transfer data from Domo to supported destinations, you must meet these requirements:
## Minimum requirements
The following are minimum requirements:
+ You have a Domo account enabled for API access.
+ You have an app under your Domo developer account that provides the client credentials that AWS Glue uses to access your data securely when it makes authenticated calls to your account. For more information, see [Creating a Domo developer app](#domo-configuring-creating-developer-app).
If you meet these requirements, you’re ready to connect AWS Glue to your Domo account.
## Creating a Domo developer app
To get the Client ID and Client Secret you create a developer account.
1. Go to the [Domo developer login page.](https://developer.domo.com/manage-clients)
1. Choose **Login**.
1. Provide the domain name and click **Continue**.
1. Hover on **My Account** and choose **New Client**.
1. Provide the Name and Description and select the scope ("data") and choose **Create**.
1. Retrieve the generated **Client Id** and **Client Secret** from the new client created.
# Configuring Domo connections
Domo supports the CLIENT\$1CREDENTIALS grant type for OAuth2.
+ This grant type is considered two-legged OAuth as only the client application authenticates itself to the server, with no involvement to the user.
+ Users may opt to create their own connected app in Domo and provide their own client ID and client secret when creating connections through the AWS Glue console.
+ For public Domo documentation on creating a connected app for the Authorization Code OAuth flow, see [OAuth Authentication](https://developer.domo.com/portal/1845fc11bbe5d-api-authentication).
To configure a Domo connection:
1. In AWS Secrets Manager, create a secret with the following details:
1. For the customer managed connected app, the Secret should contain the connected app access token, `client_id`, and `client_secret`.
1. Note: you must create a secret for your connections in AWS Glue.
1. In AWS Glue Glue Studio, create a connection under **Data Connections** by following the steps below:
1. Under **Connections**, choose **Create connection**.
1. When selecting a **Data Source**, select Domo.
1. Select the AWS IAM role which AWS Glue can assume and has permissions for following actions:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:PutSecretValue",
"ec2:CreateNetworkInterface",
"ec2:DescribeNetworkInterfaces",
"ec2:DeleteNetworkInterface"
],
"Resource": "*"
}
]
}
```
------
1. Select the `secretName` which you want to use for this connection in AWS Glue to put the tokens.
1. Select the network options if you want to use your network.
1. Grant the IAM role associated with your AWS Glue job permission to read `secretName`.
# Reading from Domo entities
**Prerequisite**
A Domo object you would like to read from. You will need the object name such as Data Set or Data Permission Policies. The following table shows the supported entities.
**Supported entities for source**:
| Entity | Can be filtered | Supports limit | Supports Order by | Supports Select \$1 | Supports partitioning |
| --- | --- | --- | --- | --- | --- |
| Data Set | Yes | Yes | Yes | Yes | Yes |
| Data Permission Policies | No | No | No | Yes | No |
**Example**:
```
Domo_read = glueContext.create_dynamic_frame.from_options(
connection_type="domo",
connection_options={
"connectionName": "connectionName",
"ENTITY_NAME": "dataset",
"API_VERSION": "v1"
}
```
## Domo entity and field details
Entities with static metadata:
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/glue/latest/dg/domo-reading-from-entities.html)
For the following entity, Domo provides endpoints to fetch metadata dynamically, so that operator support is captured at the datatype level for the entity.
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/glue/latest/dg/domo-reading-from-entities.html)
## Partitioning queries
**Field-based partitioning**
You can provide the additional Spark options `PARTITION_FIELD`, `LOWER_BOUND`, `UPPER_BOUND`, and `NUM_PARTITIONS` if you want to utilize concurrency in Spark. With these parameters, the original query would be split into `NUM_PARTITIONS` number of sub-queries that can be executed by Spark tasks concurrently.
+ `PARTITION_FIELD`: the name of the field to be used to partition the query.
+ `LOWER_BOUND`: an **inclusive** lower bound value of the chosen partition field.
For the DateTime field, we accept the value in ISO format.
Example of valid value:
```
"2023-01-15T11:18:39.205Z"
```
For the Date field, we accept the value in ISO format.
Example of valid value:
```
"2023-01-15"
```
+ `UPPER_BOUND`: an **exclusive** upper bound value of the chosen partition field.
Example of valid value:
```
"2023-02-15T11:18:39.205Z"
```
+ `NUM_PARTITIONS`: the number of partitions.
Entity-wise partitioning field support details are captured in the following table:
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/glue/latest/dg/domo-reading-from-entities.html)
Example:
```
Domo_read = glueContext.create_dynamic_frame.from_options(
connection_type="domo",
connection_options={
"connectionName": "connectionName",
"ENTITY_NAME": "dataset",
"API_VERSION": "v1",
"PARTITION_FIELD": "permissionTime"
"LOWER_BOUND": "2023-01-15T11:18:39.205Z"
"UPPER_BOUND": "2023-02-15T11:18:39.205Z"
"NUM_PARTITIONS": "2"
}
```
**Record-based partitioning**
You can provide the additional Spark option `NUM_PARTITIONS` if you want to utilize concurrency in Spark. With this parameter, the original query would be split into `NUM_PARTITIONS` number of sub-queries that can be executed by Spark tasks concurrently.
In record based partitioning, the total number of records present is queried from Domo, and it is divided by the `NUM_PARTITIONS` number provided. The resulting number of records are then concurrently fetched by each sub-query.
Example:
```
Domo_read = glueContext.create_dynamic_frame.from_options(
connection_type="domo",
connection_options={
"connectionName": "connectionName",
"ENTITY_NAME": "dataset",
"API_VERSION": "v1",
"NUM_PARTITIONS": "2"
}
```
# Domo connection options
The following are connection options for Domo:
+ `ENTITY_NAME`(String) - (Required) Used for Read. The name of your object in Domo.
+ `API_VERSION`(String) - (Required) Used for Read. Domo Rest API version you want to use.
+ `SELECTED_FIELDS`(List) - Default: empty(SELECT \$1). Used for Read. Columns you want to select for the object.
+ `FILTER_PREDICATE`(String) - Default: empty. Used for Read. It should be in the Spark SQL format.
+ `QUERY`(String) - Default: empty. Used for Read. Full Spark SQL query.
+ `PARTITION_FIELD`(String) - Used for read. Field to be used to partition query.
+ `LOWER_BOUND`(String)- Used for read. An inclusive lower bound value of the chosen partition field.
+ `UPPER_BOUND`(String) - Used for read. An exclusive upper bound value of the chosen partition field.
+ `NUM_PARTITIONS`(Integer) - Default: 1. Used for read. Number of partitions for read.
# Domo limitations
The following are limitations or notes for Domo:
+ Due to an SDK limitation, filtration does not work as expected for the queryable fields that starts with '\$1' (for example: \$1BATCH\$1ID ).
+ Due to an API limitation, filtration works on the date prior to the date you provide. This also affects incremental pull. To overcome this limitation, select a date according to your time zone against UTC, for getting data for the required date.