AWS Mainframe Modernization Service (Managed Runtime Environment experience) is no longer open to new customers. For capabilities similar to AWS Mainframe Modernization Service (Managed Runtime Environment experience) explore AWS Mainframe Modernization Service (Self-Managed Experience). Existing customers can continue to use the service as normal. For more information, see [AWS Mainframe Modernization availability change](https://docs.aws.amazon.com/m2/latest/userguide/mainframe-modernization-availability-change.html). # File Transfer in AWS Mainframe Modernization File Transfer AWS Mainframe Modernization File Transfer lets you transfer and convert mainframe data sets to Amazon S3 for mainframe modernization, migration, and augmentation use cases. It simplifies the process of transferring data sets from your mainframe to the AWS Cloud. Key features include: discovery of source mainframe data sets and artifacts, and scalability and efficiency for faster data transfers to Amazon S3. File Transfer supports various mainframe data set types like sequential, PDS, GDS, GDG, and VSAM KSDS. The service transfers the data sets to an intermediate Amazon S3 bucket, converts them to the specified target code page, and then moves them to your desired target S3 bucket. **Topics** + [ # What is AWS Mainframe Modernization File Transfer? ](what-is-filetransfer.md) + [ # Install a File Transfer agent ](m2-agent-installation.md) + [ # Configure a File Transfer agent ](m2-agent-configuration.md) + [ # Create data transfer endpoints for File Transfer ](filetransfer-data-transfer-endpoints.md) + [ # Create transfer tasks in File Transfer ](filetransfer-transfer-tasks.md) + [ # Tutorial: Getting started with AWS Mainframe Modernization File Transfer ](tutorial-filetransfer-getting-started.md) + [ # Supported source and target encodings in AWS Mainframe Modernization File Transfer ](filetransfer-encodings.md) # What is AWS Mainframe Modernization File Transfer? What is File Transfer With AWS Mainframe Modernization File Transfer, you can transfer and convert datasets and files with a fully managed service to accelerate and simplify modernization, migration, and augmentation use cases to the AWS Mainframe Modernization service and Amazon S3. **Topics** + [ ## Benefits of AWS Mainframe Modernization File Transfer ](#filetransfer-benefit-overview) + [ ## How AWS Mainframe Modernization File Transfer works ](#filetransfer-how) ## Benefits of AWS Mainframe Modernization File Transfer AWS Mainframe Modernization File Transfer helps you transfer datasets from mainframe to Amazon S3. Some benefits include: + Discovery of source mainframe datasets and artifacts + Automated transfers and datasets conversion + Scalability, efficiency, and speed to achieve faster dataset transfers to AWS ## How AWS Mainframe Modernization File Transfer works The following figure is an overview of how AWS Mainframe Modernization File Transfer works on a conceptual level. ![\[The how it works for File Transfer feature with different phases.\]](http://docs.aws.amazon.com/m2/latest/userguide/images/filetransfer-overview.png) The following figure is an architectural overview of AWS Mainframe Modernization File Transfer feature. ![\[The architectural overview of the File Transfer feature representing data transfer between mainframe and AWS Cloud Region.\]](http://docs.aws.amazon.com/m2/latest/userguide/images/filetransfer-architecture.png) # Install a File Transfer agent Install a File Transfer agent You can use this document as a step-by-step guide to install an agent on the source mainframe. **Note** This guide is only for the mainframe system programmers. **Topics** + [ ## Step 1: Create a zFS data set for the M2-agent ](#allocate-dataset) + [ ## Step 2: Format the data set as zFS ](#format-dataset) + [ ## Step 3: Mount the filesystem ](#mount-filesystem) + [ ## Step 4: Verify the mount ](#verify-mount) + [ ## Step 5: Enter OMVS ](#enter-omvs) + [ ## Step 6: Set the agent installation directory environment variable ](#agent-directory) + [ ## Step 7: Set the work directory environment variable ](#work-directory) + [ ## Step 8: Create the work directory ](#create-work-directory) + [ ## Step 9: Copy the agent tar file and copy the work directory ](#agent-tar-package) + [ ## Step 10: Finish the agent installation ](#finish-agent-installation) ## Step 1: Create a zFS data set for the M2-agent Create a zFS for the M2-agent installation using the JCL (Job Control Language) below: ``` DEFINE EXEC PGM=IDCAMS SYSPRINT DD SYSOUT=A SYSIN DD * DEFINE CLUSTER (NAME(yourhlq.M2AGENT.ZFS) - VOLUMES(*) - LINEAR CYL(1000 200)) ``` ## Step 2: Format the data set as zFS After creating the data set, format it as a zFS filesystem. One way to do that is using the following JCL: ``` FORMAT EXEC PGM=IOEAGFMT, PARM=('-aggregate yourhlq.M2AGENT.ZFS -size 1200') , SYSPRINT DD SYSOUT=* ``` Submit this job and check if it completed successfully. ## Step 3: Mount the filesystem To mount the filesystem, use the `MOUNT` command. You can mount the filesystem in command line in ISPF or in batch. For example: ``` MOUNT FILESYSTEM('yourhlq.M2AGENT.ZFS') TYPE(ZFS) MODE(RDWR) MOUNTPOINT('/usr/lpp/aws/m2-agent') ``` You will use this mount point in step 6. **Note** Defining the mount path is optional and you should use an existing directory for this. ## Step 4: Verify the mount Verify that the filesystem is correctly mounted using `D OMVS,F` command or by checking within Unix System Service (USS). ## Step 5: Enter OMVS Use the following command to enter OMVS: ``` TSO OMVS ``` ## Step 6: Set the agent installation directory environment variable Use the following command to set the agent installation directory environment: ``` export AGENT_DIR=/usr/lpp/aws/m2-agent ``` **Note** Mount point is defined in step 3. ## Step 7: Set the work directory environment variable Use the following command to set the work directory environment variable: ``` export WORK_DIR=$AGENT_DIR/tmp ``` ## Step 8: Create the work directory Use the following command to set the work directory environment: ``` mkdir -p $WORK_DIR ``` ## Step 9: Copy the agent tar file and copy the work directory Download the agent tar file from AWS using the [ M2 agent link](https://drm0z31ua8gi7.cloudfront.net/filetransfer/m2-agent-v1.0.0.tar). The transfer mechanism will depend on your environment, but make sure that the tar file is transferred in binary mode. ## Step 10: Finish the agent installation Follow these steps to finish the agent installation. 1. Set the m2-agent version environment variable to the version currently being installed using the following command: ``` export M2_AGENT_VERSION=1.0.0 ``` 1. Extract the agent tar package using the following command: ``` tar -xpf m2-agent-$M2_AGENT_VERSION.tar -C $AGENT_DIR ``` 1. Create a `current-version` symbolic link to the current agent installation directory with the following command: ``` ln -s $AGENT_DIR/m2-agent-v$M2_AGENT_VERSION $AGENT_DIR/current-version ``` 1. Update and submit `CPY#PDS` to create the File Transfer agent data sets. **Note** JCL uses the `SYS2.AWS.M2 HLQ`. To create the File Transfer agent, update the three symbolic variables HLQ (High level qualifier), `VOLSER`, and `AGNTPATH` to be used later in the JCL: ``` oedit $AGENT_DIR/current-version/installation/CPY#PDS ``` **Note** This JCL is tailored for setting up certain aspects of the agent installation on the mainframe. It allocates necessary data sets and then copies specific files from the Unix filesystem to these data sets. # Configure a File Transfer agent Configure a File Transfer agent Once you have installed a file transfer agent, follow these steps to configure the agent. If you need to install a new agent, follow instructions on the [Install a File Transfer agent](m2-agent-installation.md) page. **Topics** + [ ## Step 1: Configure permissions and Started Task Control (STC) ](#configure-permissions) + [ ## Step 2: Create Amazon S3 buckets ](#filetransfer-s3-buckets) + [ ## Step 3: Create an AWS KMS customer managed key for encryption ](#filetransfer-kms-encryption) + [ ## Step 4: Create an AWS Secrets Manager secret for the mainframe credentials ](#filetransfer-secret-manager) + [ ## Step 5: Create an IAM policy ](#filetransfer-IAM-policy) + [ ## Step 6: Create an IAM user with long-term access credentials ](#filetransfer-create-IAM-user) + [ ## Step 7: Create an IAM role for the agent to assume ](#filetransfer-create-IAM-role) + [ ## Step 8: Agent configuration ](#agent-configuration) ## Step 1: Configure permissions and Started Task Control (STC) 1. Update and submit one of `SYS2.AWS.M2.SAMPLIB(SEC#RACF)` (for setting up RACF permissions) or `SYS2.AWS.M2.SAMPLIB(SEC#TSS)` (for setting up TSS permissions) in accordance with their instructions. These members were created by the previous `CPY#PDS` step. **Note** `SYS2.AWS.M2` should be replaced with the high-level qualifier (HLQ) chosen during installation. 1. Update the PWD export in the `SYS2.AWS.M2.SAMPLIB(M2AGENT)` STC JCL, if the default File Transfer agent directory path(`/usr/lpp/aws/m2-agent`) was changed. 1. Update the PROC according to your site standards: 1. Update the PROC card per your installation requirements. 1. Update the STEPLIB with the `M2 LOADLIB PDSE ALIAS`. 1. Edit PWD to point the agent installation path (only this is included). 1. Update `JAVA_HOME` if required. 1. Update and copy the `SYS2.AWS.M2.SAMPLIB(M2AGENT)` JCL to `SYS1.PROCLIB` or a one of the PROCLIBs in your `PROCLIB` concatenation. 1. Add `SYS2.AWS.M2.LOADLIB` to the APF list using the following command: ``` SETPROG APF ADD DSNAME(SYS2.AWS.M2.LOADLIB) SMS ``` 1. Set the agent’s group and owner to the agent user/group (M2USER/M2GROUP). Use the following command in the OMVS: ``` chown -R M2USER:M2GROUP $AGENT_DIR/current-version ``` **Note** Edit the M2USER and M2GROUP with the names you used in the security definitions job. ## Step 2: Create Amazon S3 buckets AWS Mainframe Modernization File Transfer requires an intermediate Amazon S3 bucket as a work area. We recommend creating a bucket specifically for this. Optionally, create a new target Amazon S3 bucket for the transferred data sets. Otherwise you can also use your existing Amazon S3 bucket. For more information on creating Amazon S3 buckets, see [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html). ## Step 3: Create an AWS KMS customer managed key for encryption **To create a customer managed key in AWS KMS** 1. Open the AWS KMS console at [https://console.aws.amazon.com/kms](https://console.aws.amazon.com/kms). 1. Choose **Customer managed keys** in left navigation pane. 1. Choose **Create key**. 1. Under **Configure key**, choose **Key type** as **Symmetric**, and **Key usage** as **Encrypt and decrypt**. Use other default configurations. 1. Choose **Next**. 1. In **Add labels**, add Alias and description for your key. 1. Choose **Next**. 1. Under **Define key administrative permissions**, choose at least one IAM user and role who administers this key. 1. Choose **Next**. 1. Optionally, under **Define key administrative permissions**, choose at least one IAM user and role who can use this key. 1. Choose **Next**. 1. In **Edit key policy** section, choose **Edit**, and add the following syntax to the **Key policy**. This allows the AWS Mainframe Modernization service to read and use these keys for encryption/decryption. **Important** Add the statement to the existing statements. Don't replace what's already in the policy. ``` { "Sid" : "Enable AWS M2 File Transfer Permissions", "Effect" : "Allow", "Principal" : { "Service" : "m2.amazonaws.com" }, "Action" : [ "kms:Encrypt", "kms:Decrypt" ], "Resource" : "*" }, ``` 1. Choose **Next**. 1. On the **Review** page, check all the details, and choose **Finish**. Copy and save the ARN for the customer managed key by opening the newly created KMS key. It will be used in the policy later. ## Step 4: Create an AWS Secrets Manager secret for the mainframe credentials Mainframe credentials are required to access the data sets to be transferred and these must be stored as an AWS Secrets Manager secret. **To create an AWS Secrets Manager secret** 1. Open Secrets manager console at [https://console.aws.amazon.com/secretsmanager](https://console.aws.amazon.com/secretsmanager). 1. Choose **Store a new secret**. 1. In **Choose Secret type**, choose **Other type of secret**. 1. Use the key value `userId` for the mainframe userId that has access to the data sets.. Use the key value `password` for the password field. 1. For **Encryption Key**, choose the AWS customer managed key created earlier. 1. Choose **Next**. 1. On the **Configure secret** page, provide a name and description. 1. On the same page, edit the **Resource permissions**, and use the following resource policy so the AWS Mainframe Modernization service can access it. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement" : [ { "Effect" : "Allow", "Principal" : { "Service" : "m2.amazonaws.com" }, "Action" : [ "secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret" ], "Resource" : "*" } ] } ``` ------ 1. Choose **Save** to save the updated permissions. 1. Choose **Next**. 1. Skip through **Configure rotations** page, and choose **Next**. 1. On the **Review** page, check all configurations and choose **Store** to save the secret. **Important** The `userId` and `password` secret keys are case-sensitive and must be entered as shown. ## Step 5: Create an IAM policy **To create a new policy with the permissions required for the agent** 1. Open the IAM console at [https://console.aws.amazon.com/iam](https://console.aws.amazon.com/iam). 1. Choose **Policies** under **Access management**. 1. Choose **Create policy**. 1. On **Specify permissions** page, under **Policy editor**, switch from the Visual editor to the JSON editor and replace the contents with the following template: 1. Replace the `111122223333` in the request-queue and response-queue ARN’s with your account. **Note** These are wildcard ARN’s that match the two Amazon SQS queues created during the data transfer endpoint initialization. After creating a File Transfer endpoint, optionally replace these ARN’s with the actual values from Amazon SQS. 1. Replace `file-transfer-endpoint-intermediate-bucket-arn` with the ARN of the transfer bucket created earlier. Leave the “/\$1” wildcard at the end. 1. Replace `kms-key-arn` with the ARN of the AWS KMS key created earlier. 1. Choose **Next**. 1. On the **Review and create** page, add the Policy name and description. 1. Choose **Create policy**. ## Step 6: Create an IAM user with long-term access credentials Create an IAM user that allows the mainframe agent to connect to your AWS account. The agent will connect with this user and then assume a role you define with permissions to use Amazon SQS response and request queues and to save datasets to Amazon S3 buckets. **To create this IAM user** 1. Navigate to the IAM console at [https://console.aws.amazon.com/iam](https://console.aws.amazon.com/iam). 1. Choose **Users** under **Access management**. 1. Choose **Create user**. 1. Add a meaningful **User name** under **User details**. For example, `Configure-ft-agent`. 1. Choose **Next**. 1. In the **Permissions options**, choose the **Attach policies directly** option but do not attach any permissions policies. These permissions will be managed by a role that will be attached. 1. Choose **Next**. 1. Review the details, and choose **Create user**. 1. Once the user is created, choose the user and open **Security credentials** tab. 1. Under **Access keys**, choose **Create access key**. 1. Then, choose **Other** when prompted for Use case. 1. Choose **Next**. 1. Optionally, you can set description tag such as, `Access key for configuring file transfer agent`. 1. Choose **Create access key**. 1. Copy, and securely save the generated **Access key**, and **Secret access key**. These will be used later. For more information on creating IAM access key, see [Managing access keys for IAM users](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). **Important** Save the **Access key** and **Secret access key** displayed on the last page of the access key creation wizard, before choosing **Done**. These keys are used to configure the mainframe agent, and cannot be retrieved later. **Note** Save the IAM user ARN used to set up a trust relationship with an IAM role. ## Step 7: Create an IAM role for the agent to assume **To create a new IAM role for the agent** 1. Choose **Roles** in the IAM console at [https://console.aws.amazon.com/iam](https://console.aws.amazon.com/iam). 1. Choose **Create role**. 1. On the **Select trusted entity** page, choose **Custom trust policy** for the **Trusted entity type**. 1. Replace the Custom trust policy with the following and replace `` with the ARN of the user created earlier. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Sid": "FileTransferAgent", "Effect": "Allow", "Principal": { "AWS": "" }, "Action": "sts:AssumeRole" } ] } ``` ------ 1. Choose **Next**. 1. In **Add Permissions**, filter for the **Policy name** you created earlier and choose it. 1. Choose **Next**. 1. Name the role, and choose **Create Role**. **Note** Save the *role name*, which you will use later to configure the mainframe agent. ## Step 8: Agent configuration **To configure the File Transfer agent** 1. Navigate to `$AGENT_DIR/current-version/config`. 1. Edit the agent’s configuration file `appication.properties` to add an environments configuration using the following command: ``` oedit $AGENT_DIR/current-version/config/application.properties ``` For example: ``` agent.environments[0].account-id= agent.environments[0].agent-role-name= agent.environments[0].access-key-id= agent.environments[0].secret-access-id= agent.environments[0].bucket-name= agent.environments[0].environment-name= agent.environments[0].region= zos.complex-name= ``` Where: + `AWS_ACCOUNT_ID` is the ID of the AWS account. + `AWS_IAM_ROLE_NAME` is the name of the IAM role created in the [Step 7: Create an IAM role for the agent to assume](#filetransfer-create-IAM-role). + `AWS_IAM_ROLE_ACCESS_KEY` is the access key of the IAM user created in [Step 6: Create an IAM user with long-term access credentials](#filetransfer-create-IAM-user). + `AWS_IAM_ROLE_SECRET_KEY` is the access secret key for the IAM user created in [Step 6: Create an IAM user with long-term access credentials](#filetransfer-create-IAM-user). + `AWS_S3_BUCKET_NAME` is the name of the transfer bucket created with the data transfer endpoint. + `AWS_REGION` is the region in which you configure the File Transfer agent. **Note** You can have the File Transfer agent transfer to multiple regions and accounts in AWS by defining multiple environments. + (Optional). `zos.complex-name` is the complex name you created when creating a File Transfer endpoint. **Note** This field is necessary only if you want to customize the complex name (which defaults to your sysplex name) that is the same as you defined when creating your File Transfer endpoint. For more information, see [Create data transfer endpoints for File Transfer](filetransfer-data-transfer-endpoints.md). **Important** There can be several such sections, as long as the index in brackets — `[0]`— is incremented for each. You must restart the agent for changes to take effect. **Requirements** 1. When a parameter is added or removed, the agent has to be stopped and started. Start the File transfer agent using the following command in the CLI: ``` /S M2AGENT ``` To stop the M2 agent, use the following command in CLI: ``` /P M2AGENT ``` 1. You can have the File Transfer agent configured to transfer data to multiple regions and accounts in AWS by defining environment entries. **Note** Replace the values with the parameter values you created and configured previously. ``` #Region 1 agent.environments[0].account-id=AWS_ACCOUNT_ID agent.environments[0].agent-role-name=AWS_IAM_ROLE_NAME agent.environments[0].access-key-id=AWS_IAM_ROLE_ACCESS_KEY agent.environments[0].secret-access-id=AWS_IAM_ROLE_SECRET_KEY agent.environments[0].bucket-name=AWS_S3_BUCKET_NAME agent.environments[0].environment-name=AWS_REGION agent.environments[0].region=AWS_REGION #Region 2 agent.environments[1].account-id=AWS_ACCOUNT_ID agent.environments[1].agent-role-name=AWS_IAM_ROLE_NAME agent.environments[1].access-key-id=AWS_IAM_ROLE_ACCESS_KEY agent.environments[1].secret-access-id=AWS_IAM_ROLE_SECRET_KEY agent.environments[1].bucket-name=AWS_S3_BUCKET_NAME agent.environments[1].environment-name=AWS_REGION agent.environments[1].region=AWS_REGION ``` # Create data transfer endpoints for File Transfer Create data transfer endpoints Data transfer endpoints enable connectivity with the source mainframe, and support high availability, scalability, and streamlined management of agents. Individual agents are installed on mainframe LPARs and can be grouped together into a data transfer endpoint. When a request is made to transfer a dataset, one agent in the data transfer endpoint will handle that specific transfer. To initiate data transfers, at least one agent on the data transfer endpoint must be online. This procedure assumes that you have completed the steps in [Set up for AWS Mainframe Modernization](setting-up.md) and [Configure File Transfer agent on the source mainframe](tutorial-filetransfer-getting-started.md#filetransfer-configure). ## Create data transfer endpoints To create data transfer endpoints for File Transfer, follow these steps in the AWS Mainframe Modernization console. **To create a data transfer endpoint** 1. Open the AWS Mainframe Modernization console at [https://console.aws.amazon.com/m2/](https://console.aws.amazon.com/m2/). 1. In the AWS Region selector, choose the region where you want to transfer files from your mainframe to an Amazon S3 bucket. 1. On the **Data transfer endpoints** page, under **File Transfer**, choose **Create data transfer endpoint**. 1. On the **Data transfer endpoint prerequisites** page, read all the instructions to make sure you have completed these steps on the source mainframe. Once confirmed, choose **Next**. 1. On the **Configure data transfer endpoint** page, add basic information for your data transfer endpoint. 1. In the basic information section, enter your data transfer endpoint name. **Note** The data transfer endpoint name must match the Sysplex name, unless you specify a complex name in the agent configuration. 1. An optional description. 1. The KMS key used to encrypt the secret. **Note** You must add the following resource-based policy for KMS so the AWS Mainframe Modernization service can read and use these keys for encryption/decryption: ``` { "Sid" : "Enable AWS M2 Permissions", "Effect" : "Allow", "Principal" : { "Service" : "m2.amazonaws.com" }, "Action" : [ "kms:Encrypt", "kms:Decrypt" ], "Resource" : "*" } ``` 1. Specify the **S3 location for intermediate data**, which is the intermediate S3 location where transferred datasets from the mainframe are stored before they are converted and transferred to the target Amazon S3 bucket. . **Note** It's recommended that you create a new Amazon S3 bucket for your transfer tasks. For additional information, see [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html). You can also browse your existing Amazon S3 buckets by choosing **Browse S3** option. 1. After entering required fields, choose **Next**. 1. On the **Review and create data transfer endpoint** page, check if you have completed prerequisites, and review basic information. Once confirmed, choose **Create data transfer endpoint**. You will be redirected to the **Data transfer endpoints overview** page where you can see the list of all data transfer endpoints. You will also be able to see the data transfer endpoints that are available or have failed. You can also search data transfer endpoints by name and access additional information for each available agent. # Create transfer tasks in File Transfer Create transfer tasks Transfer tasks are used to specify the data sets to be transferred from the mainframe to Amazon S3 and allow you to choose the code page conversion options. These instructions assume that you have completed the steps in [Set up for AWS Mainframe Modernization](setting-up.md) and have created [Create data transfer endpoints for File Transfer](filetransfer-data-transfer-endpoints.md). **Topics** + [ ## Create transfer tasks ](#filetransfer-console-create-task) + [ ## View transfer tasks ](#filetransfer-console-view-task) ## Create transfer tasks To create transfer tasks in File Transfer, follow these steps in the AWS Mainframe Modernization console. **To create a transfer task** **Important** You must have at least one data transfer endpoint to create new transfer tasks. 1. Open the AWS Mainframe Modernization console at [https://console.aws.amazon.com/m2/](https://console.aws.amazon.com/m2/). 1. In the AWS Region selector, choose the Region where you want to transfer files from your mainframe to an Amazon S3 bucket. 1. On the **Transfer tasks** page, you can choose any data transfer endpoint to create transfer tasks. 1. On the **Create transfer task** page, set up properties for your transfer task. If you have not created any transfer tasks previously, you can create your first one by choosing the **Create Transfer task** option. + On this page, enter the basic information of your transfer task, including the transfer task name, description, and secret key. **Note** Encrypt the secret using the KMS key defined with the data transfer endpoint. The secret should contain the mainframe credentials needed to access data sets on the mainframe using the `userId` and `password` keys. For more information, see the [AWS Secrets Manager secret](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html). You must configure the secret key with the following resource-based policy so that AWS Mainframe Modernization service can access it to perform data transfer task. **** ``` { "Version":"2012-10-17", "Statement" : [ { "Effect" : "Allow", "Principal" : { "Service" : "m2.amazonaws.com" }, "Action" : [ "secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret" ], "Resource" : "*" } ] } ``` **Note** The current maximum supported dataset size for transfer is 90 GB. + Next, select the target Amazon S3 bucket location where the target data sets from the mainframe will be transferred. + The previously chosen data transfer endpoint will be selected. You can also select another endpoint from the available endpoints. 1. Choose **Next**. 1. On the **Add data sets** page, in the **Transfer task configuration** section, you can choose to either configure your transfer task in binary mode or to convert and transfer your data sets. + **Transfer in binary mode** option allows you to transfer data sets by skipping code page conversions and retaining their Record Descriptor Word (RDW) bytes. + **Transfer and convert data sets** option allows you to transfer data sets by setting the source and target code pages for your data sets. You can see the available code pages for File Transfer on the [Supported source and target encodings in AWS Mainframe Modernization File Transfer](filetransfer-encodings.md) page. 1. Enter your query in the **Search mainframe for data sets** to search the mainframe for data sets to be included in your transfer task. Choose **View data sets**. The following wildcard symbols can be used as part of the data set search criteria for mainframe: + A single asterisk (\$1) as a qualifier (between periods or after the final period) matches a single qualifier in that position. + A single asterisk (\$1) within a qualifier matches zero or more characters in that position. + A double asterisk (\$1\$1) as a qualifier (between periods or after the final period) matches zero or more qualifiers in that position. + A double asterisk (\$1\$1) within a qualifier is not a valid query. + A single percent sign (%) matches any single alphanumeric or national character in that position. You can use up to eight percent signs in each qualifier. **Note** We suggest always ending your search criteria with a period followed by a double asterisk (.\$1\$1) and then refine the search further, if needed. For more information on wildcard rules, see the [Filtering data set names](https://www.ibm.com/docs/en/zos/3.1.0?topic=processed-filtering-by-data-set-names) in the *IBM documentation*. 1. These data sets will load under **Mainframe data sets** section, where you can search or choose one or more data sets you want to configure code page conversions for. These chosen data sets will be displayed in the **Added data sets** section. If no data sets are loaded, you need to revisit step 7. **Note** You can select data sets from multiple search queries and add them to your transfer task. 1. In the **Added data sets** section, you will see name, type, and volume name of your data sets. **Important** For **Transfer and convert data sets** option, you need to manually enter the source code page and target code page for each of your chosen data set. *Source code page* is the source data set format, and *target code page* is the target data set format used to convert the data sets and store them in the target Amazon S3 bucket. 1. After confirming the data sets in the **Added data sets** section, (and source and target code pages for Transfer and convert data sets option), choose **Next**. 1. On the **Review and create** page, you can review or edit information for your transfer task. 1. Then, choose **Create transfer task**. **Important** Choosing the **Create transfer task** button will start the data transfer, which is billable per the [AWS Mainframe Modernization pricing](https://aws.amazon.com/mainframe-modernization/pricing/) page. This billing is based on the amount of data (GB) transferred as measured by the data set size. ## View transfer tasks To view transfer tasks in File Transfer, you must follow these steps in the AWS Mainframe Modernization console. **To view transfer tasks** 1. Open the AWS Mainframe Modernization console at [https://console.aws.amazon.com/m2/](https://console.aws.amazon.com/m2/). 1. In the AWS Region selector, choose the Region where you want to transfer files from your mainframe to an Amazon S3 bucket. 1. On the **Transfer tasks** page, select the data transfer endpoint to view your transfer tasks. 1. For endpoints that have pre-existing transfer tasks, these will be displayed under the **Transfer tasks** section. You can choose to view details of any transfer task from this list. # Tutorial: Getting started with AWS Mainframe Modernization File Transfer Tutorial: Getting started with File Transfer AWS Mainframe Modernization File Transfer lets you transfer and convert mainframe data sets for mainframe modernization, migration, and augmentation use cases. Follow the steps in this tutorial to understand how AWS Mainframe Modernization File Transfer works. ## Overview File Transfer consists of the following: 1. An agent to be installed on the source mainframe. 1. Access to dataset discovery, transfer, and conversion capabilities directly from the AWS Mainframe Modernization management service console. As a user, you can transfer datasets from the mainframe to your Amazon S3 bucket. **Topics** + [ ## Overview ](#filetransfer-getting-started-overview) + [ ## Step 1: Transfer the agent binaries tar package from AWS to the mainframe logical partition ](#filetransfer-agent-binaries) + [ ## Step 2: Configure the File Transfer agent on the source mainframe ](#filetransfer-configure) + [ ## Step 3: Create a data transfer endpoint ](#filetransfer-data-transfer-endpoint) + [ ## Step 4: Create a transfer task ](#filetransfer-create-transfer-tasks) + [ ## Step 5: View transfer task progress ](#view-transfer-tasks) ## Step 1: Transfer the agent binaries tar package from AWS to the mainframe logical partition Download tar files from the [M2-agent tar](https://drm0z31ua8gi7.cloudfront.net/filetransfer/m2-agent-v1.0.0.tar) link. ## Step 2: Configure the File Transfer agent on the source mainframe In this step, you configure and start the AWS Mainframe Modernization File Transfer agent on the source mainframe. The agent is required to facilitate communications between the File Transfer service feature and the source mainframe. At least one agent is required per mainframe. More than one agent can be started for high availability and enhanced scalability. Follow the instructions in [Configure a File Transfer agent](m2-agent-configuration.md) guide to complete File Transfer agent installation on the mainframe. ## Step 3: Create a data transfer endpoint Follow steps on [Create data transfer endpoints for File Transfer](filetransfer-data-transfer-endpoints.md) page to create a new data transfer endpoint. ## Step 4: Create a transfer task Follow steps on [Create transfer tasks in File Transfer](filetransfer-transfer-tasks.md) page to create and manage your transfer tasks. ## Step 5: View transfer task progress You can view your transfer task's progress in the AWS Mainframe Modernization console. For more details, refer [View transfer tasks](filetransfer-transfer-tasks.md#filetransfer-console-view-task) section. # Supported source and target encodings in AWS Mainframe Modernization File Transfer Supported source and target code pages AWS Mainframe Modernization File Transfer supports various data set types and code page conversion options. ## Mainframe data set types AWS Mainframe Modernization File Transfer supports the following mainframe data set types: + Non-VSAM: Sequential (PS), PDS, GDS, GDG + VSAM types: KSDS ## Supported code pages AWS Mainframe Modernization File Transfer supports the following code pages for data set conversion (from/to): "BIG5" , "BIG5\$1HKSCS" , "CESU\$18" , "EUC\$1JP" , "EUC\$1KR" , "GB18030" , "GB2312" , "GBK" , "IBM00858" , "IBM01140" , "IBM01141" , "IBM01142", "IBM01143" , "IBM01144" , "IBM01145" , "IBM01146" , "IBM01147" , "IBM01148" , "IBM01149" , "IBM037" , "IBM1026" , "IBM1047" , "IBM273" , "IBM277" , "IBM278" , "IBM280" , "IBM284" , "IBM285" , "IBM290" , "IBM297" , "IBM420" , "IBM424" , "IBM437" , "IBM500" , "IBM775" , "IBM850" , "IBM852" , "IBM855" , "IBM857" , "IBM860" , "IBM861" , "IBM862" , "IBM863" , "IBM864" , "IBM865" , "IBM866" , "IBM868" , "IBM869" , "IBM870" , "IBM871" , "IBM918" , "IBM\$1THAI" , "ISO\$12022\$1CN" , "ISO\$12022\$1JP" , "ISO\$12022\$1JP\$12" , "ISO\$12022\$1KR" , "ISO\$18859\$11" , "ISO\$18859\$113" , "ISO\$18859\$115" , "ISO\$18859\$116" , "ISO\$18859\$12" , "ISO\$18859\$13" , "ISO\$18859\$14" , "ISO\$18859\$15" , "ISO\$18859\$16" , "ISO\$18859\$17" , "ISO\$18859\$18" , "ISO\$18859\$19" ,"JIS\$1X0201" , "JIS\$1X0212\$11990" , "KOI8\$1R" , "KOI8\$1U" , "SHIFT\$1JIS" , "TIS\$1620" , "US\$1ASCII" , "UTF\$116" , "UTF\$116BE" , "UTF\$116LE" , "UTF\$132" , "UTF\$132BE" , "UTF\$132LE" , "UTF\$18" , "WINDOWS\$11250" , "WINDOWS\$11251" , "WINDOWS\$11252" , "WINDOWS\$11253" ,"WINDOWS\$11254" , "WINDOWS\$11255" , "WINDOWS\$11256" , "WINDOWS\$11257" , "WINDOWS\$11258" , "WINDOWS\$131J" , "X\$1BIG5\$1HKSCS\$12001" , "X\$1BIG5\$1SOLARIS" , "X\$1EUCJP\$1OPEN" , "X\$1EUC\$1JP\$1LINUX" , "X\$1EUC\$1TW" , "X\$1IBM1006" , "X\$1IBM1025" , "X\$1IBM1046" , "X\$1IBM1097" , "X\$1IBM1098" , "X\$1IBM1112" , "X\$1IBM1122" , "X\$1IBM1123" , "X\$1IBM1124" , "X\$1IBM1129" , "X\$1IBM1166" , "X\$1IBM1364" , "X\$1IBM1381" , "X\$1IBM1383" , "X\$1IBM29626C" , "X\$1IBM300" , "X\$1IBM33722" , "X\$1IBM737" , "X\$1IBM833" , "X\$1IBM834" , "X\$1IBM856" , "X\$1IBM874" , "X\$1IBM875" , "X\$1IBM921" , "X\$1IBM922" , "X\$1IBM930" , "X\$1IBM933" , "X\$1IBM935" , "X\$1IBM937" , "X\$1IBM939" , "X\$1IBM942" , "X\$1IBM942C" , "X\$1IBM943" , "X\$1IBM943C" , "X\$1IBM948" , "X\$1IBM949" , "X\$1IBM949C" , "X\$1IBM950" , "X\$1IBM964" , "X\$1IBM970" , "X\$1ISCII91" , "X\$1ISO\$12022\$1CN\$1CNS" , "X\$1ISO\$12022\$1CN\$1GB" , "X\$1ISO\$18859\$111" , "X\$1JIS0208" , "X\$1JISAUTODETECT" , "X\$1JOHAB", "X\$1MACARABIC" , "X\$1MACCENTRALEUROPE" , "X\$1MACCROATIAN" , "X\$1MACCYRILLIC" , "X\$1MACDINGBAT", "X\$1MACGREEK" , "X\$1MACHEBREW" , "X\$1MACICELAND" , "X\$1MACROMAN" , "X\$1MACROMANIA" , "X\$1MACSYMBOL" , "X\$1MACTHAI" , "X\$1MACTURKISH" , "X\$1MACUKRAINE" , "X\$1MS932\$10213" , "X\$1MS950\$1HKSCS" , "X\$1MS950\$1HKSCS\$1XP" , "X\$1MSWIN\$1936" , "X\$1PCK" , "X\$1SJIS\$10213" , "X\$1UTF\$116LE\$1BOM" , "X\$1UTF\$132BE\$1BOM" , "X\$1UTF\$132LE\$1BOM" , "X\$1WINDOWS\$150220" , "X\$1WINDOWS\$150221" , "X\$1WINDOWS\$1874" , "X\$1WINDOWS\$1949" , "X\$1WINDOWS\$1950" , "X\$1WINDOWS\$1ISO2022j"