We are no longer updating the Amazon Machine Learning service or accepting new users for it. This documentation is available for existing users, but we are no longer updating it. For more information, see [ What is Amazon Machine Learning](https://docs.aws.amazon.com/machine-learning/latest/dg/what-is-amazon-machine-learning.html).
# Tutorial: Using Amazon ML to Predict Responses to a Marketing Offer
With Amazon Machine Learning (Amazon ML), you can build and train predictive models and host your applications in a scalable cloud solution. In this tutorial, we show you how to use the Amazon ML console to create a datasource, build a machine learning (ML) model, and use the model to generate predictions that you can use in your applications.
Our sample exercise shows how to identify potential customers for a targeted marketing campaign, but you can apply the same principles to create and use a variety of ML models. To complete the sample exercise, you will use publicly available banking and marketing datasets from the [University of California at Irvine (UCI) Machine Learning Repository](http://archive.ics.uci.edu/ml/datasets.php). These datasets contain general information about customers, and information about how they responded to previous marketing contacts. You will use this data to identify which customers are most likely to subscribe to your new product, a bank term deposit, also known as a certificate of deposit (CD).
**Warning**
This tutorial is not included in the AWS free tier. For more information about Amazon ML pricing, see [https://aws.amazon.com/machine-learning/pricing/](https://aws.amazon.com/machine-learning/pricing/).
## Prerequisite
To perform the tutorial, you need to have an AWS account. If you don't have an AWS account, see [Setting Up Amazon Machine Learning](https://docs.aws.amazon.com/machine-learning/latest/dg/setting-up-amazon-machine-learning.html).
## Steps
+ [Step 1: Prepare Your Data](step-1-download-edit-and-upload-data.md)
+ [Step 2: Create a Training Datasource](step-2-create-a-datasource.md)
+ [Step 3: Create an ML Model](step-3-create-an-ml-model.md)
+ [Step 4: Review the ML Model's Predictive Performance and Set a Score Threshold](step-4-review-model-and-set-cutoff.md)
+ [Step 5: Use the ML Model to Generate Predictions](step-5-create-predictions.md)
+ [Step 6: Clean Up](step-6-clean-up.md)
# Step 1: Prepare Your Data
In machine learning, you typically obtain the data and ensure that it is well formatted before starting the training process. For the purposes of this tutorial, we obtained a sample dataset from the [UCI Machine Learning Repository](http://archive.ics.uci.edu/ml/), formatted it to conform to Amazon ML guidelines, and made it available for you to download. Download the dataset from our Amazon Simple Storage Service (Amazon S3) storage location and upload it to your own S3 bucket by following the procedures in this topic.
For Amazon ML formatting requirements, see [Understanding the Data Format for Amazon ML](understanding-the-data-format-for-amazon-ml.md).
**To download the datasets**
1. Download the file that contains the historical data for customers who have purchased products similar to your bank term deposit by clicking [banking.zip](samples/banking.zip). Unzip the folder and save the banking.csv file to your computer.
1. Download the file that you will use to predict whether potential customers will respond to your offer by clicking [banking-batch.zip](samples/banking-batch.zip). Unzip the folder and save the banking-batch.csv file to your computer.
1. Open `banking.csv`. You will see rows and columns of data. The *header row* contains the attribute names for each column. An *attribute* is a unique, named property that describes a particular characteristic of each customer; for example, nr\$1employed indicates the customer's employment status. Each row represents the collection of observations about a single customer.
![\[Spreadsheet preview showing header row with columns for euribor3m, nr_employed, and y.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image1.png)
You want your ML model to answer the question "Will this customer subscribe to my new product?". In the `banking.csv` dataset, the answer to this question is attribute **y**, which contains the values 1 (for yes) or 0 (for no). The attribute that you want Amazon ML to learn how to predict is known as the *target attribute*.
**Note**
Attribute **y** is a binary attribute. It can contain only one of two values, in this case 0 or 1. In the original UCI dataset, the **y** attribute is either Yes or No. We have edited the original dataset for you. All values of attribute **y** that mean yes are now 1, and all values that mean no are now 0. If you use your own data, you can use other values for a binary attribute. For more information about valid values, see [Using the AttributeType Field](creating-a-data-schema-for-amazon-ml.md#assigning-data-types).
The following examples show the data before and after we changed the values in attribute **y** to the binary attributes 0 and 1.
![\[Data table showing 'banking.csv' with columns for 'euribor3m', 'nr_employed', and binary 'y' values.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image2.png)
![\[Partial view of a CSV file showing banking data with columns for euribor3m, nr_employed, and y.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image3.png)
The `banking-batch.csv` file doesn’t contain the **y** attribute. After you have created an ML model, you will use the model to predict **y** for each record in that file.
Next, upload the `banking.csv `and `banking-batch.csv` files to Amazon S3.
**To upload the files to an Amazon S3 location**
1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/).
1. In the **All Buckets** list, create a bucket or choose the location where you want to upload the files.
1. In the navigation bar, choose **Upload**.
1. Choose **Add Files**.
1. In the dialog box, navigate to your desktop, choose `banking.csv` and `banking-batch.csv`, and then choose **Open**.
Now you are ready to [create your training datasource](step-2-create-a-datasource.md).
# Step 2: Create a Training Datasource
After you upload the `banking.csv` dataset to your Amazon Simple Storage Service (Amazon S3) location, you use it to create a training datasource. A datasource is an Amazon Machine Learning (Amazon ML) object that contains the location of your input data and important metadata about your input data. Amazon ML uses the datasource for operations like ML model training and evaluation.
To create a datasource, provide the following:
+ The Amazon S3 location of your data and permission to access the data
+ The schema, which includes the names of the attributes in the data and the type of each attribute (Numeric, Text, Categorical, or Binary)
+ The name of the attribute that contains the answer that you want Amazon ML to learn to predict, the target attribute
**Note**
The datasource doesn't actually store your data, it only references it. Avoid moving or changing the files stored in Amazon S3. If you do move or change them, Amazon ML can't access them to create an ML model, generate evaluations, or generate predictions.
**To create the training datasource**
1. Open the Amazon Machine Learning console at [https://console.aws.amazon.com/machinelearning/](https://console.aws.amazon.com/machinelearning/).
1. Choose **Get started**.
**Note**
This tutorial assumes that this is your first time using Amazon ML. If you have used Amazon ML before, you can use the **Create new...** drop down list on the Amazon ML dashboard to create a new datasource.
1. On the **Get started with Amazon Machine Learning** page, choose **Launch**.
![\[Amazon Machine Learning interface with "Launch" button highlighted for standard setup.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/get-started-launch.png)
1. On the **Input Data** page, for **Where is your data located**?, make sure that **S3** is selected.
![\[Radio button selection between S3 and Redshift options, with S3 selected.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image5.png)
1. For **S3 Location**, type the full location of the `banking.csv `file from Step 1: Prepare Your Data. For example: *your-bucket***/banking.csv**. Amazon ML prepends s3:// to your bucket name for you.
1. For **Datasource name**, type **Banking Data 1**.
![\[S3 location input field and Datasource name field for entering banking data information.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image6.png)
1. Choose **Verify**.
1. In the **S3 permissions** dialog box, choose **Yes**.
![\[Dialog box asking to grant Amazon Machine Learning read permission for S3 location.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image7.png)
1. If Amazon ML can access and read the data file at the S3 location, you will see a page similar to the following. Review the properties, and then choose **Continue**.
![\[Validation success message with datasource details including name, location, and file information.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image8.png)
Next, you establish a schema. A *schema* is the information Amazon ML needs to interpret the input data for an ML model, including attribute names and their assigned data types, and the names of special attributes. There are two ways to provide Amazon ML with a schema:
+ Provide a separate schema file when you upload your Amazon S3 data.
+ Allow Amazon ML to infer the attribute types and create a schema for you.
In this tutorial, we'll ask Amazon ML to infer the schema.
For information about creating a separate schema file, see [Creating a Data Schema for Amazon ML](creating-a-data-schema-for-amazon-ml.md).
**To allow Amazon ML to infer the schema**
1. On the **Schema** page, Amazon ML shows you the schema that it inferred. Review the data types that Amazon ML inferred for the attributes. It is important that attributes are assigned the correct data type to help Amazon ML ingest the data correctly and to enable the correct feature processing on the attributes.
+ Attributes that have only two possible states, such as yes or no, should be marked as **Binary**.
+ Attributes that are numbers or strings that are used to denote a category should be marked as **Categorical**.
+ Attributes that are numeric quantities for which the order is meaningful should be marked as **Numeric**.
+ Attributes that are strings that you would like to treat as words delimited by spaces should be marked as **Text**.
![\[Data table showing fields like age, campaign, and contact with their data types and sample values.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image10.png)
1. In this tutorial, Amazon ML has correctly identified the data types for all of the attributes, so choose **Continue**.
Next, select a target attribute.
Remember that the target is the attribute that the ML model must learn to predict. Attribute **y** indicates whether an individual has subscribed to a campaign in the past: 1 (yes) or 0 (no).
**Note**
Choose a target attribute only if you will use the datasource for training and evaluating ML models.
**To select y as the target attribute**
1. In the lower right of the table, choose the single arrow to advance to the last page of the table, where the attribute named `y` appears.
![\[Navigation buttons for a paginated table, with the last page arrow highlighted.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image11.png)
1. In the **Target** column, select `y`.
![\[Checkbox in Target column next to variable 'y' with Binary data type.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image12.png)
Amazon ML confirms that **y** is selected as your target.
1. Choose **Continue**.
1. On the **Row ID** page, for **Does your data contain an identifier? **, make sure that **No**, the default, is selected.
1. Choose **Review**, and then choose **Continue**.
Now that you have a training datasource, you're ready to [create your model](step-3-create-an-ml-model.md).
# Step 3: Create an ML Model
After you've created the training datasource, you use it to create an ML model, train the model, and then evaluate the results. The ML model is a collection of patterns that Amazon ML finds in your data during training. You use the model to create predictions.
**To create an ML model**
1. Because the Get started wizard creates both a training datasource and a model, Amazon Machine Learning (Amazon ML) automatically uses the training datasource that you just created, and takes you directly to the **ML model settings** page. On the **ML model settings** page, for **ML model name**, make sure that the default, **ML model: Banking Data 1**, is displayed.
Using a friendly name, such as the default, helps you easily identify and manage the ML model.
1. For **Training and evaluation settings**, ensure that **Default** is selected.
![\[Select training and evaluation settings interface with Default option selected.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image19.png)
1. For **Name this evaluation**, accept the default, **Evaluation: ML model: Banking Data 1**.
1. Choose **Review**, review your settings, and then choose **Finish**.
After you choose **Finish**, Amazon ML adds your model to the processing queue. When Amazon ML creates your model, it applies the defaults and performs the following actions:
+ Splits the training datasource into two sections, one containing 70% of the data and one containing the remaining 30%
+ Trains the ML model on the section that contains 70% of the input data
+ Evaluates the model using the remaining 30% of the input data
While your model is in the queue, Amazon ML reports the status as **Pending**. While Amazon ML creates your model, it reports the status as **In Progress**. When it has completed all actions, it reports the status as **Completed**. Wait for the evaluation to complete before proceeding.
Now you are ready to [review your model's performance and set a cut-off score](step-4-review-model-and-set-cutoff.md).
For more information about training and evaluating models, see [Training ML Models](training-ml-models.md) and [Evaluating ML Models](evaluating_models.md).
# Step 4: Review the ML Model's Predictive Performance and Set a Score Threshold
Now that you've created your ML model and Amazon Machine Learning (Amazon ML) has evaluated it, let's see if it is good enough to put to use. During evaluation, Amazon ML computed an industry-standard quality metric, called the Area Under a Curve (AUC) metric, that expresses the performance quality of your ML model. Amazon ML also interprets the AUC metric to tell you if the quality of the ML model is adequate for most machine learning applications. (Learn more about AUC in [Measuring ML Model Accuracy](binary-model-insights.md#measuring-ml-model-accuracy).) Let's review the AUC metric, and then adjust the score threshold or cut-off to optimize your model's predictive performance.
**To review the AUC metric for your ML model**
1. On the **ML model summary** page, in the **ML model report** navigation pane, choose **Evaluations**, choose **Evaluation: ML model: Banking model 1**, and then choose **Summary**.
1. On the **Evaluation summary** page, review the evaluation summary, including the model's AUC performance metric.
![\[ML model performance metric showing extremely good quality score with AUC of 0.94.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image24.png)
The ML model generates numeric prediction scores for each record in a prediction datasource, and then applies a threshold to convert these scores into binary labels of 0 (for no) or 1 (for yes). By changing the *score threshold*, you can adjust how the ML model assigns these labels. Now, set the score threshold.
**To set a score threshold for your ML model**
1. On the **Evaluation Summary** page, choose **Adjust Score Threshold.**
![\[ML model performance chart showing predicted distributions for "1" and "0" records with adjustable score threshold.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image25.png)
You can fine-tune your ML model performance metrics by adjusting the score threshold. Adjusting this value changes the level of confidence that the model must have in a prediction before it considers the prediction to be positive. It also changes how many false negatives and false positives you are willing to tolerate in your predictions.
You can control the cutoff for what the model considers a positive prediction by increasing the score threshold until it considers only the predictions with the highest likelihood of being true positives as positive. You can also reduce the score threshold until you no longer have any false negatives. Choose your cutoff to reflect your business needs. For this tutorial, each false positive costs campaign money, so we want a high ratio of true positives to false positives.
1. Let's say you want to target the top 3% of the customers that will subscribe to the product. Slide the vertical selector to set the score threshold to a value that corresponds to **3% of the records are predicted as "1"**.
![\[ML model performance chart showing distribution of predicted answers for "1" and "0" records.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image26.png)
Note the impact of this score threshold on the ML model's performance: the false positive rate is 0.007. Let's assume that that false positive rate is acceptable.
1. Choose **Save score threshold at 0.77**.
Every time you use this ML model to make predictions, it will predict records with scores over 0.77 as "1", and the rest of the records as "0".
To learn more about the score threshold, see [Binary Classification](binary-classification.md).
Now you are ready to [create predictions using your model](step-5-create-predictions.md).
# Step 5: Use the ML Model to Generate Predictions
Amazon Machine Learning (Amazon ML) can generate two types of predictions—batch and real-time.
A *real-time prediction* is a prediction for a single observation that Amazon ML generates on demand. Real-time predictions are ideal for mobile apps, websites, and other applications that need to use results interactively.
A *batch prediction* is a set of predictions for a group of observations. Amazon ML processes the records in a batch prediction together, so processing can take some time. Use batch predictions for applications that require predictions for set of observations or predictions that don't use results interactively.
For this tutorial, you will generate a real-time prediction that predicts whether one potential customer will subscribe to the new product. You will also generate predictions for a large batch of potential customers. For the batch prediction, you will use the `banking-batch.csv` file that you uploaded in [Step 1: Prepare Your Data](step-1-download-edit-and-upload-data.md).
Let's start with a real-time prediction.
**Note**
For applications that require real-time predictions, you must create a real-time endpoint for the ML model. You accrue charges while a real-time endpoint is available. Before you commit to using real-time predictions and begin incurring the cost associated with them, you can try using the real-time prediction feature in your web browser, without creating a real-time endpoint. That's what we'll do for this tutorial.
**To try a real-time prediction**
1. In the **ML model report** navigation pane, choose **Try real-time predictions**.
![\[Navigation pane with "Try real-time predictions" option circled in red.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/try-real-time-prediction-menu.png)
1. Choose **Paste a record**.
![\[Button labeled "Paste a record" highlighted in a web interface form.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/paste-a-record.png)
1. In the **Paste a record** dialog box, paste the following observation:
```
32,services,divorced,basic.9y,no,unknown,yes,cellular,dec,mon,110,1,11,0,nonexistent,-1.8,94.465,-36.1,0.883,5228.1
```
1. In the **Paste a record** dialog box, choose **Submit** to confirm that you want to generate a prediction for this observation. Amazon ML populates the values in the real-time prediction form.
![\[Table row showing 'age' attribute with Numeric type and Value of 32.0.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/value-column.png)
**Note**
You can also populate the **Value** fields by typing in individual values. Regardless of the method you choose, you should provide an observation that wasn't used to train the model.
1. At the bottom of the page, choose **Create prediction**.
The prediction appears in the **Prediction results** pane on the right. This prediction has a **Predicted label** of `0`, which means that this potential customer is unlikely to respond to the campaign. A **Predicted label** of `1` would mean that the customer is likely to respond to the campaign.
![\[Prediction results showing binary ML model with predicted label 0 and score 0.033486433.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/real-time-result.png)
Now, create a batch prediction. You will provide Amazon ML with the name of the ML model you are using; the Amazon Simple Storage Service (Amazon S3) location of the input data for which you want to generate predictions (Amazon ML will create a batch prediction datasource from this data); and the Amazon S3 location for storing the results.
**To create a batch prediction**
1. Choose **Amazon Machine Learning**, and then choose **Batch Predictions**.
![\[Amazon Machine Learning dropdown menu with Batch Predictions option highlighted.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image29.png)
1. Choose **Create new batch prediction**.
1. On the **ML model for batch predictions** page, choose **ML model: Banking Data 1**.
Amazon ML displays the ML model name, ID, creation time, and the associated datasource ID.
1. Choose **Continue**.
1. To generate predictions, you need to provide Amazon ML the data that you need predictions for. This is called the *input data*. First, put the input data into a datasource so that Amazon ML can access it.
For **Locate the input data**, choose **My data is in S3, and I need to create a datasource**.
![\[Radio button options for locating input data, with the second option selected.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image31.png)
1. For **Datasource name**, type **Banking Data 2**.
1. For **S3 Location**, type the full location of the `banking-batch.csv` file: *your-bucket***/banking-batch.csv**.
1. For **Does the first line in your CSV contain the column names?**, choose **Yes**.
1. Choose **Verify**.
Amazon ML validates the location of your data.
1. Choose **Continue**.
1. For **S3 destination**, type the name of the Amazon S3 location where you uploaded the files in Step 1: Prepare Your Data. Amazon ML uploads the prediction results there.
1. For **Batch prediction name**, accept the default, **Batch prediction: ML model: Banking Data 1**. Amazon ML chooses the default name based on the model it will use to create predictions. In this tutorial, the model and the predictions are named after the training datasource, `Banking Data 1`.
1. Choose **Review**.
1. In the **S3 permissions** dialog box, choose **Yes**.
![\[Dialog box asking to grant Amazon Machine Learning write permission on S3 location.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image32.png)
1. On the **Review** page, choose **Finish**.
The batch prediction request is sent to Amazon ML and entered into a queue. The time it takes Amazon ML to process a batch prediction depends on the size of your datasource and the complexity of your ML model. While Amazon ML processes the request, it reports a status of **In Progress**. After the batch prediction has completed, the request's status changes to **Completed**. Now, you can view the results.
**To view the predictions**
1. Choose **Amazon Machine Learning**, and then choose **Batch Predictions**.
![\[Amazon Machine Learning dropdown menu showing options including Batch Predictions.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image33.png)
1. In the list of predictions, choose **Batch prediction: ML model: Banking Data 1**. The **Batch prediction info** page appears.
![\[Batch prediction details including name, ID, status, and associated data sources and models.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image34.png)
1. To view the results of the batch prediction, go to the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/) and navigate to the Amazon S3 location referenced in the **Output S3 URL** field. From there, navigate to the results folder, which will have a name similar to `s3://aml-data/batch-prediction/result`.
![\[AWS S3 console showing a single file in the batch-prediction result folder.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image35.png)
The prediction is stored in a compressed .gzip file with the .gz extension.
1. Download the prediction file to your desktop, uncompress it, and open it.
![\[Table showing bestAnswer scores with numerical values ranging from 0.00046 to 0.30811.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image36.png)
The file has two columns, **bestAnswer** and **score**, and a row for each observation in your datasource. The results in the **bestAnswer** column are based on the score threshold of 0.77 that you set in [Step 4: Review the ML Model's Predictive Performance and Set a Score Threshold](step-4-review-model-and-set-cutoff.md). A **score** greater than 0.77 results in a **bestAnswer** of 1, which is a positive response or prediction, and a **score** less than 0.77 results in a **bestAnswer** of 0, which is a negative response or prediction.
The following examples show positive and negative predictions based on the score threshold of 0.77.
Positive prediction:
![\[Table showing bestAnswer score of 1 with a corresponding value of 0.822876.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image37.png)
In this example, the value for **bestAnswer** is 1, and the value of **score** is 0.8228876. The value for **bestAnswer** is 1 because the **score** is greater than the score threshold of 0.77. A **bestAnswer** of 1 indicates that the customer is likely to purchase your product, and is, therefore, considered a positive prediction.
Negative prediction:
![\[Table showing bestAnswer score of 0 and a numerical score of 0.7693356.\]](http://docs.aws.amazon.com/machine-learning/latest/dg/images/image38.png)
In this example, the value of **bestAnswer** is 0 because the **score** value is 0.7695356, which is less than the score threshold of 0.77. The **bestAnswer** of 0 indicates that the customer is unlikely to purchase your product, and is, therefore, considered a negative prediction.
Each row of the batch result corresponds to a row in your batch input (an observation in your datasource).
After analyzing the predictions, you can execute your targeted marketing campaign; for example, by sending fliers to everyone with a predicted score of `1`.
Now that you have created, reviewed, and used your model, [clean up the data and AWS resources you created](step-6-clean-up.md) to avoid incurring unnecessary charges and to keep your workspace uncluttered.
# Step 6: Clean Up
To avoid accruing additional Amazon Simple Storage Service (Amazon S3) charges, delete the data stored in Amazon S3. You are not charged for other unused Amazon ML resources, but we recommend that you delete them to keep your workspace clean.
**To delete the input data stored in Amazon S3**
1. Open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/).
1. Navigate to the Amazon S3 location where you stored the `banking.csv` and `banking-batch.csv` files.
1. Select the `banking.csv`, `banking-batch.csv`, and `.writePermissionCheck.tmp` files.
1. Choose **Actions**, and then choose **Delete**.
1. When prompted for confirmation, choose **OK**.
Although you aren't charged for keeping the record of the batch prediction that Amazon ML ran or the datasources, model, and evaluation that you created during the tutorial, we recommend that you delete them to prevent cluttering your workspace.
**To delete the batch predictions**
1. Navigate to the Amazon S3 location where you stored the output of the batch prediction.
1. Choose the `batch-prediction` folder.
1. Choose **Actions**, and then choose **Delete**.
1. When prompted for confirmation, choose **OK**.
**To delete the Amazon ML resources**
1. On the Amazon ML dashboard, select the following resources.
+ The `Banking Data 1` datasource
+ The `Banking Data 1_[percentBegin=0, percentEnd=70, strategy=sequential]` datasource
+ The `Banking Data 1_[percentBegin=70, percentEnd=100, strategy=sequential]` datasource
+ The `Banking Data 2` datasource
+ The `ML model: Banking Data 1` ML model
+ The `Evaluation: ML model: Banking Data 1` evaluation
1. Choose **Actions**, and then choose **Delete**.
1. In the dialog box, choose **Delete** to delete all selected resources.
You have now successfully completed the tutorial. To continue using the console to create datasources, models, and predictions see the [Amazon Machine Learning Developer Guide](https://docs.aws.amazon.com/machine-learning/latest/dg/). To learn how to use the API, see the [Amazon Machine Learning API Reference](https://docs.aws.amazon.com/machine-learning/latest/APIReference/).