# Preparing your product in SageMaker AI
Before you can publish your product in AWS Marketplace, you must prepare it in Amazon SageMaker AI. There are two types of SageMaker AI products listed in AWS Marketplace: model packages and algorithms. For more information, see [Machine learning products in AWS Marketplace](machine-learning-products.md). This topic provides an overview of the three steps that are required to prepare your product:
1. [Packaging your code into images for machine learning products in AWS Marketplace](ml-packaging-your-code-into-images.md) – To prepare a model package or algorithm product, you must create the Docker container images for your product.
1. [Uploading your images to Amazon Elastic Container Registry](ml-uploading-your-images.md) – After packaging your code in container images and testing them locally, upload the images and scan them for known vulnerabilities. Fix any vulnerabilities before continuing.
1. [Creating your Amazon SageMaker AI resource](ml-creating-your-amazon-sagemaker-resource.md) – After your images are scanned successfully, you can use them to create a model package or algorithm resource in SageMaker AI.
# Packaging your code into images for machine learning products in AWS Marketplace
Machine learning products in AWS Marketplace use Amazon SageMaker AI to create and run the machine learning logic you provide for buyers. SageMaker AI runs Docker container images that contain your logic. SageMaker AI runs these containers in a secure and scalable infrastructure. For more information, see [Security and intellectual property with Amazon SageMaker AI](ml-security-and-intellectual-property.md). The following sections provide information about how to package your code into Docker container images for SageMaker AI.
**Topics**
+ [Which type of container image do I create?](#ml-which-type-of-container-image-do-i-create)
+ [Creating model package images](ml-model-package-images.md)
+ [Creating algorithm images](ml-algorithm-images.md)
## Which type of container image do I create?
The two types of container images are an inference image and a training image.
To create a model package product, you need only an inference image. For detailed instructions, see [Creating model package images](ml-model-package-images.md).
To create an algorithm product, you need both training and inference images. For detailed instructions, see [Creating algorithm images](ml-algorithm-images.md).
To package code properly into a container image, the container must adhere to the SageMaker AI file structure. The container must expose the correct endpoints to ensure that the service can pass data to and from your container. The following sections explain the details of this process.
**Important**
For security purposes, when a buyer subscribes to your containerized product, the Docker containers run in an isolated environment without an internet connection. When you create your containers, don't rely on outgoing calls over the internet because they will fail. Calls to AWS services will also fail. For more information, see the [Security and intellectual property with Amazon SageMaker AI](ml-security-and-intellectual-property.md) section.
Optionally, when creating your inference and training images, use a container from [Available Deep Learning Containers Images](https://aws.amazon.com/releasenotes/available-deep-learning-containers-images/) as a starting point. The images are already properly packaged with different machine learning frameworks.
# Creating model package images
An Amazon SageMaker AI model package is a pre-trained model that makes predictions and does not require any further training by the buyer. You can create a model package in SageMaker AI and publish your machine learning product on AWS Marketplace. The following sections you how to create a model package for AWS Marketplace. This includes creating the container image and building and testing the image locally.
**Topics**
+ [Overview](#ml-model-package-images-overview)
+ [Create an inference image for model packages](#ml-creating-an-inference-image-for-model-packages)
## Overview
A model package includes the following components:
+ An inference image stored in [Amazon Elastic Container Registry](https://aws.amazon.com/ecr/) (Amazon ECR)
+ (Optional) Model artifacts, stored separately in [Amazon S3](https://aws.amazon.com/s3/)
**Note**
Model artifacts are files your model uses to make predictions and are generally the result of your own training processes. Artifacts can be any file type that is needed by your model but must use.tar.gz compression. For model packages, they can either be bundled within your inference image or stored separately in Amazon SageMaker AI. The model artifacts stored in Amazon S3 are loaded into the inference container at runtime. When publishing your model package, those artifacts are published and stored in AWS Marketplace owned Amazon S3 buckets that are inaccessible by the buyer directly.
**Tip**
If your inference model is built with a deep learning framework such as Gluon, Keras, MXNet, PyTorch, TensorFlow, TensorFlow-Lite, or ONNX, consider using Amazon SageMaker AI Neo. Neo can automatically optimize inference models that deploy to a specific family of cloud instance types such as `ml.c4`, `ml.p2`, and others. For more information, see [Optimize model performance using Neo](https://docs.aws.amazon.com/sagemaker/latest/dg/neo.html) in the *Amazon SageMaker AI Developer Guide*.
The following diagram shows the workflow for publishing and using model package products.
![\[Diagram of how a seller creates a model package image and how a buyer uses it.\]](http://docs.aws.amazon.com/marketplace/latest/userguide/images/ml-model-package-images.png)
The workflow for creating a SageMaker AI model package for AWS Marketplace includes the following steps:
1. The seller creates an inference image (no network access when deployed) and pushes it to the Amazon ECR Registry.
The model artifacts can either be bundled in the inference image or stored separately in S3.
1. The seller then creates a model package resource in Amazon SageMaker AI and publishes their ML product on AWS Marketplace.
1. The buyer subscribes to the ML product and deploys the model.
**Note**
The model can be deployed as an endpoint for real-time inferences or as a batch job to get predictions for an entire dataset at once. For more information, see [Deploy Models for Inference](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model.html).
1. SageMaker AI runs the inference image. Any seller-provided model artifacts not bundled in the inference image are loaded dynamically at runtime.
1. SageMaker AI passes the buyer’s inference data to the container by using the container’s HTTP endpoints and returns the prediction results.
## Create an inference image for model packages
This section provides a walkthrough for packaging your inference code into an inference image for your model package product. The process consists of the following steps:
**Topics**
+ [Step 1: Create the container image](#ml-step-1-creating-the-container-image)
+ [Step 2: Build and testing the image locally](#ml-step-2-building-and-testing-the-image-locally)
The inference image is a Docker image containing your inference logic. The container at runtime exposes HTTP endpoints to allow SageMaker AI to pass data to and from your container.
**Note**
The following is only one example of packaging code for an inference image. For more information, see [Using Docker containers with SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms.html) and the [AWS Marketplace SageMaker AI examples](https://github.com/aws/amazon-sagemaker-examples/tree/master/aws_marketplace) on GitHub.
The following example uses a web service, [Flask](https://pypi.org/project/Flask/), for simplicity, and is not considered production-ready.
### Step 1: Create the container image
For the inference image to be compatible with SageMaker AI, the Docker image must expose HTTP endpoints. While your container is running, SageMaker AI passes buyer inputs for inference to the container’s HTTP endpoint. The inference results are returned in the body of the HTTP response.
The following walkthrough uses the Docker CLI in a development environment using a Linux Ubuntu distribution.
+ [Create the web server script](#ml-create-the-web-server-script)
+ [Create the script for the container run](#ml-create-the-script-for-the-container-run)
+ [Create the `Dockerfile`](#ml-create-the-dockerfile)
+ [Package or upload the model artifacts](#ml-package-or-upload-the-model-artifacts)
#### Create the web server script
This example uses a Python server called [Flask](https://pypi.org/project/Flask/), but you can use any web server that works for your framework.
**Note**
[Flask](https://pypi.org/project/Flask/) is used here for simplicity. It is not considered a production-ready web server.
Create a Flask web server script that serves the two HTTP endpoints on TCP port 8080 that SageMaker AI uses. The following are the two expected endpoints:
+ `/ping` – SageMaker AI makes HTTP GET requests to this endpoint to check if your container is ready. When your container is ready, it responds to HTTP GET requests at this endpoint with an HTTP 200 response code.
+ `/invocations` – SageMaker AI makes HTTP POST requests to this endpoint for inference. The input data for inference is sent in the body of the request. The user-specified content type is passed in the HTTP header. The body of the response is the inference output. For details about timeouts, see [Requirements and best practices for creating machine learning products](ml-listing-requirements-and-best-practices.md).
**`./web_app_serve.py`**
```
# Import modules
import json
import re
from flask import Flask
from flask import request
app = Flask(__name__)
# Create a path for health checks
@app.route("/ping")
def endpoint_ping():
return ""
# Create a path for inference
@app.route("/invocations", methods=["POST"])
def endpoint_invocations():
# Read the input
input_str = request.get_data().decode("utf8")
# Add your inference code between these comments.
#
#
#
#
#
# Add your inference code above this comment.
# Return a response with a prediction
response = {"prediction":"a","text":input_str}
return json.dumps(response)
```
In the previous example, there is no actual inference logic. For your actual inference image, add the inference logic into the web app so it processes the input and returns the actual prediction.
Your inference image must contain all of its required dependencies because it will not have internet access, nor will it be able to make calls to any AWS services.
**Note**
This same code is called for both real-time and batch inferences
#### Create the script for the container run
Create a script named `serve` that SageMaker AI runs when it runs the Docker container image. The following script starts the HTTP web server.
**`./serve`**
```
#!/bin/bash
# Run flask server on port 8080 for SageMaker
flask run --host 0.0.0.0 --port 8080
```
#### Create the `Dockerfile`
Create a `Dockerfile` in your build context. This example uses Ubuntu 18.04, but you can start from any base image that works for your framework.
`./Dockerfile`
```
FROM ubuntu:18.04
# Specify encoding
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
# Install python-pip
RUN apt-get update \
&& apt-get install -y python3.6 python3-pip \
&& ln -s /usr/bin/python3.6 /usr/bin/python \
&& ln -s /usr/bin/pip3 /usr/bin/pip;
# Install flask server
RUN pip install -U Flask;
# Add a web server script to the image
# Set an environment to tell flask the script to run
COPY /web_app_serve.py /web_app_serve.py
ENV FLASK_APP=/web_app_serve.py
# Add a script that Amazon SageMaker AI will run
# Set run permissions
# Prepend program directory to $PATH
COPY /serve /opt/program/serve
RUN chmod 755 /opt/program/serve
ENV PATH=/opt/program:${PATH}
```
The `Dockerfile` adds the two previously created scripts to the image. The directory of the `serve` script is added to the PATH so it can run when the container runs.
#### Package or upload the model artifacts
The two ways to provide the model artifacts from training the model to the inference image are as follows:
+ Packaged statically with the inference image.
+ Loaded dynamically at runtime. Because it's loaded dynamically, you can use the same image for packaging different machine learning models.
If you want to package your model artifacts with the inference image, include the artifacts in the `Dockerfile`.
If you want to load your model artifacts dynamically, store those artifacts separately in a compressed file (.tar.gz) in Amazon S3. When creating the model package, specify the location of the compressed file, and SageMaker AI extracts and copies the contents to the container directory `/opt/ml/model/` when running your container. When publishing your model package, those artifacts are published and stored in AWS Marketplace owned Amazon S3 buckets that are inaccessible by the buyer directly.
### Step 2: Build and testing the image locally
In the build context, the following files now exist:
+ `./Dockerfile`
+ `./web_app_serve.py`
+ `./serve`
+ Your inference logic and (optional) dependencies
Next build, run, and test the container image.
#### Build the image
Run the Docker command in the build context to build and tag the image. This example uses the tag `my-inference-image`.
```
sudo docker build --tag my-inference-image ./
```
After running this Docker command to build the image, you should see output as Docker builds the image based on each line in your `Dockerfile`. When it finishes, you should see something similar to the following.
```
Successfully built abcdef123456
Successfully tagged my-inference-image:latest
```
#### Run locally
After your build has completed, you can test the image locally.
```
sudo docker run \
--rm \
--publish 8080:8080/tcp \
--detach \
--name my-inference-container \
my-inference-image \
serve
```
The following are details about the command:
+ `--rm` – Automatically remove the container after it stops.
+ `--publish 8080:8080/tcp` – Expose port 8080 to simulate the port that SageMaker AI sends HTTP requests to.
+ `--detach` – Run the container in the background.
+ `--name my-inference-container` – Give this running container a name.
+ `my-inference-image` – Run the built image.
+ `serve` – Run the same script that SageMaker AI runs when running the container.
After running this command, Docker creates a container from the inference image you built and runs it in the background. The container runs the `serve` script, which launches your web server for testing purposes.
#### Test the ping HTTP endpoint
When SageMaker AI runs your container, it periodically pings the endpoint. When the endpoint returns an HTTP response with status code 200, it signals to SageMaker AI that the container is ready for inference. You can test this by running the following command, which tests the endpoint and includes the response header.
```
curl --include http://127.0.0.1:8080/ping
```
Example output is as follows.
```
HTTP/1.0 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 0
Server: MyServer/0.16.0 Python/3.6.8
Date: Mon, 21 Oct 2019 06:58:54 GMT
```
#### Test the inference HTTP endpoint
When the container indicates it is ready by returning a 200 status code to your ping, SageMaker AI passes the inference data to the `/invocations` HTTP endpoint via a `POST` request. Test the inference point by running the following command.
```
curl \
--request POST \
--data "hello world" \
http://127.0.0.1:8080/invocations
```
Example output is as follows.
`{"prediction": "a", "text": "hello world"}`
With these two HTTP endpoints working, the inference image is now compatible with SageMaker AI.
**Note**
The model of your model package product can be deployed in two ways: real time and batch. In both deployments, SageMaker AI uses the same HTTP endpoints while running the Docker container.
To stop the container, run the following command.
```
sudo docker container stop my-inference-container
```
When your inference image is ready and tested, you can continue to [Uploading your images to Amazon Elastic Container Registry](ml-uploading-your-images.md).
# Creating algorithm images
An Amazon SageMaker AI algorithm requires that the buyer bring their own data to train before it makes predictions. As an AWS Marketplace seller, you can use SageMaker AI to create machine learning (ML) algorithms and models that your buyers can deploy in AWS. The following sections you how to create algorithm images for AWS Marketplace. This includes creating the Docker training image for training your algorithm and the inference image that contains your inference logic. Both the training and inference images are required when publishing an algorithm product.
**Topics**
+ [Overview](#ml-algorithm-images-overview)
+ [Creating a training image for algorithms](#ml-creating-a-training-image-for-algorithms)
+ [Creating an inference image for algorithms](#ml-creating-an-inference-image-for-algorithms)
## Overview
An algorithm includes the following components:
+ A training image stored in [Amazon ECR](https://aws.amazon.com/ecr/)
+ An inference image stored in Amazon Elastic Container Registry (Amazon ECR)
**Note**
For algorithm products, the training container generates model artifacts that are loaded into the inference container on model deployment.
The following diagram shows the workflow for publishing and using algorithm products.
![\[Diagram of how a seller creates an algorithm package image and how a buyer uses it.\]](http://docs.aws.amazon.com/marketplace/latest/userguide/images/ml-algorithm-package-images.png)
The workflow for creating a SageMaker AI algorithm for AWS Marketplace includes the following steps:
1. The seller creates a training image and an inference image (no network access when deployed) and uploads it to the Amazon ECR Registry.
1. The seller then creates an algorithm resource in Amazon SageMaker AI and publishes their ML product on AWS Marketplace.
1. The buyer subscribes to the ML product.
1. The buyer creates a training job with a compatible dataset and appropriate hyperparameter values. SageMaker AI runs the training image and loads the training data and hyperparameters into the training container. When the training job completes, the model artifacts located in `/opt/ml/model/` are compressed and copied to the buyer’s [Amazon S3](https://aws.amazon.com/s3/) bucket.
1. The buyer creates a model package with the model artifacts from the training stored in Amazon S3 and deploys the model.
1. SageMaker AI runs the inference image, extracts the compressed model artifacts, and loads the files into the inference container directory path `/opt/ml/model/` where it is consumed by the code that serves the inference.
1. Whether the model deploys as an endpoint or a batch transform job, SageMaker AI passes the data for inference on behalf of the buyer to the container via the container’s HTTP endpoint and returns the prediction results.
**Note**
For more information, see [Train Models](https://docs.aws.amazon.com/sagemaker/latest/dg/train-model.html).
## Creating a training image for algorithms
This section provides a walkthrough for packaging your training code into a training image. A training image is required to create an algorithm product.
A *training image* is a Docker image containing your training algorithm. The container adheres to a specific file structure to allow SageMaker AI to copy data to and from your container.
Both the training and inference images are required when publishing an algorithm product. After creating your training image, you must create an inference image. The two images can be combined into one image or remain as separate images. Whether to combine the images or separate them is up to you. Typically, inference is simpler than training, and you might want separate images to help with inference performance.
**Note**
The following is only one example of packaging code for a training image. For more information, see [Use your own algorithms and models with the AWS Marketplace](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-marketplace.html) and the [AWS Marketplace SageMaker AI examples](https://github.com/aws/amazon-sagemaker-examples/tree/master/aws_marketplace) on GitHub.
**Topics**
+ [Step 1: Creating the container image](#ml-step-1-creating-the-container-image-1)
+ [Step 2: Building and testing the image locally](#ml-step-2-building-and-testing-the-image-locally-1)
### Step 1: Creating the container image
For the training image to be compatible with Amazon SageMaker AI, it must adhere to a specific file structure to allow SageMaker AI to copy the training data and configuration inputs to specific paths in your container. When the training completes, the generated model artifacts are stored in a specific directory path in the container where SageMaker AI copies from.
The following uses Docker CLI installed in a development environment on an Ubuntu distribution of Linux.
+ [Prepare your program to read configuration inputs](#ml-prepare-your-program-to-read-configuration-inputs)
+ [Prepare your program to read data inputs](#ml-prepare-your-program-to-read-data-inputs)
+ [Prepare your program to write training outputs](#ml-prepare-your-program-to-write-training-outputs)
+ [Create the script for the container run](#ml-create-the-script-for-the-container-run-1)
+ [Create the `Dockerfile`](#ml-create-the-dockerfile-1)
#### Prepare your program to read configuration inputs
If your training program requires any buyer-provided configuration input, the following is where those are copied to inside your container when ran. If required, your program must read from those specific file paths.
+ `/opt/ml/input/config` is the directory that contains information which controls how your program runs.
+ `hyperparameters.json` is a JSON-formatted dictionary of hyperparameter names and values. The values are strings, so you may need to convert them.
+ `resourceConfig.json` is a JSON-formatted file that describes the network layout used for [ distributed training](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo-running-container.html#your-algorithms-training-algo-running-container-dist-training). If your training image does not support distributed training, you can ignore this file.
**Note**
For more information about configuration inputs, see [ How Amazon SageMaker AI Provides Training Information](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo-running-container.html).
#### Prepare your program to read data inputs
Training data can be passed to the container in one of the following two modes. Your training program that runs in the container digests the training data in one of those two modes.
**File mode**
+ `/opt/ml/input/data//` contains the input data for that channel. The channels are created based on the call to the `CreateTrainingJob` operation, but it's generally important that channels match what the algorithm expects. The files for each channel are copied from [Amazon S3](https://aws.amazon.com/s3/) to this directory, preserving the tree structure indicated by the Amazon S3 key structure.
**Pipe mode**
+ `/opt/ml/input/data/_` is the pipe for a given epoch. Epochs start at zero and increase by one each time you read them. There is no limit to the number of epochs that you can run, but you must close each pipe before reading the next epoch.
#### Prepare your program to write training outputs
The output of the training is written to the following container directories:
+ `/opt/ml/model/` is the directory where you write the model or the model artifacts that your training algorithm generates. Your model can be in any format that you want. It can be a single file or a whole directory tree. SageMaker AI packages any files in this directory into a compressed file (.tar.gz). This file is available at the Amazon S3 location returned by the `DescribeTrainingJob` API operation.
+ `/opt/ml/output/` is a directory where the algorithm can write a `failure` file that describes why the job failed. The contents of this file are returned in the `FailureReason` field of the `DescribeTrainingJob` result. For jobs that succeed, there is no reason to write this file because it’s ignored.
#### Create the script for the container run
Create a `train` shell script that SageMaker AI runs when it runs the Docker container image. When the training completes and the model artifacts are written to their respective directories, exit the script.
**`./train`**
```
#!/bin/bash
# Run your training program here
#
#
#
#
```
#### Create the `Dockerfile`
Create a `Dockerfile` in your build context. This example uses Ubuntu 18.04 as the base image, but you can start from any base image that works for your framework.
**`./Dockerfile`**
```
FROM ubuntu:18.04
# Add training dependencies and programs
#
#
#
#
#
# Add a script that SageMaker AI will run
# Set run permissions
# Prepend program directory to $PATH
COPY /train /opt/program/train
RUN chmod 755 /opt/program/train
ENV PATH=/opt/program:${PATH}
```
The `Dockerfile` adds the previously created `train` script to the image. The script’s directory is added to the PATH so it can run when the container runs.
In the previous example, there is no actual training logic. For your actual training image, add the training dependencies to the `Dockerfile`, and add the logic to read the training inputs to train and generate the model artifacts.
Your training image must contain all of its required dependencies because it will not have internet access.
For more information, see [Use your own algorithms and models with the AWS Marketplace](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-marketplace.html) and the [AWS Marketplace SageMaker AI examples](https://github.com/aws/amazon-sagemaker-examples/tree/master/aws_marketplace) on GitHub.
### Step 2: Building and testing the image locally
In the build context, the following files now exist:
+ `./Dockerfile`
+ `./train`
+ Your training dependencies and logic
Next you can build, run, and test this container image.
#### Build the image
Run the Docker command in the build context to build and tag the image. This example uses the tag `my-training-image`.
```
sudo docker build --tag my-training-image ./
```
After running this Docker command to build the image, you should see output as Docker builds the image based on each line in your `Dockerfile`. When it finishes, you should see something similar to the following.
```
Successfully built abcdef123456
Successfully tagged my-training-image:latest
```
#### Run locally
After that has completed, test the image locally as shown in the following example.
```
sudo docker run \
--rm \
--volume ':/opt/ml/input:ro' \
--volume ':/opt/ml/model' \
--volume ':/opt/ml/output' \
--name my-training-container \
my-training-image \
train
```
The following are command details:
+ `--rm` – Automatically remove the container after it stops.
+ `--volume ':/opt/ml/input:ro'` – Make test input directory available to container as read-only.
+ `--volume ':/opt/ml/model'` – Bind mount the path where the model artifacts are stored on the host machine when the training test is complete.
+ `--volume ':/opt/ml/output'` – Bind mount the path where the failure reason in a `failure` file is written to on the host machine.
+ `--name my-training-container` – Give this running container a name.
+ `my-training-image` – Run the built image.
+ `train` – Run the same script SageMaker AI runs when running the container.
After running this command, Docker creates a container from the training image you built and runs it. The container runs the `train` script, which starts your training program.
After your training program finishes and the container exits, check that the output model artifacts are correct. Additionally, check the log outputs to confirm that they are not producing logs that you do not want, while ensuring enough information is provided about the training job.
This completes packaging your training code for an algorithm product. Because an algorithm product also includes an inference image, continue to the next section, [Creating an inference image for algorithms](#ml-creating-an-inference-image-for-algorithms).
## Creating an inference image for algorithms
This section provides a walkthrough for packaging your inference code into an inference image for your algorithm product.
The inference image is a Docker image containing your inference logic. The container at runtime exposes HTTP endpoints to allow SageMaker AI to pass data to and from your container.
Both the training and inference images are required when publishing an algorithm product. If you have not already done so, see the previous section about [Creating a training image for algorithms](#ml-creating-a-training-image-for-algorithms). The two images can be combined into one image or remain as separate images. Whether to combine the images or separate them is up to you. Typically, inference is simpler than training, and you might want separate images to help with inference performance.
**Note**
The following is only one example of packaging code for an inference image. For more information, see [ Use your own algorithms and models with the AWS Marketplace](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-marketplace.html) and the [AWS Marketplace SageMaker AI examples](https://github.com/aws/amazon-sagemaker-examples/tree/master/aws_marketplace) on GitHub.
The following example uses a web service, [Flask](https://pypi.org/project/Flask/), for simplicity, and is not considered production-ready.
**Topics**
+ [Step 1: Creating the inference image](#ml-step-1-creating-the-inference-image)
+ [Step 2: Building and testing the image locally](#ml-step-2-building-and-testing-the-image-locally-2)
### Step 1: Creating the inference image
For the inference image to be compatible with SageMaker AI, the Docker image must expose HTTP endpoints. While your container is running, SageMaker AI passes inputs for inference provided by the buyer to your container’s HTTP endpoint. The result of the inference is returned in the body of the HTTP response.
The following uses Docker CLI installed in a development environment on an Ubuntu distribution of Linux.
+ [Create the web server script](#ml-create-the-web-server-script-1)
+ [Create the script for the container run](#ml-create-the-script-for-the-container-run-2)
+ [Create the `Dockerfile`](#ml-create-the-dockerfile-2)
+ [Preparing your program to dynamically load model artifacts](#ml-preparing-your-program-to-dynamically-load-model-artifacts)
#### Create the web server script
This example uses a Python server called [Flask](https://pypi.org/project/Flask/), but you can use any web server that works for your framework.
**Note**
[Flask](https://pypi.org/project/Flask/) is used here for simplicity. It is not considered a production-ready web server.
Create the Flask web server script that serves the two HTTP endpoints on TCP port 8080 that SageMaker AI uses. The following are the two expected endpoints:
+ `/ping` – SageMaker AI makes HTTP GET requests to this endpoint to check if your container is ready. When your container is ready, it responds to HTTP GET requests at this endpoint with an HTTP 200 response code.
+ `/invocations` – SageMaker AI makes HTTP POST requests to this endpoint for inference. The input data for inference is sent in the body of the request. The user-specified content type is passed in the HTTP header. The body of the response is the inference output.
**`./web_app_serve.py`**
```
# Import modules
import json
import re
from flask import Flask
from flask import request
app = Flask(__name__)
# Create a path for health checks
@app.route("/ping")
def endpoint_ping():
return ""
# Create a path for inference
@app.route("/invocations", methods=["POST"])
def endpoint_invocations():
# Read the input
input_str = request.get_data().decode("utf8")
# Add your inference code here.
#
#
#
#
#
# Add your inference code here.
# Return a response with a prediction
response = {"prediction":"a","text":input_str}
return json.dumps(response)
```
In the previous example, there is no actual inference logic. For your actual inference image, add the inference logic into the web app so it processes the input and returns the prediction.
Your inference image must contain all of its required dependencies because it will not have internet access.
#### Create the script for the container run
Create a script named `serve` that SageMaker AI runs when it runs the Docker container image. In this script, start the HTTP web server.
**`./serve`**
```
#!/bin/bash
# Run flask server on port 8080 for SageMaker AI
flask run --host 0.0.0.0 --port 8080
```
#### Create the `Dockerfile`
Create a `Dockerfile` in your build context. This example uses Ubuntu 18.04, but you can start from any base image that works for your framework.
**`./Dockerfile`**
```
FROM ubuntu:18.04
# Specify encoding
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
# Install python-pip
RUN apt-get update \
&& apt-get install -y python3.6 python3-pip \
&& ln -s /usr/bin/python3.6 /usr/bin/python \
&& ln -s /usr/bin/pip3 /usr/bin/pip;
# Install flask server
RUN pip install -U Flask;
# Add a web server script to the image
# Set an environment to tell flask the script to run
COPY /web_app_serve.py /web_app_serve.py
ENV FLASK_APP=/web_app_serve.py
# Add a script that Amazon SageMaker AI will run
# Set run permissions
# Prepend program directory to $PATH
COPY /serve /opt/program/serve
RUN chmod 755 /opt/program/serve
ENV PATH=/opt/program:${PATH}
```
The `Dockerfile` adds the two created previously scripts to the image. The directory of the `serve` script is added to the PATH so it can run when the container runs.
#### Preparing your program to dynamically load model artifacts
For algorithm products, the buyer uses their own datasets with your training image to generate unique model artifacts. When the training process completes, your training container outputs model artifacts to the container directory` /opt/ml/model/`. SageMaker AI compresses the contents in that directory into a .tar.gz file and stores it in the buyer’s AWS account in Amazon S3.
When the model deploys, SageMaker AI runs your inference image, extracts the model artifacts from the .tar.gz file stored in the buyer’s account in Amazon S3,and loads them into the inference container in the `/opt/ml/model/` directory. At runtime, your inference container code uses the model data.
**Note**
To protect any intellectual property that might be contained in the model artifact files, you can choose to encrypt the files before outputting them. For more information, see [Security and intellectual property with Amazon SageMaker AI](ml-security-and-intellectual-property.md).
### Step 2: Building and testing the image locally
In the build context, the following files now exist:
+ `./Dockerfile`
+ `./web_app_serve.py`
+ `./serve`
Next you can build, run, and test this container image.
#### Build the image
Run the Docker command to build and tag the image. This example uses the tag `my-inference-image`.
```
sudo docker build --tag my-inference-image ./
```
After running this Docker command to build the image, you should see output as Docker builds the image based on each line in your `Dockerfile`. When it finishes, you should see something similar to the following.
```
Successfully built abcdef123456
Successfully tagged my-inference-image:latest
```
#### Run locally
After your build has completed, you can test the image locally.
```
sudo docker run \
--rm \
--publish 8080:8080/tcp \
--volume ':/opt/ml/model:ro' \
--detach \
--name my-inference-container \
my-inference-image \
serve
```
The following are command details:
+ `--rm` – Automatically remove the container after it stops.
+ `--publish 8080:8080/tcp` – Expose port 8080 to simulate the port SageMaker AI sends HTTP requests to.
+ `--volume ':/opt/ml/model:ro'` – Bind mount the path to where the test model artifacts are stored on the host machine as read-only to make them available to your inference code in the container.
+ `--detach` – Run the container in the background.
+ `--name my-inference-container` – Give this running container a name.
+ `my-inference-image` – Run the built image.
+ `serve` – Run the same script SageMaker AI runs when running the container.
After running this command, Docker creates a container from the inference image and runs it in the background. The container runs the `serve` script, which starts your web server for testing purposes.
#### Test the ping HTTP endpoint
When SageMaker AI runs your container, it periodically pings the endpoint. When the endpoint returns an HTTP response with status code 200, it signals to SageMaker AI that the container is ready for inference.
Run the following command to test the endpoint and include the response header.
```
curl --include http://127.0.0.1:8080/ping
```
Example output is shown in the following example.
```
HTTP/1.0 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 0
Server: MyServer/0.16.0 Python/3.6.8
Date: Mon, 21 Oct 2019 06:58:54 GMT
```
#### Test the inference HTTP endpoint
When the container indicates it is ready by returning a 200 status code, SageMaker AI passes the inference data to the `/invocations` HTTP endpoint via a `POST` request.
Run the following command to test the inference endpoint.
```
curl \
--request POST \
--data "hello world" \
http://127.0.0.1:8080/invocations
```
Example output is shown in the following example..
```
{"prediction": "a", "text": "hello world"}
```
With these two HTTP endpoints working, the inference image is now compatible with SageMaker AI.
**Note**
The model of your algorithm product can be deployed in two ways: real time and batch. For both deployments, SageMaker AI uses the same HTTP endpoints while running the Docker container.
To stop the container, run the following command.
```
sudo docker container stop my-inference-container
```
After both your training and inference images for your algorithm product are ready and tested, continue to [Uploading your images to Amazon Elastic Container Registry](ml-uploading-your-images.md).
# Uploading your images to Amazon Elastic Container Registry
After you create your inference and training images you can upload them to Amazon Elastic Container Registry. [Amazon ECR](https://aws.amazon.com/ecr/) is a fully managed Docker registry. Amazon SageMaker AI pulls images from Amazon ECR to create a model package for inference or an algorithm for training jobs. AWS Marketplace also retrieves these images from Amazon ECR to publish your model package and algorithm products. This topic provides a walkthrough for uploading your inference and training images to Amazon ECR
**Topics**
+ [Which images must I upload?](#ml-which-images-must-i-upload)
+ [What IAM permissions are required?](#ml-what-iam-permissions-are-required)
+ [Log your Docker client into AWS](#ml-log-in-your-docker-client)
+ [Create repository and upload image](#ml-create-repository-and-upload-image)
+ [Scan your uploaded image](#ml-scan-your-uploaded-image)
## Which images must I upload?
If you're publishing a model package, upload only an inference image. If you're publishing an algorithm, upload both an inference image and a training image. If the inference and training images are combined, upload the combined image only once.
## What IAM permissions are required?
The following steps assume that the local machine has the correct AWS credentials for an AWS Identity and Access Management (IAM) role or user in the seller AWS account. The role or user must have the correct policies in place for both AWS Marketplace and Amazon ECR. For example, you could use the following AWS managed policies:
+ [https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AWSMarketplaceSellerProductsFullAccess.html](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AWSMarketplaceSellerProductsFullAccess.html) – For access to AWS Marketplace
+ [https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonEC2ContainerRegistryFullAccess.html](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonEC2ContainerRegistryFullAccess.html) – For access to Amazon ECR
**Note**
The links take you to the *AWS Managed Policy Reference*.
## Log your Docker client into AWS
Set a variable for the AWS Region that you want to publish from (see [Supported AWS Regions for publishing](ml-service-restrictions-and-limits.md#ml-supported-aws-regions-for-publishing)). For this example, use the US East (Ohio) Region.
```
region=us-east-2
```
Run the following command to set a variable with your AWS account ID. This example assumes that the current AWS Command Line Interface (AWS CLI) credentials belong to the seller’s AWS account.
```
account=$(aws sts get-caller-identity --query Account --output text)
```
To authenticate your Docker CLI client with your AWS account Amazon ECR Docker registry for your Region, run the following command.
```
aws ecr get-login-password \
--region ${region} \
| sudo docker login \
--username AWS \
--password-stdin \
${account}.dkr.ecr.${region}.amazonaws.com
```
## Create repository and upload image
Set a variable for the tag of the uploaded image and another variable for the name of the uploaded image repository.
```
image=my-inference-image
repo=my-inference-image
```
**Note**
In previous sections of this guide where the inference and training images were built, they were tagged as **my-inference-image** and **my-training-image**, respectively. For this example, create and upload the inference image to a repository with the same name.
Run the following command to create the image repository in Amazon ECR.
```
aws ecr --region ${region} create-repository --repository-name "${repo}"
```
The full name of the Amazon ECR repository location is made up of the following parts: ` .dkr.ecr..amazonaws.com/`
To push the image to the repository, you must tag it with the full name of the repository location.
Set a variable for the full name of the image repository location along with the `latest` tag.
```
fullname="${account}.dkr.ecr.${region}.amazonaws.com/${repo}:latest"
```
Tag the image with the full name.
```
sudo docker tag ${image} ${fullname}
```
Finally, push the inference image to the repository in Amazon ECR.
```
sudo docker push ${fullname}
```
After the upload completes, the image appears in the [repository list of the Amazon ECR console](https://console.aws.amazon.com/ecr/repositories?region=us-east-2) in the Region that you are publishing from. In the previous example, the image was pushed to a repository in the US East (Ohio) Region.
## Scan your uploaded image
In the [Amazon ECR console](https://console.aws.amazon.com/ecr/repositories?region=us-east-2), choose the AWS Region that you are publishing from, and open the repository that the image was uploaded to. Select your uploaded image and start a scan to check for known vulnerabilities. AWS Marketplace checks the Amazon ECR scan results of the container images used in your Amazon SageMaker AI resource before publishing it. Before you can create your product, you must fix container images that have vulnerabilities with either a Critical or High severity.
After your images are scanned successfully, they can be used to create a model package or algorithm resource.
If you believe that your product had errors in the scan that are false positives, contact the [AWS Marketplace Seller Operations](https://aws.amazon.com/marketplace/management/contact-us) team with information about the error.
**Next steps**
+ See size limits in [Requirements and best practices for creating machine learning products](ml-listing-requirements-and-best-practices.md)
+ Continue to [Creating your Amazon SageMaker AI resource](ml-creating-your-amazon-sagemaker-resource.md)
# Creating your Amazon SageMaker AI resource
To publish a model package or an algorithm product, you must create the respective [ model package resource](https://docs.aws.amazon.com/marketplace/latest/userguide/ml-creating-your-amazon-sagemaker-resource.html#ml-creating-your-model-package-product) or [ algorithm resource](https://docs.aws.amazon.com/marketplace/latest/userguide/ml-creating-your-amazon-sagemaker-resource.html#ml-creating-your-algorithm-product) in Amazon SageMaker AI. When you create your resource for an AWS Marketplace product, it must be certified through a validation step. The validation step requires that you provide data to test your model package or algorithm resource before it can be published. The following sections show you how to create your SageMaker AI resource, either a model package resource or an algorithm resources. This includes setting the validation specifications that tell SageMaker AI how to perform the validation.
**Note**
If you haven't yet created the images for your product and uploaded them to Amazon Elastic Container Registry (Amazon ECR), see [Packaging your code into images for machine learning products in AWS Marketplace](ml-packaging-your-code-into-images.md) and [Uploading your images to Amazon Elastic Container Registry](ml-uploading-your-images.md) for information about how to do so.
**Topics**
+ [Creating your model package](#ml-creating-your-model-package-product)
+ [Creating your algorithm](#ml-creating-your-algorithm-product)
## Creating your model package
The following are requirements for creating a model package for AWS Marketplace:
+ An inference image stored in [Amazon ECR](https://aws.amazon.com/ecr/)
+ (Optional) Model artifacts, stored separately in [Amazon S3](https://aws.amazon.com/s3/)
+ Your test data used for inferences, stored in Amazon Simple Storage Service
**Note**
The following is about creating a model package product. For more information about model packages in SageMaker AI, see [Create a Model Package Resource](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-mkt-create-model-package.html).
### Creating the model package resources
The following procedures step you through creating the model package resources.
**Step 1: To create the model package resources**
1. Open the [ Amazon SageMaker AI console](https://us-east-2.console.aws.amazon.com/sagemaker/home).
1. Ensure that you are in the AWS Region that you want to publish from by looking at the top right of the page. For publishing, see the [Supported AWS Regions for publishing](ml-service-restrictions-and-limits.md#ml-supported-aws-regions-for-publishing) section. The inference image you uploaded to Amazon ECR in previous steps must be in the same Region.
1. In the left navigation menu, choose **Model packages**.
1. Choose **Create model package**.
After you create the package, you need to set the specifications of the inference package.
**Step 2: To set inference specifications**
1. Provide a **Name** for your model package (for example, *my-model-package*).
1. For **Location of inference image**, enter the URI of your inference image that was uploaded to Amazon ECR. You can retrieve the URI by locating your image in the [Amazon ECR console](https://us-east-2.console.aws.amazon.com/ecr/repositories).
1. If your model artifacts from training are bundled with your logic in your inference image, leave the **Location of model data artifacts** empty. Otherwise, specify the full Amazon S3 location of the compressed file (.tar.gz) of your model artifacts.
1. Using the dropdown box, choose the supported instance types of your inference image for both real-time inference (also known as *endpoint*) and batch-transform jobs.
1. Choose **Next**.
Before your model package can be created and published, validation is necessary to ensure that it functions as expected. This requires that you run a batch transform job with test data for inference that you provide. The validation specifications tell SageMaker AI how to perform the validation.
**Step 3: To set validation specifications**
1. Set **Publish this model package in AWS Marketplace** to **Yes**. If you set this to **No**, you can't publish this model package later. Choosing **Yes** [ certifies](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModelPackage.html#sagemaker-CreateModelPackage-request-CertifyForMarketplace) your model package for AWS Marketplace and requires the validation step.
1. If this is the first time completing this process, choose **Create a new role** for the **IAM role**. Amazon SageMaker AI uses this role when it deploys your model package. This includes actions, such as pulling images from Amazon ECR and artifacts from Amazon S3. Review the settings, and choose **Create role**. Creating a role here grants permissions described by the [ AmazonSageMakerFullAccess](https://console.aws.amazon.com/iam/home#/policies/arn:aws:iam::aws:policy/AmazonSageMakerFullAccess) IAM policy to the role that you create.
1. Edit the **JSON** in the validation profile. For details about allowed values, see [TransformJobDefinition](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_TransformJobDefinition.html).
1. `TransformInput.DataSource.S3Uri`: Set to where your test data for inference is stored.
1. `TransformInput.ContentType`: Specify your test data content type (for example, `application/json`, `text/plain`, `image/png `, or any other value). SageMaker AI does not validate the actual input data. This value is passed to your container HTTP endpoint in the `Content-type` header value.
1. `TransformInput.CompressionType`: Set to `None` if your test data for inference in Amazon S3 is not compressed.
1. `TransformInput.SplitType`: Set to `None` to pass each object in Amazon S3 as a whole for inference.
1. `TransformOutput.S3OutputPath`: Set to the location that the inference output is stored.
1. `TransformOutput.AssembleWith`: Set to `None` to output each inference as separate objects in Amazon S3.
1. Choose **Create model package**.
SageMaker AI pulls the inference image from Amazon ECR, copies any artifacts to the inference container, and runs a batch transform job using your test data for inference. After the validation succeeds, the status changes to **Completed**.
**Note**
The validation step does not evaluate the accuracy of the model with your test data. The validation step checks if the container runs and responds as expected.
You have completed creating your model product resources. Continue to [Listing your product in AWS Marketplace](ml-publishing-your-product-in-aws-marketplace.md).
## Creating your algorithm
The following are requirements for creating an algorithm for AWS Marketplace:
+ An inference image, stored in Amazon ECR
+ A training image, stored in Amazon ECR
+ Your test data for training, stored in Amazon S3
+ Your test data for inference, stored in Amazon S3
**Note**
The following walkthrough creates an algorithm product. For more information, see [Create an Algorithm Resource](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-mkt-create-algo.html).
### Creating the algorithm resources
The following procedures step you through creating the resources in your algorithm package.
**Step 1: To create the algorithm resources**
1. Open the [ Amazon SageMaker AI console](https://us-east-2.console.aws.amazon.com/sagemaker/home).
1. Ensure that you are in the AWS Region that you want to publish from by looking at the top right of the page (see [Supported AWS Regions for publishing](ml-service-restrictions-and-limits.md#ml-supported-aws-regions-for-publishing)). The training and inference images you uploaded to Amazon ECR in previous steps must be in this same Region.
1. In the left navigation menu, choose **Algorithms**.
1. Choose **Create algorithm**.
After you have created the algorithm package, you must set the specifications for the training and tuning of your model.
**Step 2: To set the training and tuning specifications**
1. Enter the **Name** for your algorithm (for example, *my-algorithm*).
1. For **Training image**, paste the full URI location of your training image that was uploaded to Amazon ECR. You can retrieve the URI by locating your image in the [Amazon ECR console](https://us-east-2.console.aws.amazon.com/ecr/repositories).
1. Using the dropdown box, choose the **instance types for training** that your training image supports.
1. Under the **Channel specification** section, add a channel for each input dataset that your algorithm supports, up to 20 channels of input sources. For more information, see [ Input Data Configuration](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo-running-container.html#your-algorithms-training-algo-running-container-inputdataconfig).
1. Choose **Next**.
1. If your algorithm supports hyperparameters and hyperparameter tuning, you must specify the tuning parameters.
1. Choose **Next**.
**Note**
We highly recommend that your algorithm supports hyperparameter tuning and makes appropriate parameters tunable. This allows data scientists to tune models to get the best results.
After you have set the tuning parameters, if any, you must set the specifications for your inference image.
**Step 3: To set inference image specification**
1. For **Location of inference image**, paste the URI of the inference image that was uploaded to Amazon ECR. You can retrieve the URI by locating your image in the [Amazon ECR Console](https://us-east-2.console.aws.amazon.com/ecr/repositories).
1. Using the dropdown box, choose the supported instance types for your inference image for both real-time inference (also known as *endpoint*) and batch-transform jobs.
1. Choose **Next**.
Before your algorithm can be created and published, validation is necessary to ensure that it functions as expected. This requires that you run both a training job with test data for training and a batch transform job with test data for inference that you provide. The validation specifications tell SageMaker AI how to perform the validation.
**Step 4: To set validation specifications**
1. Set **Publish this algorithm in AWS Marketplace** to **Yes**. If you set this to **No**, you can't publish this algorithm later. Choosing **Yes** [ certifies](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateAlgorithm.html#sagemaker-CreateAlgorithm-request-CertifyForMarketplace) your algorithm for AWS Marketplace and requires the validation specification.
1. If this is your first time creating a machine learning package for AWS Marketplace, choose **Create a new role** for the **IAM role**. Amazon SageMaker AI uses this role when training your algorithm and deploying the subsequent model package. This includes actions such as pulling images from Amazon ECR, storing artifacts in Amazon S3, and copying training data from Amazon S3. Review the settings, and choose **Create role**. Creating a role here grants permissions described by the [ AmazonSageMakerFullAccess](https://console.aws.amazon.com/iam/home#/policies/arn:aws:iam::aws:policy/AmazonSageMakerFullAccess) IAM policy to the role that you create.
1. Edit the **JSON** file in the validation profile for **Training job definition**. For more information about allowed values, see [ TrainingJobDefinition](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_TrainingJobDefinition.html).
1. `InputDataConfig`: In this JSON array, add a [Channel object](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_Channel.html) for each channel that you specified in the training-specification step. For each channel, specify where your test data for training is stored.
1. `OutputDataConfig`: After the training completes, the model artifacts in the training container directory path `/opt/ml/model/` are compressed and copied out to Amazon S3. Specify the Amazon S3 location of where the compressed file (.tar.gz) is stored.
1. Edit the JSON file in the validation profile for **Transform job definition**. For more information about allowed values, see [ TransformJobDefinition](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_TransformJobDefinition.html).
1. `TransformInput.DataSource.S3Uri`: Set to where your test data for inference is stored.
1. `TransformInput.ContentType`: Specify your test data content type. For example, `application/json`, `text/plain`, `image/png`, or any other value. Amazon SageMaker AI does not validate the actual input data. This value is passed to your container HTTP endpoint in the `Content-type` header value.
1. `TransformInput.CompressionType`: Set to `None` if your test data for inference in Amazon S3 is not compressed.
1. `TransformInput.SplitType`: Choose how you want objects in S3 split. For example, `None` passes each object in Amazon S3 as a whole for inference. For more details, see [ SplitType](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_TransformInput.html#sagemaker-Type-TransformInput-SplitType) in the Amazon SageMaker AI API Reference.
1. `TransformOutput.S3OutputPath`: Set to the location where the inference output is stored.
1. `TransformOutput.AssembleWith`: Set to `None` to output each inference as separate objects in Amazon S3.
1. Choose **Create algorithm package**.
SageMaker AI pulls the training image from Amazon ECR, runs a test-training job using your data, and stores the model artifacts in Amazon S3. It then pulls the inference image from Amazon ECR, copies the artifacts from Amazon S3 into the inference container, and runs a batch transform job using your test data for inference. After the validation succeeds, the status changes to **Completed**.
**Note**
The validation step does not evaluate the accuracy of the training or the model with your test data. The validation step checks if the containers run and respond as expected.
The validation step only validates batch processing. It is up to you to validate that real-time processing works with your product.
You have completed creating your algorithm product resources. Continue to [Listing your product in AWS Marketplace](ml-publishing-your-product-in-aws-marketplace.md).