# Get started with AWS DeepRacer
<a name="deepracer-get-started"></a>

 To get started with AWS DeepRacer, let's first walk through the steps to use the AWS DeepRacer console to configure an agent with appropriate sensors for your autonomous driving requirements, to train a reinforcement learning model for the agent with the specified sensors, and to evaluate the trained model to determine the quality of the model. Once you've trained your model, you can iterate on it and submit it to a race. 

**Topics**
+ [Train your first AWS DeepRacer model](deepracer-get-started-training-model.md)
+ [Evaluate your AWS DeepRacer models in simulation](deepracer-get-started-test-in-simulator.md)

# Train your first AWS DeepRacer model
<a name="deepracer-get-started-training-model"></a>

This walkthrough demonstrates how to train your first model using the AWS DeepRacer console. 

## Train a reinforcement learning model using the AWS DeepRacer console
<a name="deepracer-get-started-train-model-proc"></a>

Learn where to find the **Create model** button in the AWS DeepRacer console to start your model training journey.

**To train a reinforcement learning model**

1. If this is your first time using AWS DeepRacer, choose **Create model** from the service landing page or select **Get started** under the **Reinforcement learning** heading on the main navigation pane. 

1. On the **Get started with reinforcement learning** page, under **Step 2: Create a model**, choose **Create model**.

   Alternatively, choose **Your models** under the **Reinforcement learning** heading from the main navigation pane. On the **Your models** page, choose **Create model**.

## Specify the model name and environment
<a name="deepracer--create-model-step-one.title"></a>

Name your model and learn how to pick the simulation track that's right for you.

**To specify the model name and environment**

1. On the **Create model** page, under **Training details**, enter a name for your model.

1. Optionally, add a training job description.

1. To learn more about adding optional tags, see [Tagging](deepracer-tagging.md).

1. Under **Environment simulation**, choose a track to serve as a training environment for your AWS DeepRacer agent. Under **Track direction**, choose **Clockwise** or **Counterclockwise**. Then, choose **Next**.

   For your first run, choose a track with a simple shape and smooth turns. In later iterations, you can choose more complex tracks to progressively improve your models. To train a model for a particular racing event, choose the track most similar to the event track.

1. Choose **Next** at the bottom of the page.

## Choose a race type and training algorithm
<a name="deepracer--create-model-step-two"></a>

The AWS DeepRacer console has three race types and two training algorithms from which to choose. Learn which are appropriate for your skill level and training goals. 

**To choose a race type and training algorithm**

1. On the **Create model** page, under **Race type**, select **Time trial**, **Object avoidance**, or **Head-to-bot**.

   For your first run, we recommend choosing **Time trial**. For guidance on optimizing your agent's sensor configuration for this race type, see [Tailor AWS DeepRacer training for time trials](deepracer-choose-race-type.md#deepracer-get-started-training-simple-time-trial).

1. Optionally, on later runs, choose **Object avoidance** to go around stationary obstacles placed at fixed or random locations along the chosen track. For more information, see [Tailor AWS DeepRacer training for object avoidance races](deepracer-choose-race-type.md#deepracer-get-started-training-object-avoidance).

   1. Choose **Fixed location** to generate boxes in fixed, user designated locations across the two lanes of the track or select **Random location** to generate objects that are randomly distributed across the two lanes at the beginning of each episode of your training simulation.

   1. Next, choose a value for the **Number of objects on a track**.

   1.  If you chose **Fixed location** you can adjust each object's placement on the track. For **Lane placement**, choose between the inside lane and the outside lane. By default, objects are evenly distributed across the track. To change how far between the start and the finish line an object is, enter a percentage of that distance between seven and 90 on the **Location (%) between start and finish** field. 

1. Optionally, for more ambitious runs, choose **Head-to-bot racing** to race against up to four bot vehicles moving at a constant speed. To learn more, see [Tailor AWS DeepRacer training for head-to-bot races](deepracer-choose-race-type.md#deepracer-get-started-training-h2h-racing).

   1. Under **Choose the number of bot vehicles**, select with how many bot vehicles you want your agent to train.

   1. Next, choose the speed in millimeters per second at which you want the bot vehicles to travel around the track.

   1. Optionally, check the **Enable lane changes** box to give the bot vehicles the ability to randomly change lanes every 1-5 seconds.

1. Under **Training algorithm and hyperparameters**, choose the **Soft Actor Critic (SAC)** or **Proximal Policy Optimization (PPO)** algorithm. In the AWS DeepRacer console, SAC models must be trained in continuous action spaces. PPO models can be trained in either continuous or discrete action spaces.

1. Under **Training algorithm and hyperparameters**, use the default hyperparameter values as-is.

   Later on, to improve training performance, expand **Hyperparameters** and modify the default hyperparameter values as follows:

   1. For **Gradient descent batch size**, choose [available options](deepracer-console-train-evaluate-models.md#deepracer-iteratively-adjust-hyperparameters). 

   1. For **Number of epochs**, set a [valid value](deepracer-console-train-evaluate-models.md#deepracer-iteratively-adjust-hyperparameters). 

   1. For **Learning rate**, set a [valid value](deepracer-console-train-evaluate-models.md#deepracer-iteratively-adjust-hyperparameters). 

   1. For **SAC alpha value** (SAC algorithm only), set a [valid value](deepracer-console-train-evaluate-models.md#deepracer-iteratively-adjust-hyperparameters).

   1. For **Entropy**, set a [valid value](deepracer-console-train-evaluate-models.md#deepracer-iteratively-adjust-hyperparameters).

   1. For **Discount factor**, set a [valid value](deepracer-console-train-evaluate-models.md#deepracer-iteratively-adjust-hyperparameters).

   1. For **Loss type**, choose [available options](deepracer-console-train-evaluate-models.md#deepracer-iteratively-adjust-hyperparameters).

   1. For **Number of experience episodes between each policy-updating iteration**, set a [valid value](deepracer-console-train-evaluate-models.md#deepracer-iteratively-adjust-hyperparameters).

   For more information about hyperparameters, see [Systematically tune hyperparameters](deepracer-console-train-evaluate-models.md#deepracer-iteratively-adjust-hyperparameters).

1. Choose **Next**.

## Define action space
<a name="deepracer--create-model-specify-action-space"></a>

On the **Define action space** page, if you've chosen to train with the Soft Actor Critic (SAC) algorithm, your default action space is the continuous action space. If you've chosen to train with the Proximal Policy Optimization (PPO) algorithm, choose between **Continuous action space** and **Discrete action space**. To learn more about how each action space and algorithm shapes the agent's training experience, see [AWS DeepRacer action space and reward function](deepracer-how-it-works-action-space.md).

### To define continuous action space (SAC or PPO algorithms)
<a name="deepracer--create-model-specify-action-space"></a>

1. Under **Define continuous action space**, choose the degrees of your **Left steering angle range** and **Right steering angle range**.

   Try entering different degrees for each steering angle range and watch the visualization of your range change to represent your choices on the **Dynamic sector graph**.  
![\[\]](http://docs.aws.amazon.com/deepracer/latest/developerguide/images/deepracer-dynamic-sector-graph.png)

1. Under **Speed**, enter a minimum and maximum speed for your agent in millimeters per second.

   Notice how your changes are reflected on the **Dynamic sector graph**.

1. Optionally, choose **Reset to default values** to clear unwanted values. We encourage trying out different values on the graph to experiment and learn.

1. Choose **Next**.

### To define discrete action space (PPO algorithm only)
<a name="deepracer--create-model-specify-action-space"></a>

1. Choose a value for **Steering angle granularity** from the dropdown list.

1. Choose a value in degrees between 1-30 for your agent's **Maximum steering angle**. 

1. Choose a value for **Speed granularity** from the dropdown list.

1. Choose a value in millimeters per second between 0.1-4 for your agent's **Maximum speed**. 

1. Use the default action settings on the **Action list** or, optionally, toggle on **Advanced configuration** to fine tune your settings. If you choose **Previous** or toggle off **Advanced configuration** after adjusting values, you lose your changes.  
![\[\]](http://docs.aws.amazon.com/deepracer/latest/developerguide/images/deepracer-advanced-configuration-toggle.png)

   1. Enter a value in degrees between -30 and 30 in the **Steering angle** column.

   1. Enter a value between 0.1 and 4 in millimeters per second for up to nine actions in the **Speed** column.

   1. Optionally, select **Add an action** to increase the number of rows in the action list.  
![\[\]](http://docs.aws.amazon.com/deepracer/latest/developerguide/images/deepracer-add-an-action.png)

   1. Optionally, select **X** on a row to remove it.

1. Choose **Next**.

## Choose a virtual car
<a name="deepracer-create-model-step-four"></a>

Learn how to get started with virtual cars. Earn new custom cars, paint jobs, and modifications by competing in the Open Division each month.

**To choose a virtual car**

1. On the **Choose vehicle shell and sensor configuration** page, choose a shell that is compatible with your race type and action space. If you don't have a car in your garage that matches, go to **Your garage** under the **Reinforcement learning** heading on the main navigation pane to create one.

   For **Time trial** training, the default sensor configuration and single-lens camera of **The Original DeepRacer** is all you need, but all other shells and sensor configurations work as long as the action space matches. For more information, see [Tailor AWS DeepRacer training for time trials](deepracer-choose-race-type.md#deepracer-get-started-training-simple-time-trial).

   For **Object avoidance** training, stereo cameras are helpful, but a single camera can also be used for avoiding stationary obstacles in fixed locations. A LiDAR sensor is optional. See [AWS DeepRacer action space and reward function](deepracer-how-it-works-action-space.md).

   For **Head-to-bot** training, in addition to either a single camera or a stereo camera, a LiDAR unit is optimal for detecting and avoiding blind spots while passing other moving vehicles. To learn more, see [Tailor AWS DeepRacer training for head-to-bot races](deepracer-choose-race-type.md#deepracer-get-started-training-h2h-racing).

1. Choose **Next**.

## Customize your reward function
<a name="deepracer-create-model-step-five"></a>

The reward function is at the core of reinforcement learning. Learn to use it to incentivize your car (agent) to take specific actions as it explores the track (environment). Like encouraging and discouraging certain behaviors in a pet, you can use this tool to encourage your car to finish a lap as fast as possible and discourage it from driving off of the track or colliding with objects.

**To customize your reward function**

1. On the **Create model** page, under **Reward function**, use the default reward function example as-is for your first model.   
![\[\]](http://docs.aws.amazon.com/deepracer/latest/developerguide/images/deepracer-reward-function-example-editor.png)

   Later on, you can choose **Reward function examples** to select another example function and then choose **Use code** to accept the selected reward function.

   There are four example functions with which you can start. They illustrate how to follow the track center (default), how to keep the agent inside the track borders, how to prevent zig-zag driving, and how to avoid crashing into stationary obstacles or other moving vehicles. 

   To learn more about the reward function, see [AWS DeepRacer reward function reference](deepracer-reward-function-reference.md).

1. Under **Stop conditions**, leave the default **Maximum time** value as-is, or set a new value to terminate the training job, to help prevent long-running (and possible run-away) training jobs. 

   When experimenting in the early phase of training, you should start with a small value for this parameter and then progressively train for longer amounts of time.

1. Under **Automatically submit to the AWS DeepRacer**, **Submit this model to the AWS DeepRacer automatically after training completion and get a chance to win prizes** is checked by default. Optionally, you may opt out of entering your model by selecting the checkmark.

1. Under **League requirements**, select your **Country of residence** and accept the the terms and conditions by checking the box. 

1. Choose **Create model** to start creating the model and provisioning the training job instance. 

1. After the submission, watch your training job being initialized and then run. 

   The initialization process takes a few minutes to change from **Initializing** to **In progress**.

1. Watch the **Reward graph** and **Simulation video stream** to observe the progress of your training job. You can choose the refresh button next to **Reward graph** periodically to refresh the **Reward graph** until the training job is complete.   
![\[\]](http://docs.aws.amazon.com/deepracer/latest/developerguide/images/deepracer-training-in-progress.png)

The training job runs on the AWS Cloud, so you don't need to keep the AWS DeepRacer console open. You can always come back to the console to check on your model at any point while the job is in progress. 

If the** Simulation video stream** window or the **Reward graph** display become unresponsive, refresh the browser page to get the training progress updated.

# Evaluate your AWS DeepRacer models in simulation
<a name="deepracer-get-started-test-in-simulator"></a>

 After your training job is complete, you should evaluate the trained model to assess its convergency behavior. The evaluation proceeds by completing a number of trials on a chosen track and having the agent move on the track according to likely actions inferred by the trained model. The performance metrics include a percentage of track completion and the time running on each track from start to finish or going off-track. 

To evaluate your trained model, you can use the AWS DeepRacer console. To do so, follow the steps in this topic. 

**To evaluate a trained model in the AWS DeepRacer console**

1. Open the AWS DeepRacer console at https://console.aws.amazon.com/deepracer. 

1. From the main navigation pane, choose **Models** and then choose the model you just trained from the **Models** list to open the model details page.

1.  Select the **Evaluation** tab. 

1. In **Evaluation details**, choose **Start evaluation**.  
![\[Evaluation details section showing training complete message and option to start evaluation.\]](http://docs.aws.amazon.com/deepracer/latest/developerguide/images/evaluation_details_start.png)

   You can start an evaluation after your training job status changes to **Completed** or the model's status changes to **Ready** if the training job wasn't completed. 

   A model is ready when the training job is complete. If the training wasn't completed, the model can also be in a **Ready** state if it's trained up to the failing point.

1. On the **Evaluate model** page, under **Race type**, enter a name for your evaluation, then choose the racing type that you chose to train the model. 

   For evaluation you can choose a race type different from the race type used in training. For example, you can train a model for head-to-bot races and then evaluate it for time trials. In general, the model must generalize well if the training race type differs from the evaluation race type. For your first run, you should use the same race type for both evaluation and training. 

1. On the **Evaluate model** page, under **Evaluate criteria**, choose the number of trials you want to run, then choose a track to evaluate the model on.  
![\[\]](http://docs.aws.amazon.com/deepracer/latest/developerguide/images/evaluation_select_track.png)

   Typically, you want to choose a track that is the same as or similar to the one you used in [training the model](deepracer-get-started-training-model.md#deepracer-get-started-train-model-proc). You can choose any track for evaluating your model, however, you can expect the best performance on a track most similar to the one used in training. 

   To see if your model generalizes well, choose an evaluation track different from the one used in training. 

1. On the **Evaluate model** page, under **Virtual Race Submission**, for your first model, turn off the **Submit model after evaluation** option. Later, if you want to participate in a racing event, leave this option turned on.  
![\[Virtual race submission section with options for submitting a model for evaluation.\]](http://docs.aws.amazon.com/deepracer/latest/developerguide/images/deepracer-get-started-evaluate-virtual-race-submit.png)

1. On the **Evaluate model** page, choose **Start evaluation** to start creating and initializing the evaluation job. 

   This initialization process takes about 3 minutes to complete. 

1. As the evaluation progresses, the evaluation results, including the trial time and track completion rate, are displayed under **evaluation details** after each trial. In the **Simulation video stream** window, you can watch how the agent performs on the chosen track.

    You can stop an evaluation job before it completes. To stop the evaluation job, choose **Stop evaluation** on the upper-right corner of the **Evaluation** card and then confirm to stop the evaluation. 

1. After the evaluation job is complete, examine the performance metrics of all the trials under **Evaluation results**. The accompanying simulation video stream is no longer available. 

   A history of your model's evaluations is available in the **Evaluation selector**. To view the details of a specific evaluation, select the evaluation from the **Evaluation selector** list, then choose **Load evaluation** from the top-right corner of the **Evaluation selector** card.   
![\[\]](http://docs.aws.amazon.com/deepracer/latest/developerguide/images/evaluation_selector_complete.png)

   For this particular evaluation job, the trained model completes the trials with a significant off-track time penalty. As a first run, this is not unusual. Possible reasons include that the training didn't converge and the training needs more time, the action space needs to be enlarged to give the agent more room to react, or the reward function needs to be updated to handle varying environments. 

    You can continue to improve the model by cloning a previously trained one, modifying the reward function, tuning hyperparameters, and then iterating the process until the total reward converges and the performance metrics improve. For more information on how to improve the training, see [Train and evaluate AWS DeepRacer models](create-deepracer-project.md). 

 To transfer your completely trained model to your AWS DeepRacer device for driving in a physical environment, you need to download the model artifacts. To do so, choose **Download model** on the model's details page. If your AWS DeepRacer physical device doesn't support new sensors and your model has been trained with the new sensor types, you'll get an error message when you use the model on your AWS DeepRacer device in a real-world environment. For more information about testing an AWS DeepRacer model with a physical device, see [Operate your AWS DeepRacer vehicle](operate-deepracer-vehicle.md).

Once you've trained your model on a track identical or similar to the one specified in an AWS DeepRacer League racing event or an AWS DeepRacer community race, you can submit the model to the virtual races in the AWS DeepRacer console. To do this, follow **AWS virtual circuit** or **Community races** on the main navigation pane. For more information, see [Join an AWS DeepRacer race](deepracer-racing-series.md). 



To train a model for obstacle avoidance or head-to-bot racing, you may need to add new sensors to the model and the physical device. For more information, see [Understanding racing types and enabling sensors supported by AWS DeepRacer](deepracer-choose-race-type.md).