Multi-turn reinforcement learning

Agent: the decision-making entity in the system. It observes the current situation, decides what action to take, and learns over time to improve its strategy. In practice, the agent is powered by an AI model, but the two are not the same thing. The model is the underlying neural network; the agent is the broader system that uses it to perceive, decide, and act. In simple terms: the customer service rep handling the ticket.

Environment/Agentic Application: everything the agent interacts with, including the conversation history, the customer's account details, and any tools the agent can use. In simple terms: the support platform, the customer, and all the information available to the rep.

State: a snapshot of the current situation the agent observes before deciding what to do next. In simple terms: what the rep knows right now, such as the customer's issue, what has already been tried, and the latest reply.

Action: what the agent does at each step, such as asking a clarifying question, calling a tool, or sending a response. In simple terms: the rep's next move, whether that is asking a question, looking something up, or sending a fix.

Reward: the feedback signal the agent receives, either at each step or at the end of the task, indicating how well things went. In simple terms: did the customer leave satisfied? That outcome is the reward.

Policy: the strategy the agent has learned. It maps a given state to the action most likely to lead to a good outcome. In simple terms: the rep's knowledge built up from experience, knowing what tends to work in a given situation.

Episode: one complete run of a task from start to finish. In simple terms: one full support conversation, from the customer's first message to the resolution.

Turn: a single exchange within an episode, typically one action taken by the agent and the response it receives from the environment. In a conversation, this is one message and its reply. In simple terms: one step in the support conversation, like the agent asking a question and the customer responding.

Trajectory: the full sequence of states, actions, and rewards recorded across an entire episode. It is the complete record of what the agent did and what happened as a result, used to compute the reward and update the policy. In simple terms: the entire transcript of the support conversation from start to finish, including every decision the agent made and how things unfolded.

Cumulative reward: the total reward accumulated across all steps in an episode, which is what the agent is ultimately trying to maximize. In simple terms: not just whether one step went well, but whether the whole conversation went well overall.

Your task requires back-and-forth interaction: The model needs to ask questions, gather information, and adapt based on what it learns along the way. A single prompt-response cycle is not enough. Example: a support agent that diagnoses an issue by asking follow-up questions before suggesting a fix.

The quality of the outcome depends on a sequence of actions: Getting the right answer at the end requires making the right moves throughout. Rewarding only the final output is not enough if the path to get there matters. Example: a coding assistant that plans, writes, runs, and debugs code across multiple steps.

The model needs to use tools across multiple steps: The model calls external tools like search, APIs, or code execution, and the results of one tool call influence what it does next. Example: a research assistant that searches for information, evaluates the results, and refines its query before producing a summary.

Mistakes mid-task should be recoverable: You want the model to recognize when something is not working and course-correct, rather than committing to a bad path from the start. Example: an agent that tries a solution, checks if it worked, and tries a different approach if it did not.

You are seeing good single-turn performance but poor end-to-end task completion: If your model handles individual steps well but struggles to string them together into a coherent, successful outcome, multi-turn RL can help bridge that gap.

Prefill: the cost of processing the input tokens fed into the model at the start of each training step. This includes the prompt, conversation history, and any context the model receives before generating a response.

Sample: the cost of the tokens the model generates during the training rollout. This is where the model produces its responses, which are then evaluated and used to compute the reward signal.

Train: the cost of the backward pass, where the model's weights are updated based on the reward signal. This is the core learning step in the RL process.

Prerequisites

Preparing your agent

Creating assets for multi-turn reinforcement learning

Training job submission

Model evaluation

Model deployment

Hyperparameters reference

Configure a VPC for multi-turn RL jobs

Encryption at rest for multi-turn reinforcement learning

Troubleshooting

Quotas

Sample datasets and tutorials