# Fine-tune open-weight models using OpenAI-compatible APIs
Amazon Bedrock provides OpenAI compatible API endpoints for fine-tuning foundation models. These endpoints allow you to use familiar OpenAI SDKs and tools to create, monitor, and manage fine-tuning jobs with Amazon Bedrock models. This page highlights using these APIs for reinforcement fine-tuning.
## Key capabilities
+ **Upload training files** – Use the Files API to upload and manage training data for fine-tuning jobs
+ **Create fine-tuning jobs** – Start fine-tuning jobs with custom training data and reward functions
+ **List and retrieve jobs** – View all fine-tuning jobs and get detailed information about specific jobs
+ **Monitor job events** – Track fine-tuning progress through detailed event logs
+ **Access checkpoints** – Retrieve intermediate model checkpoints created during training
+ **Immediate inference** – After fine-tuning completes, use the resulting fine-tuned model for on-demand inference through Amazon Bedrock's OpenAI-compatible APIs (Responses/chat completions API) without additional deployment steps
+ **Easy migration** – Compatible with existing OpenAI SDK codebases
## Reinforcement fine-tuning workflow for open-weight models
Before fine-tuning, ensure you have the pre-requisites as Amazon Bedrock needs specific permissions to create and manage the fine-tuning process. For comprehensive security and permissions information, see [Access and security for open-weight models](rft-open-weight-access-security.md).
Run reinforcement fine-tuning for open-weight models in 5 steps:
1. **Upload Training Dataset** – Use the Files API to upload prompts in required format (e.g., JSONL) with purpose "fine-tune" as the reinforcement fine-tuning training dataset. For more information, see [Prepare data for open-weight models](rft-prepare-data-open-weight.md).
1. **Configure Reward Function** – Define a grader to score model responses based on correctness, structure, tone, or other objectives using Lambda functions. For more information, see [Setting up reward functions for open-weight models](reward-functions-open-weight.md).
1. **Create Fine-tuning Job** – Launch the reinforcement fine-tuning job using the OpenAI-compatible API by specifying base model, dataset, reward function, and other optional settings such as hyperparameters. For more information, see [Create fine-tuning job](fine-tuning-openai-job-create.md#fine-tuning-openai-create-job).
1. **Monitor Training Progress** – Track job status, events, and training metrics using the fine-tuning jobs APIs. For more information, see [List fine-tuning events](fine-tuning-openai-job-create.md#fine-tuning-openai-list-events). Access intermediate model checkpoints to evaluate performance at different training stages, see [List fine-tuning checkpoints](fine-tuning-openai-job-create.md#fine-tuning-openai-list-checkpoints).
1. **Run Inference** – Use the fine-tuned model ID directly for inference through Amazon Bedrock's OpenAI-compatible Responses or Chat Completions APIs. For more information, see [Run inference with fine-tuned model](fine-tuning-openai-job-create.md#fine-tuning-openai-inference).
## Supported regions and endpoints
The following table shows the foundation models and regions that support OpenAI compatible fine-tuning APIs:
**Supported models and regions for OpenAI compatible fine-tuning APIs**
| Provider | Model | Model ID | Region name | Region | Endpoint |
| --- | --- | --- | --- | --- | --- |
| OpenAI | Gpt-oss-20B | openai.gpt-oss-20b | US West (Oregon) | us-west-2 | bedrock-mantle.us-west-2.api.aws |
| Qwen | Qwen3 32B | qwen.qwen3-32b | US West (Oregon) | us-west-2 | bedrock-mantle.us-west-2.api.aws |
# Access and security for open-weight models
Before you begin reinforcement fine-tuning (RFT), make sure that you understand what kind of access Amazon Bedrock needs for RFT-specific operations. RFT requires additional permissions beyond standard fine-tuning due to its reward function execution capabilities.
## Prerequisites
Before using Amazon Bedrock's OpenAI-compatible fine-tuning APIs, ensure you have the following:
1. An AWS account with appropriate permissions to access Amazon Bedrock
1. **Authentication** – You can authenticate using:
+ Amazon Bedrock API key (required for OpenAI SDK and available for HTTP requests)
+ AWS credentials (supported for HTTP requests)
**Note**
If you are using Amazon Bedrock short-term/long-term API keys, then make sure that your role has access to the following IAM policy permissions: `AmazonBedrockMantleFullAccess` and [AWSLambdaRole](https://docs.aws.amazon.com/bedrock/latest/ug/rft-open-weight-access-security#openai-fine-tuning-lambda-permissions).
1. **OpenAI SDK (optional)** – Install the OpenAI Python SDK if using SDK-based requests.
1. **Environment variables** – Set the following environment variables:
+ `OPENAI_API_KEY` – Set to your Amazon Bedrock API key
+ `OPENAI_BASE_URL` – Set to the Amazon Bedrock endpoint for your region (for example, `https://bedrock-mantle.us-west-2.api.aws/v1`)
For more information, see [Responses API](bedrock-mantle.md#bedrock-mantle-responses).
1. **Training data** formatted as JSONL files with the purpose `fine-tune`. For more information, see [Prepare data for open-weight models](rft-prepare-data-open-weight.md).
## Lambda permissions for reward functions
You must add Lambda invocation permissions. The following shows an example policy you can use:
```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction"
],
"Resource": [
"arn:aws:lambda:*:*:function:reward-function-name"
]
}
]
}
```
You can also use Amazon Bedrock hosted models as Judges for setting up reward functions. You will need to add specific permissions to invoke foundation models to the Lambda execution role. In your lambda role, you can configure these managed policies for LLMs for grading. See [AmazonBedrockLimitedAccess](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonBedrockLimitedAccess.html).
The following is an example for invoking Amazon Bedrock foundation models as judge using the Invoke API:
```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel"
],
"Resource": [
"arn:aws:bedrock:*:*:foundation-model/*"
]
}
]
}
```
# Prepare data for open-weight models
When you fine-tune open-weight models with reinforcement fine-tuning using OpenAI-compatible APIs, provide training data by bringing your own prompts in JSONL format with the purpose `fine-tune`.
## Training data format and requirements
Training data must follow the OpenAI chat completions format with 100-20K examples. Each training example contains:
+ `messages`: In this field, include the user, system or assistant role containing the input prompt provided to the model.
+ `reference_answer`: In this field, it should contain the expected output or evaluation criteria that your reward function uses to score the model's response. It is not limitedto structured outputs—it can contain any format that helps your reward function evaluate quality.
+ [Optional] You can add fields used by grader Lambda for grading.
**Requirements:**
+ JSONL format with prompts in OpenAI chat completion format (one prompt per line)
+ Purpose must be set to `fine-tune`
+ A minimum of 100 records in training dataset
+ Amazon Bedrock automatically validates training dataset format
------
#### [ Example: General question-answering ]
```
{
"messages": [
{
"role": "system",
"content": "You are a helpful assistant"
},
{
role": "user",
"content": "What is machine learning?"}
],
"reference_answer": "Machine learning is a subset of artificial intelligence that enables computers to learn and make decisions from data without being explicitly programmed."
}
```
------
#### [ Example: Math problem ]
```
{
"id": "sample-001",
"messages": [
{
"role": "system",
"content": "You are a math tutor"
},
{
"role": "user",
"content": "Solve: 2x + 5 = 13"
}
],
"reference_answer": {
"solution": "x = 4",
"steps": ["2x = 13 - 5", "2x = 8", "x = 4"]
}
}
```
------
## Files API
You can use OpenAI-compatible files api to upload your training data for fine-tuning jobs. Files are stored securely in Amazon Bedrock, and are used when creating fine-tuning jobs. For complete API details, see the [OpenAI Files documentation](https://platform.openai.com/docs/api-reference/files).
### Upload training file
To upload a training file, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# Upload training file
with open(TRAINING_FILE_PATH, 'rb') as f:
file_response = client.files.create(
file=f,
purpose='fine-tune'
)
# Store file ID for next steps
training_file_id = file_response.id
print(f"✅ Training file uploaded successfully: {training_file_id}")
```
------
#### [ HTTP request ]
Make a POST request to `/v1/files`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/files \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-F purpose="fine-tune" \
-F file="@training_data.jsonl"
```
------
### Retrieve file details
To retrieve details about a specific file, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# Retrieve file details
file_details = client.files.retrieve(training_file_id)
# Print raw response
print(json.dumps(file_details.model_dump(), indent=2))
```
------
#### [ HTTP request ]
Make a GET request to `/v1/files/{file_id}`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/files/file-abc123 \
-H "Authorization: Bearer $OPENAI_API_KEY"
```
------
### List files
To list uploaded files, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# List files
files_response = client.files.list(purpose='fine-tune')
# Print raw response
print(json.dumps(files_response.model_dump(), indent=2))
```
------
#### [ HTTP request ]
Make a GET request to `/v1/files`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/files?purpose=fine-tune \
-H "Authorization: Bearer $OPENAI_API_KEY"
```
------
### Delete file
To delete a file, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# Delete file
delete_response = client.files.delete(training_file_id)
```
------
#### [ HTTP request ]
Make a DELETE request to `/v1/files/{file_id}`:
```
curl -X DELETE https://bedrock-mantle.us-west-2.api.aws/v1/files/file-abc123 \
-H "Authorization: Bearer $OPENAI_API_KEY"
```
------
## Characteristics of effective training data
Effective RFT training data requires three key characteristics:
+ **Clarity and consistency** – Use clear, unambiguous prompts with consistent formatting. Avoid contradictory labels, ambiguous instructions, or conflicting reference answers that mislead training.
+ **Diversity** – Include varied input formats, edge cases, and difficulty levels that reflect production usage patterns across different user types and scenarios.
+ **Efficient reward functions** – Design functions that execute quickly (seconds, not minutes), parallelize with AWS Lambda, and return consistent scores for cost-effective training.
## Additional properties
The RFT data format supports custom fields beyond the core schema requirements (`messages` and `reference_answer`). This flexibility allows you to add any additional data your reward function needs for proper evaluation.
**Note**
You don't need to configure this in your recipe. The data format inherently supports additional fields. Simply include them in your training data JSON, and they will be passed to your reward function in the `metadata` field.
**Common additional properties**
+ `task_id` – Unique identifier for tracking
+ `difficulty_level` – Problem complexity indicator
+ `domain` – Subject area or category
+ `expected_reasoning_steps` – Number of steps in solution
These additional fields are passed to your reward function during evaluation, enabling sophisticated scoring logic tailored to your specific use case.
**Examples with additional properties**
------
#### [ Chemistry problem ]
```
{
"id": "chem-001",
"messages": [
{
"role": "system",
"content": "You are a helpful chemistry assistant"
},
{
"role": "user",
"content": "Predict hydrogen bond donors and acceptors for this SMILES: CCN(CC)CCC(=O)c1sc(N)nc1C"
}
],
"reference_answer": {
"donor_bond_counts": 2,
"acceptor_bond_counts": 4
}
}
```
The `reference_answer` field contains the expected output or evaluation criteria that your reward function uses to score the model's response. It is not limited to structured outputs—it can contain any format that helps your reward function evaluate quality.
------
#### [ Math problem with metadata ]
```
{
"messages": [
{
"role": "system",
"content": "You are a math tutor"
},
{
"role": "user",
"content": "Solve: 2x + 5 = 13"
}
],
"reference_answer": {
"solution": "x = 4",
"steps": ["2x = 13 - 5", "2x = 8", "x = 4"]
},
"task_id": "algebra_001",
"difficulty_level": "easy",
"domain": "algebra",
"expected_reasoning_steps": 3
}
```
------
# Setting up reward functions for open-weight models
Reward functions evaluate response quality and provide feedback signals for model training. You can set up reward functions using custom Lambda functions. Choose the approach that matches your task requirements.
## Custom Lambda functions for reward evaluation
You can set up reward functions using custom Lambda functions. Within your Lambda function, you have flexibility in how you implement the evaluation logic:
+ **Objective tasks** – For objective tasks like code generation or math reasoning, use verifiable rule-based graders that check correctness against known standards or test cases.
+ **Subjective tasks** – For subjective tasks like instruction following or chatbot interactions, call Amazon Bedrock foundation models as judges within your Lambda function to evaluate response quality based on your criteria.
Your Lambda function can implement complex logic, integrate external APIs, perform multi-step calculations, or combine multiple evaluation criteria depending on your task requirements.
**Note**
When using custom Lambda functions:
Increase the Lambda timeout from default 3 seconds to maximum 15 minutes for complex evaluations.
The Lambda execution role needs permissions to invoke the Lambda function as described in [Lambda permissions for reward functions](rft-open-weight-access-security.md#openai-fine-tuning-lambda-permissions).
## Lambda function implementation details
When implementing custom Lambda reward functions, your function must accept and return data in the following format.
------
#### [ Input structure ]
```
[{
"id": "123",
"messages": [
{
"role": "user",
"content": "Do you have a dedicated security team?"
},
{
"role": "assistant",
"content": "As an AI developed by Amazon, I don not have a dedicated security team..."
}
],
"metadata": {
"reference_answer": {
"compliant": "No",
"explanation": "As an AI developed by Company, I do not have a traditional security team..."
},
"my_key": "sample-001"
}
}]
```
------
#### [ Output structure ]
```
[{
"id": "123",
"aggregate_reward_score": 0.85,
"metrics_list": [
{
"name": "accuracy",
"value": 0.9,
"type": "Reward"
},
{
"name": "policy_compliance",
"value": 0.8,
"type": "Metric"
}
]
}]
```
------
**Design guidelines**
+ **Rank responses** – Give the best answer a clearly higher score
+ **Use consistent checks** – Evaluate task completion, format adherence, safety, and reasonable length
+ **Maintain stable scaling** – Keep scores normalized and non-exploitable
# Create and manage fine-tuning jobs for open-weight models using OpenAI APIs
The OpenAI-compatible fine-tuning job APIs allow you to create, monitor, and manage fine-tuning jobs. This page highlights using these APIs for reinforcement fine-tuning. For complete API details, see the [OpenAI Fine-tuning documentation](https://platform.openai.com/docs/api-reference/fine-tuning).
## Create fine-tuning job
Creates a fine-tuning job that begins the process of creating a new model from a given dataset. For complete API details, see the [OpenAI Create fine-tuning jobs documentation](https://developers.openai.com/api/reference/resources/fine_tuning/subresources/jobs/methods/create).
### Examples
To create a fine-tuning job with RFT method, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# Create fine-tuning job with RFT method
job_response = client.fine_tuning.jobs.create(
model=MODEL_ID,
training_file=training_file_id,
# Suffix field is not supported so commenting for now.
# suffix="rft-example", # Optional: suffix for fine-tuned model name
extra_body={
"method": {
"type": "reinforcement",
"reinforcement": {
"grader": {
"type": "lambda",
"lambda": {
"function": "arn:aws:lambda:us-west-2:123456789012:function:my-reward-function" # Replace with your Lambda ARN
}
},
"hyperparameters": {
"n_epochs": 1, # Number of training epochs
"batch_size": 4, # Batch size
"learning_rate_multiplier": 1.0 # Learning rate multiplier
}
}
}
}
)
# Store job ID for next steps
job_id = job_response.id
print({job_id})
```
------
#### [ HTTP request ]
Make a POST request to `/v1/fine_tuning/jobs`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/fine_tuning/jobs \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"training_file": "file-abc123",
"model": "gpt-4o-mini",
"method": {
"type": "reinforcement",
"reinforcement": {
"grader": {
"type": "lambda",
"lambda": {
"function": "arn:aws:lambda:us-west-2:123456789012:function:my-grader"
}
},
"hyperparameters": {
"n_epochs": 1,
"batch_size": 4,
"learning_rate_multiplier": 1.0
}
}
}
}'
```
------
## List fine-tuning events
Lists events for a fine-tuning job. Fine-tuning events provide detailed information about the progress of your job, including training metrics, checkpoint creation, and error messages. For complete API details, see the [OpenAI List fine-tuning events documentation](https://developers.openai.com/api/reference/resources/fine_tuning/subresources/jobs/methods/list_events).
### Examples
To list fine-tuning events, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# List fine-tuning events
events = client.fine_tuning.jobs.list_events(
fine_tuning_job_id="ftjob-abc123",
limit=50
)
for event in events.data:
print(f"[{event.created_at}] {event.level}: {event.message}")
if event.data:
print(f" Metrics: {event.data}")
```
------
#### [ HTTP request ]
Make a GET request to `/v1/fine_tuning/jobs/{fine_tuning_job_id}/events`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/fine_tuning/jobs/ftjob-abc123/events?limit=50
```
------
Events include information such as:
+ Training started and completed messages
+ Checkpoint creation notifications
+ Training metrics (loss, accuracy) at each step
+ Error messages if the job fails
To paginate through all events, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# Paginate through all events
all_events = []
after = None
while True:
events = client.fine_tuning.jobs.list_events(
fine_tuning_job_id="ftjob-abc123",
limit=100,
after=after
)
all_events.extend(events.data)
if not events.has_more:
break
after = events.data[-1].id
```
------
#### [ HTTP request ]
Make multiple GET requests with the `after` parameter:
```
# First request
curl https://bedrock-mantle.us-west-2.api.aws/v1/fine_tuning/jobs/ftjob-abc123/events?limit=100
# Subsequent requests with 'after' parameter
curl "https://bedrock-mantle.us-west-2.api.aws/v1/fine_tuning/jobs/ftjob-abc123/events?limit=100&after=ft-event-abc123"
```
------
## Retrieve fine-tuning job
Get detailed information about a fine-tuning job. For complete API details, see the [OpenAI Retrieve fine-tuning job documentation](https://developers.openai.com/api/reference/resources/fine_tuning/subresources/jobs/methods/retrieve).
### Examples
To retrieve specific job details, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# Retrieve specific job details
job_details = client.fine_tuning.jobs.retrieve(job_id)
# Print raw response
print(json.dumps(job_details.model_dump(), indent=2))
```
------
#### [ HTTP request ]
Make a GET request to `/v1/fine_tuning/jobs/{fine_tuning_job_id}`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/fine_tuning/jobs/ftjob-abc123 \
-H "Authorization: Bearer $OPENAI_API_KEY"
```
------
## List fine-tuning jobs
Lists your organization's fine-tuning jobs with pagination support. For complete API details, see the [OpenAI List fine-tuning jobs documentation](https://developers.openai.com/api/reference/resources/fine_tuning/subresources/jobs/methods/list).
### Examples
To list fine-tuning jobs with limit and pagination, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# List fine-tuning jobs with limit and pagination
response = client.fine_tuning.jobs.list(
limit=20 # Maximum number of jobs to return
)
# Print raw response
print(json.dumps(response.model_dump(), indent=2))
```
------
#### [ HTTP request ]
Make a GET request to `/v1/fine_tuning/jobs`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/fine_tuning/jobs?limit=20 \
-H "Authorization: Bearer $OPENAI_API_KEY"
```
------
## Cancel fine-tuning job
Cancels a fine-tuning job that is in progress. Once cancelled, the job cannot be resumed. For complete API details, see the [OpenAI Cancel fine-tuning job documentation](https://developers.openai.com/api/reference/resources/fine_tuning/subresources/jobs/methods/cancel).
### Examples
To cancel a fine-tuning job, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# Cancel fine-tuning job
cancel_response = client.fine_tuning.jobs.cancel("ftjob-abc123")
print(f"Job ID: {cancel_response.id}")
print(f"Status: {cancel_response.status}") # Should be "cancelled"
```
------
#### [ HTTP request ]
Make a POST request to `/v1/fine_tuning/jobs/{fine_tuning_job_id}/cancel`:
```
curl -X POST https://bedrock-mantle.us-west-2.api.aws/v1/fine_tuning/jobs/ftjob-abc123/cancel \
-H "Authorization: Bearer $OPENAI_API_KEY"
```
------
## List fine-tuning checkpoints
Lists checkpoints for a fine-tuning job. Checkpoints are intermediate model snapshots created during fine-tuning that can be used for inference to evaluate performance at different training stages. For more information, see the [OpenAI List fine-tuning checkpoints documentation](https://developers.openai.com/api/reference/resources/fine_tuning/subresources/jobs/subresources/checkpoints/methods/list).
### Examples
To list checkpoints for a fine-tuning job, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# List checkpoints for a fine-tuning job
checkpoints = client.fine_tuning.jobs.checkpoints.list(
fine_tuning_job_id="ftjob-abc123",
limit=10
)
for checkpoint in checkpoints.data:
print(f"Checkpoint ID: {checkpoint.id}")
print(f"Step: {checkpoint.step_number}")
print(f"Model: {checkpoint.fine_tuned_model_checkpoint}")
print(f"Metrics: {checkpoint.metrics}")
print("---")
```
------
#### [ HTTP request ]
Make a GET request to `/v1/fine_tuning/jobs/{fine_tuning_job_id}/checkpoints`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/fine_tuning/jobs/ftjob-abc123/checkpoints?limit=10
```
------
Each checkpoint includes:
+ **Checkpoint ID** – Unique identifier for the checkpoint
+ **Step number** – Training step at which the checkpoint was created
+ **Model checkpoint** – Model identifier that can be used for inference
+ **Metrics** – Validation loss and accuracy at this checkpoint
To use a checkpoint model for inference, choose the tab for your preferred method, and then follow the steps:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# Test inference with a checkpoint
response = client.chat.completions.create(
model=checkpoint.fine_tuned_model_checkpoint,
messages=[{"role": "user", "content": "What is AI?"}],
max_tokens=100
)
print(response.choices[0].message.content)
```
------
#### [ HTTP request ]
Make a POST request to `/v1/chat/completions`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "ft:gpt-4o-mini:openai:custom:7p4lURel:ckpt-step-1000",
"messages": [{"role": "user", "content": "What is AI?"}],
"max_tokens": 100
}'
```
------
## Run inference with fine-tuned model
Once your fine-tuning job is complete, you can use the fine-tuned model for inference through the Responses API or Chat Completions API. For complete API details, see [Generate responses using OpenAI APIs](bedrock-mantle.md).
### Responses API
Use the Responses API for single-turn text generation with your fine-tuned model:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# Get the fine-tuned model ID
job_details = client.fine_tuning.jobs.retrieve("ftjob-abc123")
if job_details.status == 'succeeded' and job_details.fine_tuned_model:
fine_tuned_model = job_details.fine_tuned_model
print(f"Using fine-tuned model: {fine_tuned_model}")
# Run inference with Responses API
response = client.completions.create(
model=fine_tuned_model,
prompt="What is the capital of France?",
max_tokens=100,
temperature=0.7
)
print(f"Response: {response.choices[0].text}")
else:
print(f"Job status: {job_details.status}")
print("Job must be in 'succeeded' status to run inference")
```
------
#### [ HTTP request ]
Make a POST request to `/v1/completions`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "ft:gpt-4o-mini:openai:custom-model:7p4lURel",
"prompt": "What is the capital of France?",
"max_tokens": 100,
"temperature": 0.7
}'
```
------
### Chat Completions API
Use the Chat Completions API for conversational interactions with your fine-tuned model:
------
#### [ OpenAI SDK (Python) ]
```
# Requires OPENAI_API_KEY and OPENAI_BASE_URL environment variables
from openai import OpenAI
client = OpenAI()
# Get the fine-tuned model ID
job_details = client.fine_tuning.jobs.retrieve("ftjob-abc123")
if job_details.status == 'succeeded' and job_details.fine_tuned_model:
fine_tuned_model = job_details.fine_tuned_model
print(f"Using fine-tuned model: {fine_tuned_model}")
# Run inference
inference_response = client.chat.completions.create(
model=fine_tuned_model,
messages=[
{"role": "user", "content": "What is the capital of France?"}
],
max_tokens=100
)
print(f"Response: {inference_response.choices[0].message.content}")
else:
print(f"Job status: {job_details.status}")
print("Job must be in 'succeeded' status to run inference")
```
------
#### [ HTTP request ]
Make a POST request to `/v1/chat/completions`:
```
curl https://bedrock-mantle.us-west-2.api.aws/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "ft:gpt-4o-mini:openai:custom-model:7p4lURel",
"messages": [
{"role": "user", "content": "What is the capital of France?"}
],
"max_tokens": 100
}'
```
------