# Llama 3.1 405B Instruct ## ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/models/meta.png) Meta — Llama 3.1 405B Instruct ## Model Details Llama 3.1 405B Instruct is Meta's largest open model with 405 billion parameters and a 128K context window, supporting tool use and multilingual tasks. For more information about model development and performance, see the [model/service card](https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_1/). + **Model launch date:** Jul 23, 2024 + **Model EOL date:** Legacy: July 7, 2026 + **End User License Agreements and Terms of Use:** [View](https://aws.amazon.com/legal/bedrock/third-party-models/) + **Model lifecycle:** Active + **Context window:** 128K tokens + **Max output tokens:** 4K + **Knowledge cutoff:** Dec 2023 | **Input Modalities** | **Output Modalities** | **[APIs supported](https://docs.aws.amazon.com/bedrock/latest/userguide/apis.html)** | **[Endpoints supported](https://docs.aws.amazon.com/bedrock/latest/userguide/endpoints.html)** | | --- | --- | --- | --- | | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Audio | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Embedding | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Responses | ![\[Yes\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-yes.png) bedrock-runtime | | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Image | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Image | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Chat Completions | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) bedrock-mantle | | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Speech | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Speech | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Invoke | | | ![\[Yes\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-yes.png) Text | ![\[Yes\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-yes.png) Text | ![\[Yes\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-yes.png) Converse | | | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Video | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) Video | | | ## Pricing For pricing, please refer to the [Amazon Bedrock Pricing](https://aws.amazon.com/bedrock/pricing/) page. ## Programmatic Access Use the following model IDs and endpoint URLs to access this model programmatically. For more information about the available APIs and endpoints, see [APIs supported](https://docs.aws.amazon.com/bedrock/latest/userguide/apis.html) and [Endpoints supported](https://docs.aws.amazon.com/bedrock/latest/userguide/endpoints.html). | **Endpoint** | **Model ID** | **In-Region endpoint URL** | **Geo inference ID** | **Global inference ID** | | --- | --- | --- | --- | --- | | bedrock-runtime | meta.llama3-1-405b-instruct-v1:0 | https://bedrock-runtime.\$1region\$1.amazonaws.com | us.meta.llama3-1-405b-instruct-v1:0 | Not supported | *For example, if region is us-east-1 (N. Virginia), then the bedrock-runtime endpoint URL will be "https://bedrock-runtime.us-east-1.amazonaws.com" and for bedrock-mantle will be "https://bedrock-mantle.us-east-1.api.aws/v1".* ## Service Tiers Amazon Bedrock offers multiple service tiers to match your workload requirements. **Standard** provides pay-per-token access with no commitment. **Priority** offers higher throughput with a time-based commitment. **Flex** provides lower-cost access for flexible, non-time-sensitive workloads. **Reserved** provides dedicated throughput with a term commitment for predictable workloads. For more information, see [service tiers](https://docs.aws.amazon.com/bedrock/latest/userguide/service-tiers-inference.html). | **Standard** | **Priority** | **Flex** | **Reserved** | | --- | --- | --- | --- | | ![\[Yes\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-yes.png) | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) | ## Regional Availability ***Regional availability at a glance*** Bedrock offers three inference options: **In-Region** keeps requests within a single Region for strict compliance, **Geo Cross-Region** routes across Regions within a geography (US, EU, etc.) for higher throughput while respecting data residency, and **Global Cross-Region** routes anywhere worldwide for maximum throughput when there are no residency constraints. Refer to the [Regional availability](models-region-compatibility.md) page for more details. | **Region** | **In-Region** | **Geo** | **Global** | | --- | --- | --- | --- | | us-east-2 (Ohio) | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) | ![\[Yes\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-yes.png) | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) | | us-west-2 (Oregon) | ![\[Yes\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-yes.png) | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) | ![\[No\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/icons/icon-no.png) | ***Geo inference details*** **Geo: US** Geo Inference ID: `us.meta.llama3-1-405b-instruct-v1:0` | **Source Region** | **Destination Regions** | | --- | --- | | us-east-2 (Ohio) | us-east-1 (N. Virginia), us-east-2 (Ohio), us-west-2 (Oregon) | ## Sample Code **Step 1 - AWS Account:** If you have an AWS account already, skip this step. If you are new to AWS, sign up for an [AWS account](https://portal.aws.amazon.com/billing/signup). **Step 2 - API key:** Go to the [Amazon Bedrock console](https://console.aws.amazon.com/bedrock/home#/api-keys/long-term/create) and generate a long-term API key. **Step 3 - Get the SDK:** To use this getting started guide, you must have Python already installed. Then install the relevant software depending on the APIs you are using. ``` pip install boto3 ``` **Step 4 - Set environment variables:** Configure your environment to use the API key for authentication. ``` AWS_BEARER_TOKEN_BEDROCK="" ``` **Step 5 - Run your first inference request:** Save the file as `bedrock-first-request.py` ------ #### [ Converse API ] ``` import boto3 client = boto3.client('bedrock-runtime', region_name='us-east-1') response = client.converse( modelId='meta.llama3-1-405b-instruct-v1:0', messages=[ { 'role': 'user', 'content': [{'text': 'Can you explain the features of Amazon Bedrock?'}] } ] ) print(response) ``` ------