AGENTREL01-BP05 Implement adaptive provisioning
Static provisioning forces a choice between overpaying for peak and failing under load. Agent workloads shift minute to minute, so capacity, model tier, and quota must respond to task complexity and current demand without operator intervention.
Desired outcome:
-
You have agent compute allocation that adjusts for each invocation without manual tuning.
-
You route tasks to model tiers appropriate for their complexity, with fallbacks when quotas tighten.
-
You pre-provision resources ahead of known demand patterns and scale down during quiet periods.
Common anti-patterns:
-
Running static resource provisioning and paying for peak capacity even during low-demand periods.
-
Skipping the metrics that trigger scaling decisions, so the system has no basis to provision resources when they are needed.
-
Treating every task as needing the same model, ignoring the cost and latency savings of tiering by complexity.
Benefits of establishing this best practice:
-
Performance stays consistent under variable load because resources track demand instead of a fixed provisioning plan.
-
Resource exhaustion during spikes is prevented without human intervention.
-
Cost drops during low-demand periods through automatic scale-down.
Level of risk exposed if this best practice is not established: Medium
Implementation guidance
Serverless is the baseline for adaptive compute. Amazon Bedrock AgentCore Runtime hosts agents with built-in scaling that adjusts compute allocation for each invocation, so individual agents don't need to manage fleet sizing. For LLM inference, Amazon Bedrock's on-demand mode scales without capacity reservations. Amazon Bedrock cross-region inference distributes requests across Regions to reduce the impact of regional capacity constraints.
Tiering matches model selection to workload. Low-complexity tasks route to smaller, faster Amazon Bedrock models, while complex reasoning routes to larger models. The router should adjust dynamically based on task complexity signals and current quota utilization, not a fixed rule baked into code. For latency-sensitive user-facing agents where throttling is unacceptable, use Amazon Bedrock's Priority on-demand tier for premium throughput allocation. For workloads that need consistent low latency regardless of overall service demand, use Amazon Bedrock Provisioned Throughput with fixed model units. The Amazon Bedrock Capacity, Limits, and Cost Optimization guide covers the trade-offs between Flex, Standard, Priority, and Reserved tiers.
Monitor composite health signals across agent layers and trigger coordinated scaling actions when the system approaches capacity limits. Token throughput, model-level latency, and error rates for each layer through Amazon Bedrock AgentCore Observability are the signals that drive tier adjustments. Scheduled scaling handles anticipated demand. If historical data shows a spike every Monday at 9 a.m., pre-provision before the spike lands rather than reacting during it.
Implementation steps
-
Deploy agents on AgentCore Runtime for serverless scaling: Use Amazon Bedrock AgentCore Runtime so compute allocation adjusts for each invocation without manual fleet sizing.
-
Route tasks by complexity to appropriate Amazon Bedrock models: Implement tiered model selection that sends low-complexity tasks to smaller models and reasoning-heavy tasks to larger ones based on complexity signals.
-
Enable Amazon Bedrock cross-region inference: Turn on cross-region inference to distribute requests and reduce the impact of regional capacity constraints.
-
Monitor token throughput and latency through AgentCore Observability: Watch per-tier throughput and latency through Amazon Bedrock AgentCore Observability and trigger tier adjustments when thresholds are exceeded.
-
Use scheduled scaling ahead of anticipated spikes: Pre-provision based on historical patterns so capacity is ready before demand lands.
Resources
Related best practices:
Related documents:
Related videos:
Related services:
