AWS SRA for AI - AWS Prescriptive Guidance
Amazon BedrockAmazon Bedrock GuardrailsAmazon Bedrock model evaluationAmazon Bedrock AgentCore

AWS SRA for AI

Influence the future of the AWS Security Reference Architecture (AWS SRA) by taking a short survey.

This section provides recommendations for securing AI workloads within the AWS SRA multi-account framework. It covers AWS Identity and Access Management (IAM) permissions, data protection, input/output validation, network isolation, logging, and monitoring for generative AI capabilities. This section also covers integrating these capabilities into traditional AWS workloads.

This guidance covers security for the following capabilities:

Most capability sections include the following information:

  • Rationale explains what the capability does and when to use it.

  • Security considerations describes risks that are specific to the capability.

  • Remediations reviews the AWS services and features that address the risks.

  • Recommended AWS services describes the services to build the capability securely.

All capabilities build on Capability 1 (foundation model inference) because they all invoke models. When you combine capabilities, apply security controls from each relevant section. For example, a customized model with Retrieval Augmented Generation (RAG) requires controls from capabilities 1, 2, and 3.

The following diagram shows the extension of the AWS SRA Workloads organizational unit (OU) architecture with a dedicated Generative AI OU. The Generative AI OU contains two accounts that separate concerns:

  • The Application account hosts your traditional AWS application, which provides specific business functionality.

  • The Generative AI account hosts Amazon Bedrock and associated AWS services. Your application in the Application account calls Amazon Bedrock capabilities in the Generative AI account through APIs.

This account separation provides two key benefits. First, it enforces security controls through OU-specific and account-specific service control policies. Second, it simplifies implementing least-privilege access by grouping services based on application type. The reference architecture also includes foundational accounts that apply to all application types: Org Management, Security Tooling, Log Archive, Network, and Shared Services. For more information about these accounts, see AWS Security Reference Architecture in the AWS SRA – core architecture guide.

AWS SRA architecture to support generative AI.
Design considerations

  • Merge the Application and Generative AI accounts if your architecture requires consolidating both services in one account, or if your generative AI usage spans your entire organization.

  • Separate Generative AI accounts by software development lifecycle (SDLC) environment (development, test, and production) or by model and user community:

    • Use separate OUs for each SDLC environment. This approach controls team access, isolates resources, tracks costs separately, and prevents development issues from affecting production.

    • Use AWS Identity and Access Management (IAM) roles in a single account for pre-trained models. Use separate accounts when user communities have different risk levels, or when customized models contain sensitive training data.

  • Use multiple accounts when different user communities have distinct risk profiles, when customized models contain sensitive training data, or when regulatory requirements mandate data isolation.

  • Use a single account when you only use pre-trained foundation models, when users have a consistent risk profile, or when IAM roles provide sufficient access control.

Note

This guide assumes a single generative AI account strategy with IAM roles.

Amazon Bedrock

Amazon Bedrock helps you build and scale generative AI applications with foundation models (FMs). As a fully managed service, it provides access to FMs from companies such as AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon through a single API. You can experiment with pre-trained models, customize them with your data, and deploy them into your applications without training models from scratch or managing specialized infrastructure.

Amazon Bedrock protects your data through the following security capabilities:

  • Data protection – Amazon Bedrock isolates your content by user, encrypts it at rest in your AWS Region, and encrypts it in transit using TLS 1.2 or higher.

    • Data privacy – Amazon Bedrock doesn't store or log your prompts and completions. AWS doesn't use your content to train models or share it with third parties.

    • Model customization – When you customize a foundation model, your changes use a private copy. Your data isn't shared with model providers or used to improve base models.

    • Abuse detection – Amazon Bedrock uses automated detection to identify potential violations of the AWS Responsible AI Policy.

  • Compliance – Amazon Bedrock meets these standards: International Organization for Standardization (ISO), System and Organization Controls (SOC), Federal Risk and Authorization Management Program (FedRAMP) Moderate, and Cloud Security Alliance (CSA) Security Trust Assurance and Risk (STAR) Level 2. Amazon Bedrock is Health Insurance Portability and Accountability Act (HIPAA) eligible and in compliance with the General Data Protection Regulation (GDPR).

For more information, see the AWS secure approach to generative AI.

Amazon Bedrock Guardrails

Amazon Bedrock Guardrails help you implement safeguards that align with your use cases and responsible AI policies. You configure filters to block harmful content, define restricted topics, and customize messages that users see when content is blocked.

Guardrails evaluate both user inputs and model outputs across multiple harmful categories (hate, insults, sexual, violence, misconduct, and prompt attacks). The system classifies each input and output into one of four confidence levels: none, low, medium, or high. You set filter strength for each category based on your risk tolerance. When you deploy a guardrail to production, you create a versioned instance and invoke it in your application through the Amazon Bedrock API. For more information about implementation steps, see the Guardrails section in the Amazon Bedrock documentation.

Security

By default, Amazon Bedrock encrypts guardrails with an AWS managed key in AWS Key Management Service (AWS KMS). We recommend that you use a customer managed key to encrypt your guardrails and prevent unauthorized modifications. Combine encryption with least-privilege IAM permissions to restrict who can view or modify guardrail configurations.

Amazon Bedrock model evaluation

Model evaluation helps you compare FM outputs and select the model that best fits your application requirements. Amazon Bedrock supports both automatic evaluation using prompt datasets and human evaluation with subject matter experts.

Automatic evaluation jobs assess model performance using either custom prompt datasets or built-in datasets. This approach provides quantitative metrics for comparing models at scale. For more information, see Creating an automatic model evaluation job and Use prompt datasets for model evaluation in the Amazon Bedrock documentation.

Human evaluation jobs incorporate input from employees or subject matter experts to assess model quality, relevance, and safety. You organize evaluators into work teams managed through Amazon SageMaker Ground Truth. Create and manage work teams during job setup in Amazon Bedrock, or use the Amazon Cognito or Amazon SageMaker Ground Truth consoles for ongoing management.

Security

We recommend that you run model evaluation in development environments, not production. For information about multi-account strategies, see the Organizing your AWS environment using multiple accounts whitepaper. Both evaluation types require IAM service roles with the following specific permissions:

  • Automatic evaluation jobs require permissions to access Amazon Simple Storage Service (Amazon S3) datasets and write results.

  • Human evaluation jobs require additional permissions for Amazon SageMaker Ground Truth integration.

  • Custom prompt datasets require cross-origin resource sharing (CORS) configuration on Amazon S3 buckets.

For more information, see Service role requirements for automatic model evaluation jobs and Service role requirements for human-based model evaluation jobs in the Amazon Bedrock documentation.

Amazon Bedrock creates a temporary copy of your evaluation data during the job and deletes it after completion. By default, Amazon Bedrock encrypts this data with an AWS managed key. We recommend that you use a customer managed key in AWS KMS for enhanced control over data access. For more information, see Data management and encryption in Amazon Bedrock evaluation job in the Amazon Bedrock documentation.

Amazon Bedrock AgentCore

Amazon Bedrock AgentCore provides infrastructure and controls for deploying AI agents securely at scale. AgentCore consists of the following modular services that work together or independently:

  • AgentCore Runtime – Serverless execution environment with session isolation

  • AgentCore Identity – Authentication and credential management for agents and tools

  • AgentCore Memory – Persistent storage for conversation history and context

  • AgentCore Gateway – Centralized API integration and tool access

  • AgentCore Policy – Policy-based governance for agent actions

  • AgentCore tools – Built-in sandboxed tools (Code Interpreter, Browser)

  • AgentCore Observability – Trace, debug, and monitor agent performance

  • AgentCore Evaluations – Assess agent quality through online and on-demand testing

AgentCore works with any agent framework (for example, CrewAI, LangGraph, LlamaIndex, and Strands Agents) and any foundation model (FM). AgentCore provides security capabilities including session isolation through dedicated micro virtual machines (microVMs), encrypted credential storage, namespace-based memory isolation, and comprehensive audit logging.

For detailed security guidance, see Capability 5.