# Architecture details
This section describes the components and AWS services that make up this solution and the architecture details on [how these components work together](how-aws-solution-for-prebid-server-works.md).
## AWS services in this solution
| AWS service | Description |
| --- | --- |
| [Amazon CloudFront](https://aws.amazon.com/cloudfront/) | **Core**. Serve client requests to Prebid Server application. |
| [AWS DataSync](https://aws.amazon.com/datasync/) | **Core**. Automate transfer of Prebid Server application logs and metrics from Amazon EFS to Amazon S3. |
| [Amazon ECS](https://aws.amazon.com/ecs/) | **Core**. Host and manage containerized Prebid Server application. |
| [Amazon EFS](https://aws.amazon.com/efs/) | **Core**. Centralize storage of Prebid Server application logs and metrics across containers. |
| [Amazon ElastiCache](https://aws.amazon.com/elasticache/) | **Core**. Provide serverless Redis-compatible cache (Valkey) for storing cached bid responses with configurable time-to-live (TTL). |
| [Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) | **Core**. Provide high availability and automate scaling of Prebid Server application containers hosted on Amazon ECS. Route cache requests to Lambda function. |
| [Amazon EventBridge](https://aws.amazon.com/eventbridge/) | **Core**. Send and receive messages between solution resources handling Prebid Server application metrics and logs. |
| [AWS Glue](https://aws.amazon.com/glue/) | **Core.** Transform, catalog, and partition metrics data into Amazon S3 and [AWS Glue Data Catalog](https://docs.aws.amazon.com/glue/latest/dg/catalog-and-crawler.html). |
| [AWS Identity and Access Management (IAM)](https://aws.amazon.com/iam/) | **Core**. Restricts solution resource permissions to least privilege access for security. |
| [AWS KMS](https://aws.amazon.com/kms/) | **Core**. Encrypt and decrypt the data in Amazon S3. |
| [AWS Lambda](https://aws.amazon.com/lambda/) | **Core**. Facilitate deployment and deletion of the solution through [Lambda-backed custom resources](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-lambda.html), cleaning archived log and metrics files from Amazon EFS after being moved to Amazon S3 for long term storage, triggering AWS Glue, and handling cache storage and retrieval operations. |
| [Amazon S3](https://aws.amazon.com/s3/) | **Core**. Provide long term storage of Prebid Server application logs and metrics from Amazon EFS. |
| [AWS Systems Manager](https://aws.amazon.com/systems-manager/) | **Core**. Provide application-level resource monitoring and visualization of resource operations and cost data. |
| [Amazon VPC](https://aws.amazon.com/vpc/) | **Core**. Control network permissions between solution resources. |
| [AWS WAF](https://aws.amazon.com/waf/) | **Core**. Provide layer of security around Amazon CloudFront. |
| [AWS CloudTrail](https://aws.amazon.com/cloudtrail/) | **Supporting**. Track activity across solution S3 buckets and Lambda functions. |
| [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/) | **Supporting**. View logs and subscribe to alarms for AWS Lambda and AWS Glue. |
| [Amazon Athena](https://aws.amazon.com/athena/) | **Optional**. Access AWS Glue Data Catalog and query the Prebid Server application metrics in Amazon S3. |
| [AWS RTB Fabric](https://aws.amazon.com/rtb-fabric/) | **Optional**. Provide low-latency, cost-optimized private network connectivity between Prebid Server and bidders without traversing the public internet. |
## CloudFront distribution
The solution uses Amazon CloudFront as the unified network entry point. It receives the incoming auction requests and handles outgoing responses. CloudFront speeds up the distribution of your content by routing each user request through the AWS backbone network to the edge location that can best serve your content. CloudFront provides a TLS endpoint for privacy of requests and responses in transit with the pubic internet. ALB is the configured origin for CloudFront. Direct access to ALB is restricted by using a custom header, enhancing security.
## AWS WAF
AWS Web Application Firewall (AWS WAF) and AWS Shield Standard are used as a protection mechanism from Distributed Denial of Service (DDoS) attacks against the Prebid Server cluster. AWS WAF can activate one or more managed rule groups by default after extended testing including rules in the Baseline Rule Group and the IP Reputation Rule Group. You have the option to activate, purchase, or use existing rule subscriptions, or add regular expression or CIDR matching rules as needed.
**Note**
If you want to opt out of using CloudFront and AWS WAF and directly send requests to the ALB, see [How to opt out](configure-the-solution.md#how-to-opt-out).
## Application Load Balancer (ALB)
ALB distributes incoming request traffic for Prebid Server through the cluster of containers. It provides a single entry point into the cluster and is the primary origin for the CloudFront distribution. ALB also routes cache-related requests (paths starting with `/cache`) to the cache Lambda function, which handles storage and retrieval of cached bid responses in ElastiCache.
## Amazon VPC
The Amazon Virtual Private Cloud (Amazon VPC) is configured with redundant subnets, routes, and NAT gateways. Security groups permit traffic to and from the subnets. The Amazon VPC contains the network interfaces for the Prebid Server container cluster nodes. It is configured for private IP addresses only and container networks configured within the Amazon VPC use the NAT gateway as a default route to the internet for communication.
When the bidder simulator is deployed, the solution supports two connectivity modes:
**VPC Peering (default)**
When deploying with the bidder simulator, the solution automatically creates a VPC peering connection between the Prebid Server VPC and the Bidder Simulator VPC. This provides direct, private connectivity between the two environments without traversing the public internet. The peering connection is automatically configured with appropriate routes and security group rules.
**AWS RTB Fabric (optional)**
When deploying with both the bidder simulator and RTB Fabric, the solution creates a private network connection through AWS RTB Fabric instead of VPC peering. This provides purpose-built, low-latency connectivity optimized for real-time bidding traffic. See the [AWS RTB Fabric integration](#aws-rtb-fabric-integration) section for details.
## Amazon ECS
Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service that helps you easily deploy, manage, and scale containerized Prebid Server application. These resources define the configuration, count, and thresholds to scale-out and scale-in the total container count in the ECS cluster. The ECS task and service resource define the operating environment for the cluster and thresholds for scaling and health. Scaling changes are based on CPU, process load, and network traffic (requests per target). For cost optimization, ECS uses a weighted combination of Fargate and [Fargate Spot](https://docs.aws.amazon.com/AmazonECS/latest/bestpracticesguide/ec2-and-fargate-spot.html) instances. There’s a cost benefit to using more Fargate Spot instances, but the risk of unavailability goes up. You might find that after running the solution for a while that a different ratio is better for you.
## AWS RTB Fabric integration (optional)
[AWS RTB Fabric](https://aws.amazon.com/rtb-fabric/) is a private network purpose-built for real-time bidding that provides low-latency, cost-optimized connectivity between ad tech participants without traversing the public internet. When deployed, the solution creates the following components:
**Requester Gateway**
Deployed in the Prebid Server VPC, the requester gateway sends bid requests over HTTPS (port 443) through the RTB Fabric private network. Prebid Server is configured to route bid requests to the RTB Fabric link URL instead of directly to bidder endpoints.
**Responder Gateway**
Deployed in the Bidder Simulator VPC, the responder gateway receives bid requests over HTTP (port 80) and forwards them to the bidder simulator Application Load Balancer. The connection uses asymmetric security—HTTPS from requester to responder, with HTTP responses on the internal AWS network.
**Fabric Link**
The Fabric Link connects the requester and responder gateways through AWS RTB Fabric’s private network. A Lambda function automatically accepts the Fabric Link during deployment. The link provides dedicated bandwidth and low-latency routing optimized for real-time bidding traffic patterns.
**Note**
RTB Fabric requires the bidder simulator to be deployed as it needs both requester and responder gateways. RTB Fabric must also be available in your deployment region.
## Bidder Simulator (optional)
The bidder simulator is an optional component that provides a quick start testing environment to validate your Prebid Server deployment without needing to configure external bidders. The simulator includes:
**Architecture**
The bidder simulator uses a CloudFront \$1 Application Load Balancer \$1 Lambda architecture to simulate bidder responses. It supports both banner and video ad formats, including VAST instream video.
**Bidder Simulator Adapter Integration**
When deployed, the solution automatically configures the custom prebid bidder adapter in Prebid Server allowing connectivity to the Bidder Simulator. The adapter files are conditionally included in the Docker build, and environment variables are automatically set for proper integration.
**Demo Website**
A demo website with Prebid.js integration is included to test the end-to-end flow from prebid.js through Prebid Server to the bidder simulator. The demo supports both banner and video ad units. For usage instructions, see the demo website readme at `source/loadtest/demo/README.md`.
**Connectivity Options**
The bidder simulator supports two connectivity modes to Prebid Server:
+ **VPC Peering** (default): Direct private connectivity through VPC peering connection
+ **RTB Fabric**: Private network connectivity through AWS RTB Fabric
**Note**
The bidder simulator is intended for testing and validation purposes. For production deployments, configure Prebid Server to connect to your actual bidder endpoints.
## Cache architecture
The solution includes a cache service that stores and retrieves bid responses for Prebid Server. The cache architecture uses the following components:
**ElastiCache Serverless (Valkey)**
The solution uses Amazon ElastiCache Serverless with Valkey (Redis-compatible) engine to provide a fully managed, serverless cache. The cache is deployed in the private subnets of the VPC and uses IAM authentication for secure access. Cache data is stored with configurable time-to-live (TTL) settings.
**Cache Lambda Function**
An AWS Lambda function handles cache storage and retrieval operations. The function is invoked through ALB target group rules that route requests with paths starting with `/cache` to the Lambda function. The Lambda function connects to the ElastiCache Serverless cache using IAM authentication and the Redis protocol.
**Unified Cache Endpoint**
Both Prebid Server containers and client-side code (prebid.js) use the same publicly accessible cache endpoint:
+ **CloudFront deployment**: Cache endpoint is the CloudFront distribution domain
+ **ALB-only deployment**: Cache endpoint is the external ALB DNS name
This unified approach ensures consistent cache access patterns and simplifies the architecture by eliminating the need for separate internal and external cache endpoints.
**Cache Request Flow**
When Prebid Server needs to cache a bid response:
1. Container sends cache request to the configured `CACHE_HOST` (CloudFront domain or external ALB DNS)
1. Request flows through CloudFront (if deployed) to the external ALB
1. ALB routes the request to the cache Lambda function based on path rules
1. Lambda function stores the bid response in ElastiCache Serverless and returns a cache key
1. Client-side code later retrieves the cached response using the same endpoint
**Note**
The cache service is designed to handle both server-side caching (from Prebid Server containers) and client-side retrieval (from browsers). Using a single public endpoint for both access patterns ensures optimal performance and simplifies configuration.
## Prebid Server container
This is a docker container that runs the open source Prebid Server and is hosted in [Amazon Elastic Container Registry](https://aws.amazon.com/ecr/) (Amazon ECR). The container differs from the open source project’s default container in configuration settings for areas including log output to the Console and bidding adapter configuration settings.
## Amazon EFS
The EFS file system is mounted and shared among all container instances in the ECS cluster. This file system is used for log capture (operational and metrics), and has the potential to be expanded to include shared configuration and storage related to more advertisement types (for example, video and mobile).
## DataSync (EFS to S3)
DataSync is configurated to periodically move rotated log files from each Prebid Server container’s EFS location to an equivalent location in the `DataSyncLogsBucket` S3 bucket. After each file is copied to S3 and verified, it is removed from the EFS file system through a clean-up Lambda function. Essentially, only actively written log files are retained on the EFS file system until the Prebid Server process closes it, rotates it, and starts a new file. Rotated log files are migrated with DataSync. Runtime logs are rotated every 24 hours or when reaching 100 MB. Metrics logs are rotated every one hour or when reaching 100 MB.
## Glue ETL (Metrics processing)
AWS Glue is a serverless data integration service that makes it easy for analytics users to discover, prepare, move, and integrate data from multiple sources. You can use it for analytics, machine learning, and application development. It also includes additional productivity and data ops tooling for authoring, running jobs, and implementing business workflows. This resource is responsible for periodically processing new metrics log files in the `DataSyncLogsBucket` S3 bucket. The CSV-formatted metrics are transformed into several tables and partitioned. After ETL processing completes, the new data is available to clients through AWS Glue Data Catalog.
## AWS Glue Data Catalog
AWS AWS Glue Data Catalog provides access for clients to the Prebid Server metric data through Athena or other compatible clients, such as Amazon SageMaker AI, Amazon QuickSight, and JDBC clients. Clients can query and view the Prebid Server metrics data, generate graphs, summaries or inferences using AI/ML.
## Amazon CloudWatch
CloudWatch alarms monitor specific metrics in real-time and proactively notify AWS Management Console users when predefined conditions are met. This solution has several CloudWatch alarms to help monitor its health and performance. These alarms are enabled automatically when the AWS CDK stack is deployed. For details, see the [CloudWatch Alarms](traffic-monitoring.md#amazon-cloudwatch-alarms) section.
**Note**
All resources are created in a single Region specified by the user except for CloudFront and AWS WAF. CloudFront is considered a global resource, and AWS WAF is always created in the `us-east-1` (N.Virginia) Region for configuration with CloudFront.
# How Guidance for Deploying a Prebid Server on AWS works
## Workflow of banner ads
From [docs.prebid.org](https://docs.prebid.org/prebid-server/use-cases/pbs-pbjs.html).
**Workflow diagram of banner ads**
![\[workflow\]](http://docs.aws.amazon.com/solutions/latest/prebid-server-deployment-on-aws/images/workflow.png)
1. `Prebid.js` is set up to run auctions for one or more bidders through `s2sConfig`.
1. Prebid Server parses the request and holds the auction.
1. The response, including the body of the winning creative(s), is sent back to the browser.
1. `Prebid.js` passes ad server targeting variables to the page, which forwards it to the ad server.
1. When a header bidding ad wins, the ad server responds to the page with the [Prebid Universal Creative](https://github.com/prebid/prebid-universal-creative), which calls the render function in Prebid.js.
1. The render function displays the creative.
# Amazon CloudFront, AWS WAF, and ALB
CloudFront is used as the entry point into the solution where the bid requests are received. CloudFront speeds up distribution of content through a worldwide network of data centers called edge locations. CloudFront ensures that end-user requests are served by the closest edge location. To prevent requests from bypassing the CDN and accessing the origin (ALB) directly, the solution uses [CloudFront custom headers](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/add-origin-custom-headers.html).
AWS WAF is a web application firewall that helps protect the application from common web exploits that could affect application availability, compromise security, or consume excessive resources. It is preconfigured with a set of standard [AWS managed rules](https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups.html). AWS WAF is tightly integrated with CloudFront at the edge. Traffic can be received here as HTTP or HTTPS.
**Note**
If you want to opt out of using CloudFront and AWS WAF and directly send requests to the ALB, see [How to opt out](configure-the-solution.md#how-to-opt-out).
# Prebid Server container
This container hosted in Amazon ECR runs the open source Prebid Server. The solution’s default container image is a customized build of the Prebid Server implementation in Java. The Dockerfile for building this container image is located at `deployment/ecr/prebid-server/Dockerfile` and an accompanying configuration file is at `deployment/ecr/prebid-server/config.json`. This configuration file provides the version of the prebid server through the `GIT_TAG_VERSION` parameter.
The container differs from the open source project’s default container in configuration settings for areas including file output and bidding adapter configuration settings. Alternatively, you can use the [Go implementation of Prebid Server](https://docs.prebid.org/prebid-server/versions/pbs-versions-go.html), or any build hosted in a container repository, such as [Docker Hub](https://hub.docker.com/) or [ECR Public Gallery](https://gallery.ecr.aws/). You will have the option of overriding the solution’s default container.
**Note**
If using a custom image, ensure it is compatible with the logging configuration of the container supplied with the solution to make use of the Prebid Metrics ETL.
Make changes to the Dockerfile as needed.
The following diagram shows a detailed view of the container contents, and the resources the container interacts with directly. Additionally, the container image supplied with the solution is stored in the [ECR Public Gallery for AWS Solutions](https://gallery.ecr.aws/aws-solutions/).
**Prebid Server container diagram**
![\[prebid server container diagram\]](http://docs.aws.amazon.com/solutions/latest/prebid-server-deployment-on-aws/images/prebid-server-container-diagram.png)
1. Amazon ECR is a fully managed container registry for developers to store, manage, and deploy container images for public or private access.
1. Inbound auction requests come through Application Load Balancer. ALB distributes incoming application traffic across multiple ECS containers.
1. The Guidance for Deploying a Prebid Server on AWS container instances run in a defined environment.
1. The Java-based version of Prebid Server is configured to run with [Amazon Corretto](https://aws.amazon.com/corretto/).
1. NAT gateway enables instances in a private subnet to connect to the internet but prevents the internet from initiating a connection with those instances. It is the default gateway for outgoing network packets from the private subnet within Amazon VPC.
1. The internet gateway connects a VPC with the public internet, allowing instances such as NAT gateways to send and receive traffic.
1. Auction requests are sent to bidders over the internet.
1. EFS Standard is a managed file storage service that provides a scalable file system for use with AWS Cloud services and on-premises resources. The EFS filesystem is used for capturing transaction and operational log data from each container via an NFS network connection.
1. DataSync is used as the ETL mechanism for moving data captured in raw log files on EFS to a structured and normalized layout in AWS Glue Data Catalog.
1. The `DataSyncLogsBucket` S3 bucket receives the migrated log data from the EFS filesystem. The Glue ETL process uses the contents of this bucket as source data.
# ECS Fargate containers automatic scaling
Automatic scaling is the ability to increase or decrease the desired count of tasks in your Amazon ECS service automatically. The service is configured to use target tracking scaling policies, which increase or decrease the number of tasks that your service runs. These are based on a target value for a specific metric provided by Application Auto Scaling, such as average CPU utilization, memory utilization, and average request count per target. These values are set in the `source/infrastructure/prebid_server/stack_constants.py` file with default values of `CPU_TARGET_UTILIZATION_PCT = 66`, `MEMORY_TARGET_UTILIZATION_PCT = 50`, and `REQUESTS_PER_TARGET = 5000`.
The default minimum and maximum task counts are also set in the file as `TASK_MIN_CAPACITY = 2` and `TASK_COUNT_MAX = 300`, respectively.
# AWS Fargate Spot compared to reserved instances
With Amazon ECS on AWS Fargate capacity providers, you can use [both Fargate and Fargate Spot capacity](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/fargate-capacity-providers.html) with your Amazon ECS tasks. With Fargate Spot, you can run interruption tolerant Amazon ECS tasks at a rate that’s discounted compared to the Fargate price. When AWS needs the capacity back, tasks are interrupted with a 2-minute warning. When the Spot instances are terminated, ECS requests new Spot capacity, possibly from a different AZ to replace the reclaimed instances.
The default weights used for Fargate compared to Fargate Spot instances are defined through parameters in the solution source code `FARGATE_RESERVED_WEIGHT = 1` and `FARGATE_SPOT_WEIGHT = 1` respectively.
# AWS Fargate container health check
Health check parameters that are specified in a container definition override any Docker health checks that exist in the container image. The following endpoint is used to check the health of the Prebid Server running containers: `HEALTH_ENDPOINT = "/status"`. The default interval and timeout parameters `HEALTH_CHECK_INTERVAL_SECS = 60` and `HEALTH_CHECK_TIMEOUT_SECS = 5` are used to set the time period in seconds between each health check run, and the time period in seconds to wait for a health check to succeed before it is considered a failure, respectively.
# Prebid Server metrics
The Prebid Server collects various application metrics, which are submitted to configured backends. These metrics can be used for monitoring and optimizing the performance of the server. The metrics include information such as bid requests, bid responses, and other relevant statistics. For more detailed information on the specific metrics collected and how to integrate custom analytics adapters, refer to the official [Prebid Server documentation](https://docs.prebid.org/prebid-server/developers/pbs-build-an-analytics-adapter.html#prebid-server---building-an-analytics-adapter) and the [metrics.md](https://github.com/prebid/prebid-server-java/blob/master/docs/metrics.md) file in the GitHub repository for the Prebid Server Java implementation.
In this solution, metrics that are emitted by the Prebid Server containers are captured via log files. This is achieved by hooking the operational monitoring system to the Java file logger by modifying the original source file on the GitHub repository - [MetricsConfiguration.java](https://github.com/prebid/prebid-server-java/blob/master/src/main/java/org/prebid/server/spring/config/metrics/MetricsConfiguration.java).
This data is processed through the same ETL pipeline as operational metrics, making it available for analysis through AWS Glue Data Catalog and Amazon Athena. For configuration details, see the [Configuring the analytics adapter](configure-the-solution.md#configuring-analytics-adapter) section.
# Operational and metrics Logs
Amazon EFS is mounted to the ECS cluster using NFS protocol and is shared among all container instances. Prebid Server writes two log files while running - one for `stdout/stderr` operational logging and one for auction metrics. They are both located on the EFS with a unique path based on the container instance to prevent collision. The operational log files generated by the Prebid Service container are located at `/mnt/efs/logs//prebid-server.log`. These files are rotated periodically, every day at midnight and placed in the archived folder located at `/mnt/efs/logs//archived/prebid-server.%d{yyyy-MM-dd}.%i.log.gz`.
If the generated log file exceeds a size limit of 100 MB, this rotation happens earlier. In a similar fashion, the metrics logs are located at `/mnt/efs/metrics//prebid-metrics.log` and archived at `/mnt/efs/metrics//archived/prebid-metrics.%d{yyyy-MM-dd_HH}.%i.log.gz`. These files are rotated at the top of each hour (period of one hour). The logging configurations are available through settings in the `deployment/ecr/prebid-server/prebid-logging.xml` file in the Prebid Server source code.
**Note**
If a container task receives a termination signal from ECS, for reasons such as scale-in or Spot interruptions, then the solution automatically compresses and transfers the current active log files to their respective archived folders to ensure no discontinuity or loss of log data. The transferred logs will be picked up along with any other log files in the `archived` directory when the DataSync task runs.
# Transfer of log files
DataSync is configured to periodically (every hour on the half hour mark) to move the rotated and archived metric log files from EFS to an equivalent location in S3. For details, see the [Logging](logging.md) section. This period is set through Cron schedule by `DATASYNC_METRICS_SCHEDULE = "cron(30 * * * ? *)"` and `DATASYNC_LOGS_SCHEDULE = "cron(30 * * * ? *)"`. These parameters are available through the `source/infrastructure/prebid_server/stack_constants.py` file in the solution [source code](https://github.com/aws-solutions-library-samples/prebid-server-deployment-on-aws).
After each file is copied to S3 and verified, it is removed from the EFS file system through a Lambda function. Only actively written log files are retained on the EFS.
# Metrics processing
An AWS Glue job is configured to perform ETL operation on the metrics log data present in the S3 bucket containing the raw logs. The ETL operation structures the metric data present in the log files into a single database with several tables. For details, see [Glue table schemas](glue-table-schemas.md) in the Querying metrics with Athena section. The processed data is written into the `MetricsEtl` S3 bucket, which contains the metric log data transformed and partitioned through ETL in the previous steps. The data is made available via AWS AWS Glue Data Catalog, a persistent technical metadata store for analytics users to discover and prepare the data. Also, you can immediately search and query cataloged data using Amazon Athena, [Amazon EMR](https://aws.amazon.com/emr/), and [Amazon Redshift Spectrum](https://docs.aws.amazon.com/redshift/latest/dg/c-using-spectrum.html).
# Logging
The log files generated by this solution are stored in four separate buckets depending on their context. Logs containing operational data, such as CloudFront access logs, are stored in one bucket, while logs containing business intelligence, such as Prebid Server performance metrics, are stored in another bucket. This organization is intended to facilitate data pipelines for context-specific log analysis.
The following S3 buckets are used for log storage.
**CloudFront access logs bucket**
Contains detailed information about every request received by the CloudFront distribution.
**ALB access logs bucket**
Contains detailed information about every request received by the Application Load Balancer (ALB). Elastic Load Balancing provides access logs that capture detailed information about requests sent to your load balancer. Each log contains information such as the time the request was received, the client’s IP address, latencies, request paths, and server responses. You can use these access logs to analyze traffic patterns and troubleshoot issues. For more information, see [Access logs for your Application Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html) in Elastic Load Balancing.
**DataSync logs bucket**
+ Contains `stdout` and `stderr` log streams from the Prebid Server processes running in the ECS cluster.
+ Each log stream is saved in a unique directory named according to its container identifier in ECS.
To learn more about these logs, see the [Prebid Server operation and metrics logs](#prebid-server-operation-and-metrics-logs) section.
**DataSync metrics bucket**
+ Contains metric streams for Prebid Server transactions running in the ECS cluster.
+ Each metric stream is saved in a unique directory named according to its container identifier in ECS.
To learn more about these logs, see the [Prebid Server operation and metrics logs](#prebid-server-operation-and-metrics-logs) section.
Prebid Server containers in Amazon ECS send log information to CloudWatch Logs. These logs can be found within the `AWS::Logs::LogGroup` resource shown in the CloudFormation stack. The log group includes output from the commands specified in the Dockerfile that is used to assemble Prebid Server containers. Since the Prebid Server process redirects both `stdout` and `stderr` to EFS, the CloudWatch logs will be empty unless ECS encountered a problem running the Prebid Server process.
## Prebid Server operation and metrics logs
This solution uses EFS to store logs from every Prebid Server task running in ECS. Each task writes runtime logs to `/mnt/efs/logs/ ` and metrics logs to `/mnt/efs/metrics/ `. The container identifier in the path ensures that each Prebid Server instance stores logs without interfering with the logs from other instances.
The logging policy for the prebid process is defined in the prebid-logging.xml file. It includes the following customizations:
+ Info level logging of `stdout`/`stderr` output.
+ Rolling policy to archive logs when they exceed 100 MB or at a predetermined frequency of once per hour.
+ Asynchronous logging to minimize the computational overhead of logging. For more information, see [Benchmarking synchronous and asynchronous logging](https://logback.qos.ch/performance.html).
The log files are archived periodically (every 60 minutes) and moved to the corresponding archive folders `/mnt/efs/logs//archived` and `/mnt/efs/metrics//archived`.
DataSync moves the archived log files from EFS to S3 every 60 minutes. Runtime logs are saved to `s3://-datasynclogsbucket-//archived`. Metrics logs are saved to `s3://-datasyncmetricsbucket-//archived`.
If you need to access the Prebid Server logs before they are copied by AWS DataSync, then mount the EFS filesystem directly using the procedure described in [Accessing Prebid Server logs from EFS](accessing-prebid-server-logs-from-efs.md).