# How Guidance for Deploying a Prebid Server on AWS works
## Workflow of banner ads
From [docs.prebid.org](https://docs.prebid.org/prebid-server/use-cases/pbs-pbjs.html).
**Workflow diagram of banner ads**
![\[workflow\]](http://docs.aws.amazon.com/solutions/latest/prebid-server-deployment-on-aws/images/workflow.png)
1. `Prebid.js` is set up to run auctions for one or more bidders through `s2sConfig`.
1. Prebid Server parses the request and holds the auction.
1. The response, including the body of the winning creative(s), is sent back to the browser.
1. `Prebid.js` passes ad server targeting variables to the page, which forwards it to the ad server.
1. When a header bidding ad wins, the ad server responds to the page with the [Prebid Universal Creative](https://github.com/prebid/prebid-universal-creative), which calls the render function in Prebid.js.
1. The render function displays the creative.
# Amazon CloudFront, AWS WAF, and ALB
CloudFront is used as the entry point into the solution where the bid requests are received. CloudFront speeds up distribution of content through a worldwide network of data centers called edge locations. CloudFront ensures that end-user requests are served by the closest edge location. To prevent requests from bypassing the CDN and accessing the origin (ALB) directly, the solution uses [CloudFront custom headers](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/add-origin-custom-headers.html).
AWS WAF is a web application firewall that helps protect the application from common web exploits that could affect application availability, compromise security, or consume excessive resources. It is preconfigured with a set of standard [AWS managed rules](https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups.html). AWS WAF is tightly integrated with CloudFront at the edge. Traffic can be received here as HTTP or HTTPS.
**Note**
If you want to opt out of using CloudFront and AWS WAF and directly send requests to the ALB, see [How to opt out](configure-the-solution.md#how-to-opt-out).
# Prebid Server container
This container hosted in Amazon ECR runs the open source Prebid Server. The solution’s default container image is a customized build of the Prebid Server implementation in Java. The Dockerfile for building this container image is located at `deployment/ecr/prebid-server/Dockerfile` and an accompanying configuration file is at `deployment/ecr/prebid-server/config.json`. This configuration file provides the version of the prebid server through the `GIT_TAG_VERSION` parameter.
The container differs from the open source project’s default container in configuration settings for areas including file output and bidding adapter configuration settings. Alternatively, you can use the [Go implementation of Prebid Server](https://docs.prebid.org/prebid-server/versions/pbs-versions-go.html), or any build hosted in a container repository, such as [Docker Hub](https://hub.docker.com/) or [ECR Public Gallery](https://gallery.ecr.aws/). You will have the option of overriding the solution’s default container.
**Note**
If using a custom image, ensure it is compatible with the logging configuration of the container supplied with the solution to make use of the Prebid Metrics ETL.
Make changes to the Dockerfile as needed.
The following diagram shows a detailed view of the container contents, and the resources the container interacts with directly. Additionally, the container image supplied with the solution is stored in the [ECR Public Gallery for AWS Solutions](https://gallery.ecr.aws/aws-solutions/).
**Prebid Server container diagram**
![\[prebid server container diagram\]](http://docs.aws.amazon.com/solutions/latest/prebid-server-deployment-on-aws/images/prebid-server-container-diagram.png)
1. Amazon ECR is a fully managed container registry for developers to store, manage, and deploy container images for public or private access.
1. Inbound auction requests come through Application Load Balancer. ALB distributes incoming application traffic across multiple ECS containers.
1. The Guidance for Deploying a Prebid Server on AWS container instances run in a defined environment.
1. The Java-based version of Prebid Server is configured to run with [Amazon Corretto](https://aws.amazon.com/corretto/).
1. NAT gateway enables instances in a private subnet to connect to the internet but prevents the internet from initiating a connection with those instances. It is the default gateway for outgoing network packets from the private subnet within Amazon VPC.
1. The internet gateway connects a VPC with the public internet, allowing instances such as NAT gateways to send and receive traffic.
1. Auction requests are sent to bidders over the internet.
1. EFS Standard is a managed file storage service that provides a scalable file system for use with AWS Cloud services and on-premises resources. The EFS filesystem is used for capturing transaction and operational log data from each container via an NFS network connection.
1. DataSync is used as the ETL mechanism for moving data captured in raw log files on EFS to a structured and normalized layout in AWS Glue Data Catalog.
1. The `DataSyncLogsBucket` S3 bucket receives the migrated log data from the EFS filesystem. The Glue ETL process uses the contents of this bucket as source data.
# ECS Fargate containers automatic scaling
Automatic scaling is the ability to increase or decrease the desired count of tasks in your Amazon ECS service automatically. The service is configured to use target tracking scaling policies, which increase or decrease the number of tasks that your service runs. These are based on a target value for a specific metric provided by Application Auto Scaling, such as average CPU utilization, memory utilization, and average request count per target. These values are set in the `source/infrastructure/prebid_server/stack_constants.py` file with default values of `CPU_TARGET_UTILIZATION_PCT = 66`, `MEMORY_TARGET_UTILIZATION_PCT = 50`, and `REQUESTS_PER_TARGET = 5000`.
The default minimum and maximum task counts are also set in the file as `TASK_MIN_CAPACITY = 2` and `TASK_COUNT_MAX = 300`, respectively.
# AWS Fargate Spot compared to reserved instances
With Amazon ECS on AWS Fargate capacity providers, you can use [both Fargate and Fargate Spot capacity](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/fargate-capacity-providers.html) with your Amazon ECS tasks. With Fargate Spot, you can run interruption tolerant Amazon ECS tasks at a rate that’s discounted compared to the Fargate price. When AWS needs the capacity back, tasks are interrupted with a 2-minute warning. When the Spot instances are terminated, ECS requests new Spot capacity, possibly from a different AZ to replace the reclaimed instances.
The default weights used for Fargate compared to Fargate Spot instances are defined through parameters in the solution source code `FARGATE_RESERVED_WEIGHT = 1` and `FARGATE_SPOT_WEIGHT = 1` respectively.
# AWS Fargate container health check
Health check parameters that are specified in a container definition override any Docker health checks that exist in the container image. The following endpoint is used to check the health of the Prebid Server running containers: `HEALTH_ENDPOINT = "/status"`. The default interval and timeout parameters `HEALTH_CHECK_INTERVAL_SECS = 60` and `HEALTH_CHECK_TIMEOUT_SECS = 5` are used to set the time period in seconds between each health check run, and the time period in seconds to wait for a health check to succeed before it is considered a failure, respectively.
# Prebid Server metrics
The Prebid Server collects various application metrics, which are submitted to configured backends. These metrics can be used for monitoring and optimizing the performance of the server. The metrics include information such as bid requests, bid responses, and other relevant statistics. For more detailed information on the specific metrics collected and how to integrate custom analytics adapters, refer to the official [Prebid Server documentation](https://docs.prebid.org/prebid-server/developers/pbs-build-an-analytics-adapter.html#prebid-server---building-an-analytics-adapter) and the [metrics.md](https://github.com/prebid/prebid-server-java/blob/master/docs/metrics.md) file in the GitHub repository for the Prebid Server Java implementation.
In this solution, metrics that are emitted by the Prebid Server containers are captured via log files. This is achieved by hooking the operational monitoring system to the Java file logger by modifying the original source file on the GitHub repository - [MetricsConfiguration.java](https://github.com/prebid/prebid-server-java/blob/master/src/main/java/org/prebid/server/spring/config/metrics/MetricsConfiguration.java).
This data is processed through the same ETL pipeline as operational metrics, making it available for analysis through AWS Glue Data Catalog and Amazon Athena. For configuration details, see the [Configuring the analytics adapter](configure-the-solution.md#configuring-analytics-adapter) section.
# Operational and metrics Logs
Amazon EFS is mounted to the ECS cluster using NFS protocol and is shared among all container instances. Prebid Server writes two log files while running - one for `stdout/stderr` operational logging and one for auction metrics. They are both located on the EFS with a unique path based on the container instance to prevent collision. The operational log files generated by the Prebid Service container are located at `/mnt/efs/logs//prebid-server.log`. These files are rotated periodically, every day at midnight and placed in the archived folder located at `/mnt/efs/logs//archived/prebid-server.%d{yyyy-MM-dd}.%i.log.gz`.
If the generated log file exceeds a size limit of 100 MB, this rotation happens earlier. In a similar fashion, the metrics logs are located at `/mnt/efs/metrics//prebid-metrics.log` and archived at `/mnt/efs/metrics//archived/prebid-metrics.%d{yyyy-MM-dd_HH}.%i.log.gz`. These files are rotated at the top of each hour (period of one hour). The logging configurations are available through settings in the `deployment/ecr/prebid-server/prebid-logging.xml` file in the Prebid Server source code.
**Note**
If a container task receives a termination signal from ECS, for reasons such as scale-in or Spot interruptions, then the solution automatically compresses and transfers the current active log files to their respective archived folders to ensure no discontinuity or loss of log data. The transferred logs will be picked up along with any other log files in the `archived` directory when the DataSync task runs.
# Transfer of log files
DataSync is configured to periodically (every hour on the half hour mark) to move the rotated and archived metric log files from EFS to an equivalent location in S3. For details, see the [Logging](logging.md) section. This period is set through Cron schedule by `DATASYNC_METRICS_SCHEDULE = "cron(30 * * * ? *)"` and `DATASYNC_LOGS_SCHEDULE = "cron(30 * * * ? *)"`. These parameters are available through the `source/infrastructure/prebid_server/stack_constants.py` file in the solution [source code](https://github.com/aws-solutions-library-samples/prebid-server-deployment-on-aws).
After each file is copied to S3 and verified, it is removed from the EFS file system through a Lambda function. Only actively written log files are retained on the EFS.
# Metrics processing
An AWS Glue job is configured to perform ETL operation on the metrics log data present in the S3 bucket containing the raw logs. The ETL operation structures the metric data present in the log files into a single database with several tables. For details, see [Glue table schemas](glue-table-schemas.md) in the Querying metrics with Athena section. The processed data is written into the `MetricsEtl` S3 bucket, which contains the metric log data transformed and partitioned through ETL in the previous steps. The data is made available via AWS AWS Glue Data Catalog, a persistent technical metadata store for analytics users to discover and prepare the data. Also, you can immediately search and query cataloged data using Amazon Athena, [Amazon EMR](https://aws.amazon.com/emr/), and [Amazon Redshift Spectrum](https://docs.aws.amazon.com/redshift/latest/dg/c-using-spectrum.html).