# Plan your deployment
This section describes the [cost](cost1.md), [configurations for ECS Auto Scaling](configurations-for-elastic.md), [security](security-1.md), [Region](#supported-aws-regions), and [quota](quotas.md) considerations for planning your deployment.
## Supported AWS Regions
Prebid Server on AWS is supported in the following AWS Regions:
| Region name | |
| --- | --- |
| US East (Ohio) | Asia Pacific (Tokyo) |
| US East (N. Virginia) | Canada (Central) |
| US West (Northern California) | Europe (Frankfurt) |
| US West (Oregon) | Europe (Ireland) |
| Asia Pacific (Mumbai) | Europe (London) |
| Asia Pacific (Osaka) | Europe (Paris) |
| Asia Pacific (Seoul) | Europe (Stockholm) |
| Asia Pacific (Singapore) | South America (São Paulo) |
| Asia Pacific (Sydney) | |
# Cost
You are responsible for the cost of the AWS services used while running this solution. As of this revision, the cost for running this solution with the default settings with no incoming bidding traffic to the solution in the US East (N. Virginia) Region is approximately **\$1237.50 per month** .
We recommend creating a [budget](https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-create.html) through AWS Cost Explorer to help manage costs. Prices are subject to change. For full details, see the pricing webpage for each AWS service used in this solution.
**Note**
If you want to opt out of using CloudFront and AWS WAF and directly send requests to the ALB, see [Opting out of using CloudFront and AWS WAF](configure-the-solution.md#opt-out-of-cloudfront-and-waf).
## Cost tables
The total cost of this solution includes two parts:
+ Cost for deploying the solution without incoming bidding traffic
+ Cost incurred from the traffic flowing through the solution
The following assumptions apply for calculating the cost:
+ The cost breakdown per month includes deploying this solution with the default parameters in the US East (N. Virginia) Region.
+ The Fargate to Fargate Spot pricing ratio is 1:1. With Fargate Spot instances, customers can run interrupt-tolerant Amazon ECS tasks on spare capacity at up to a 70% discount off of the regular Fargate instance price. For more information, see [AWS Fargate Pricing](https://aws.amazon.com/fargate/pricing/).
+ Each incoming HTTP request to the solution is of a fixed size (0.5 KB), and three bidders participate in a single auction request.
### Sample cost table 1
**No incoming bidding traffic to the solution, two Amazon ECS tasks**
| AWS service | Dimensions | Monthly cost [USD] |
| --- | --- | --- |
| Amazon ECS | Operating system (Linux), CPU architecture (x86), Average duration (30 days), Number of tasks or pods (2 per month), Amount of memory allocated (4 GB), Amount of ephemeral storage allocated for Amazon ECS (20 GB) | \$154.50 |
| AWS WAF | Number of Web Access Control Lists (Web ACLs) utilized (1 per month), Number of Managed Rule Groups per Web ACL (6 per month) | \$115.00 |
| Elastic Load Balancing | Number of Application Load Balancers (1) | \$117.00 |
| Amazon EC2 - other | Number of NAT gateways (2) DT inbound: Not selected (0 TB per month), DT outbound: Internet (<50 GB per month), DT Intra-Region: (0 TB per month) | \$169.00 |
| Amazon EFS | Desired storage capacity (1 TB per month), Infrequent access requests (<2 GB per month) | \$125.00 |
| Amazon S3 | S3 Standard storage | \$14.00 |
| Amazon CloudWatch | Number of Standard Resolution Alarm Metrics (20), Standard logs: Data ingested (<20 GB) | \$110.00 |
| Other servicess | Amazon CloudFront, AWS CloudTrail AWS DataSync, IAM, AWS Glue, AWS KMS, AWS Lambda, and Amazon VPC | \$147.00 |
| | **Total:** | **\$1241.50** |
### RTB Fabric cost (optional)
When deploying with the bidder simulator and RTB Fabric integration (`--include-rtb-fabric`), additional costs apply:
**RTB Fabric with 50GB monthly outbound traffic**
Assuming \$110 KB average bid request, 3 AWS RTB Fabric linked internal bidders per auction, 50 GB outbound traffic (responses are DT-IN and free):
| AWS service | Dimensions | Monthly cost [USD] |
| --- | --- | --- |
| AWS RTB Fabric | Transaction pricing (\$13/billion), Data transfer pricing (\$10.02/GB). Total auctions: \$11.67 million. Total bid requests: \$15 million (0.005 billion) | \$116.00 |
**Cost comparison: RTB Fabric vs NAT Gateway (50GB monthly outbound traffic)**
| Connectivity Method | Monthly Cost [USD] |
| --- | --- |
| RTB Fabric (transaction \$1 data transfer) | \$116.00 |
| NAT Gateway (baseline from table above) | \$169.00 |
| **Monthly savings with RTB Fabric** | **\$153.00 (77% reduction)** |
For current RTB Fabric pricing details, see the [AWS RTB Fabric pricing page](https://aws.amazon.com/rtb-fabric/pricing/). With incoming traffic, costs scale based on the volume and size of RTB requests sent through the Fabric Link. Note that bid responses returning to Prebid Server are data transfer IN and incur no charges.
### Sample cost table 2
**1,500 incoming requests per second (appoximately 1.3B × 3 = 3.7B impressions per month), 10 tasks total**
| AWS service | Dimensions | Monthly Cost [USD] |
| --- | --- | --- |
| Amazon CloudFront | Data transfer out to origin (1.5 TB per month), Data transfer out to internet (0.75 TB per month), Number of requests (HTTPS) (1.3 billion per month) | \$1967.50 |
| Amazon ECS | Operating system (Linux), CPU architecture (x86), Average duration (30 days), Number of tasks or pods (10 per month), Amount of memory allocated (4 GB), Amount of ephemeral storage allocated for Amazon ECS (20 GB) | \$1276.00 |
| AWS WAF | Number of Web Access Control Lists (Web ACLs) utilized (1 per month), Number of Managed Rule Groups per Web ACL (6 per month) | \$1211.00 |
| Elastic Load Balancing | Number of Application Load Balancers (1) | \$1133.00 |
| Amazon EC2 - other | Number of NAT Gateways (2) DT inbound: Not selected (0 TB per month), DT outbound: Internet (1.5 TB per month), DT Intra-Region: (0 TB per month), Data transfer cost (13.5) | \$1273.00 |
| Amazon EFS | Desired storage capacity (1 TB per month), Infrequent access requests (2 GB per month) | \$196.00 |
| Amazon S3 | S3 Standard storage (2.5 TB per month) | \$159.00 |
| Amazon CloudWatch | Number of Standard Resolution Alarm Metrics (20), Standard logs: Data ingested (41 GB) | \$123.00 |
| AWS Glue | Data processing unit-hour for AWS Glue ETL job, approx. 200 DPU-hour | \$192.00 |
| | **Total:** | **\$12,130.50** |
**With RTB Fabric (optional):**
Assuming \$110 KB average bid request, 3 AWS RTB Fabric linked internal bidders per auction, 1.5 TB outbound traffic:
| AWS service | Dimensions | Monthly cost [USD] |
| --- | --- | --- |
| AWS RTB Fabric | Transaction pricing (\$13/billion), Data transfer pricing (\$10.02/GB). Total auctions: \$11.3 billion. Total bid requests: \$13.9 billion (0.0039 billion) | \$142.42 |
| Amazon EC2 - other (NAT Gateway baseline) | Number of NAT Gateways (2) with 1.5 TB outbound | \$1273.00 |
| **Monthly savings with RTB Fabric** | | **\$1230.58 (84% reduction)** |
### Sample cost table 3
**9,000 incoming requests per second (7.8B × 3 = 23B impressions per month), 60 Amazon ECS tasks**
| AWS service | Dimensions | Monthly cost [USD] |
| --- | --- | --- |
| Amazon CloudFront | Data transfer out to origin (30 TB per month), Data transfer out to internet (5.75 TB per month), Number of requests (HTTPS) (7.8 billion per month) | \$16,256.00 |
| Amazon ECS | Operating system (Linux), CPU architecture (x86), Average duration (30 days), Number of tasks or pods (60 per month), Amount of memory allocated (4 GB), Amount of ephemeral storage allocated for Amazon ECS (20 GB) | \$11,660.00 |
| AWS WAF | Number of Web Access Control Lists (Web ACLs) utilized (1 per month), Number of Managed Rule Groups per Web ACL (6 per month) | \$11,717.00 |
| Elastic Load Balancing | Number of Application Load Balancers (1) | \$1717.00 |
| Amazon EC2 - other | Number of NAT Gateways (2) DT inbound: Not selected (0 TB per month), DT outbound: Internet (1.2 TB per month), DT Intra-Region: (0 TB per month), Data transfer cost (13.5) | \$1748.00 |
| Amazon EFS | Desired Storage Capacity (1 TB per month), Infrequent access requests (5 GB per month) | \$196.00 |
| Amazon S3 | S3 Standard storage (5 TB per month) | \$1118.00 |
| Amazon CloudWatch | Number of Standard Resolution Alarm Metrics (20), Standard logs: Data ingested (41 GB) | \$123.00 |
| AWS Glue | Data processing unit-hour for AWS Glue ETL job, approx. 200 DPU-hour | \$192.00 |
| | **Total:** | **\$111,427.00** |
**With RTB Fabric (optional):**
Assuming \$110 KB average bid request, 3 AWS RTB Fabric linked internal bidders per auction, 1.2 TB outbound traffic:
| AWS service | Dimensions | Monthly cost [USD] |
| --- | --- | --- |
| AWS RTB Fabric | Transaction pricing (\$13/billion), Data transfer pricing (\$10.02/GB). Total auctions: \$17.8 billion. Total bid requests: \$123.4 billion (0.0234 billion) | \$194.78 |
| Amazon EC2 - other (NAT Gateway baseline) | Number of NAT Gateways (2) with 1.2 TB outbound | \$1748.00 |
| **Monthly savings with RTB Fabric** | | **\$1653.22 (87% reduction)** |
# Configurations for Elastic Container Service (ECS) Auto Scaling
The recommended configurations for the deployed solution’s automatic scaling are dependent on the approximate maximum requests per second (RPS) and maximum number of users the solution is expected to support.
In this context, RPS means HTTP or HTTPS requests per second. A single request can contain multiple bid requests that can result in multiple bid responses inside the HTTP response. The request and response might both contain a payload. The average response time refers to the amount of time it takes to receive winning bids, measured in seconds, and the timer starts when the requests for advertisement bids are sent out and stops when the winning bids are received.
The recommendations in this section were determined via load testing with [Distributed Load Testing on AWS](https://aws.amazon.com/solutions/implementations/distributed-load-testing-on-aws/). In the load tests, 10,000 users with 16.7 new users being added per second were spawned across `us-east-1`, `us-west-1`, `us-east-2`, and `us-west-2` Regions to generate traffic to the Prebid server cluster.
In the context of load testing, a user continuously makes an auction request to the auction API. 80% of the total RPS are auction API requests. The user infrequently sends requests to the non-auction APIs. This includes information and status check requests. The approximate average payload sizes for an API request and response is 123 KB and 331 KB respectively.
The statistics in the tables below were calculated by the data collected from `us-east-1`, `us-west-1`, `us-east-2`, and `us-west-2` Regions.
## Static cluster size configurations
The following table lists the recommended static cluster sizes and their associated maximum stable RPS limits, average response time, and success rate if ECS Auto Scaling is turned off.
| ECS number of tasks with no Auto Scaling | Transactions per second | Average response time in seconds | Success rate |
| --- | --- | --- | --- |
| 1 | 800.79 | 9.56630 | 87.70% |
| 10 | 4145.95 | 1.84996 | 97.38% |
| 25 | 11074.86 | 0.69569 | 98.93% |
| 50 | 75912 | 0.35765 | 99.60% |
| 100 | 75411 | 0.17981 | 99.89% |
| 200 | 64621.02 | 0.13120 | 99.86% |
| 400 | 128793.61 | 0.07452 | 99.97% |
Significant latency and failed requests were observed when the traffic was exceeded for each number of tasks tested. Further increases to the number of tasks were able to handle the 10,000-user test load with better success rate, average response time, and RPS.
## Auto Scaling cluster configurations
Turning on Auto Scaling in ECS increases the performance of the solution’s maximum RPS. The following recommended ECS Auto Scaling policies and parameters were used in the load tests.
Parameters:
+ Minimum number of tasks: 10
+ Maximum number of tasks: 100
Policies:
+ `ALBRequestCountPerTarget`
+ Target value: 5000
+ Scale-out cooldown period: 300
+ Scale-in cooldown period: 300
+ `ECSServiceAverageCPUUtilization`
+ Target value: 66%
+ Scale-out cooldown period: 300
+ Scale-in cooldown period: 300
+ `ECSServiceAverageMemoryUtilization `
+ Target value: 50%
+ Scale-out cooldown period: 300
+ Scale-in cooldown period: 300
The following table lists the auto-scaling policies and their associated maximum stable RPS limits, average response time, and success rate.
| Auto-scaling policies | Min number of tasks scaled | Max number of tasks scaled | Transactions per second | Average response time in seconds | Success rate |
| --- | --- | --- | --- | --- | --- |
| ALBRequestCountPerTarget (ALB) | 10 | 100 | 17367.6 | 0.44486 | 99.51% |
| ECSServiceAverageCPUUtilization (CPU) | 10 | 13 | 4708.65 | 1.85956 | 96.41% |
| ECSServiceAverageMemoryUtilization (Mem) | 10 | 12 | 5820.56 | 1.31274 | 98.59% |
| ALB & CPU | 10 | 100 | 14948.84 | 0.51504 | 99.48% |
| ALB & Mem | 10 | 100 | 15208.86 | 0.50105 | 99.50% |
| CPU & Mem | 10 | 13 | 4747.65 | 1.60875 | 97.08% |
| ALB & CPU & Mem | 10 | 100 | 16211.21 | 0.49361 | 99.42% |
The `ALBRequestCountPerTarget` policy is the most important auto-scaling policy and plays the biggest influence on the performance. However, we recommend that you use all three of the Auto Scaling policies above. Removing them will decrease the maximum RPS and increase response time because then the containers are more prone to becoming overloaded. The policies also make the deployed solution more resilient to cases where there is a burst of users.
The maximum number of tasks and minimum number of tasks can be adjusted depending on the solution’s usage. We recommend to at least have 50 tasks and have Auto Scaling turned on for the deployed solution to reduce response times and the chance of errors occurring.
## Fargate Spot instances ratio configurations
We recommend that you keep the solution’s default 50:50 ratio of the Fargate Spot instances to Fargate instances at least. This is because during testing, the Fargate instances were found to help the system scale and react more quickly to user traffic and support higher RPS more quickly with higher success rate.
The following table lists the Fargate Spot instances ratio and their associated maximum stable RPS limits, average response time, and success rate.
| Fargate: Fargate Spot | Transactions per second | Average response time in seconds | Success rate |
| --- | --- | --- | --- |
| 50:50 | 17789.75 | 0.43171 | 99.60% |
| 100:0 | 134244.83 | 0.07305 | 100% |
## Example cluster size, Auto-scaling policy, and Fargate Spot instances ratio configurations
You can use the following specifications for Prebid Server, based upon the testing conducted in this document.
Parameters:
+ Minimum number of tasks: 50
+ Maximum number of tasks: 400
Policies:
+ `ALBRequestCountPerTarget`
+ Target value: 5000
+ Scale-out cooldown period: 300
+ Scale-in cooldown period: 300
+ `ECSServiceAverageCPUUtilization`
+ Target value: 66%
+ Scale-out cooldown period: 300
+ Scale-in cooldown period: 300
+ `ECSServiceAverageMemoryUtilization`
+ Target value: 50%
+ Scale-out cooldown period: 300
+ Scale-in cooldown period: 300
Fargate Spot instances ratio:
+ Fargate instances: 80
+ Fargate Spot instances: 20
The metrics achieved in testing with the above configurations are in the following table.
| Results from recommended configurations | |
| --- | --- |
| Maximum transaction per second | 190881.19 |
| Average response time in seconds | 0.05533 |
| Success rate | 100% |
# Security
When you build systems on AWS infrastructure, security responsibilities are shared between you and AWS. This [shared responsibility model](https://aws.amazon.com/compliance/shared-responsibility-model/) reduces your operational burden because AWS operates, manages, and controls the components including the host operating system, the virtualization layer, and the physical security of the facilities in which the services operate. For more information about AWS security, visit [AWS Cloud Security](https://aws.amazon.com/security/).
## IAM roles
AWS Identity and Access Management (IAM) roles allow customers to assign granular access policies and permissions to services and users on the AWS Cloud. This solution creates IAM roles and policies with minimal permission that grant access to the solution’s resources.
## Amazon CloudFront
This solution deploys an Amazon CloudFront distribution and uses the default CloudFront domain name and SSL certificate. The default CloudFront SSL certificate only supports TLSv1. To use a later TLS version (TLS1.2 and above), use your own domain name and custom SSL certificate. For more information, refer to [Using alternate domain names and HTTPS](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-https-alternate-domain-names.html) in the *Amazon CloudFront Developer Guide*.
The Amazon CloudFront distribution is the unified network entry point. It helps to reduce latency by delivering data through globally dispersed points of presence (PoPs) with automated network mapping and intelligent routing. The inbound traffic to the CloudFront can be either HTTP or HTTPS for compatibility with various clients. As the Prebid Server hosted by ECS only supports HTTP, HTTPS proxy design is used in this solution to improve security. The CloudFront distribution acts as a TLS proxy, where APIs can be delivered over HTTPS using the latest version Transport Layer Security (TLSv1.3) to encrypt and secure communication between viewer clients and the CloudFront.
## Application Load Balancer (ALB)
ALB distributes incoming request traffic for Prebid Server through the cluster of containers. ALB provides a single entry point into the cluster, and it is the primary origin for the CloudFront distribution. CloudFront and the ALB use a shared secret header to prevent external traffic from bypassing the CloudFront distribution and accessing ALB directly. For more information, see [Restricting access to Application Load Balancers](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/restrict-access-to-load-balancer.html) in the *Amazon CloudFront Developer Guide*.
## Amazon VPC
Amazon VPC is configured with multiple subnets, routes, security groups, and NAT gateways. Security groups permit traffic to and from the subnets. The VPC contains the network interfaces for the Prebid Server cluster nodes. ALB has an interface in each private subnet of the VPC. Each container instance (or node) has an interface in its private subnet of the VPC. The actual number of interfaces varies based on the number of containers running. The VPC is configured for private IP addresses only, and container networks configured within the VPC use the NAT gateway as a default route to the internet for communication.
## AWS Fargate
This solution uses Amazon ECS to containerize the Prebid Server. The container runs the open source Prebid Server and is hosted by the Elastic Container Repository for AWS Solutions. A custom build is applied to the open source project’s default container in configuration settings for areas including file output and bidder adapter. To see how the ECS containers are constructed, refer to the [Dockerfile](https://github.com/aws-solutions-library-samples/prebid-server-deployment-on-aws/blob/main/deployment/ecr/prebid-server/Dockerfile). If you need to access these containers while they are running in Fargate, see [Using Amazon ECS Exec to access your containers on AWS Fargate and Amazon EC2](https://aws.amazon.com/blogs/containers/new-using-amazon-ecs-exec-access-your-containers-fargate-ec2/) on the *AWS Blog*.
## Security groups
This solution creates an Amazon EC2 security group within an Amazon VPC and associates it with ALB to act as a virtual firewall for the EC2 instances to control incoming and outgoing traffic. A rule with prefix list is added to the security group that allows ingress only from the CloudFront distribution.
## AWS WAF
This solution deploys AWS WAF and Shield Standard as a protection mechanism from DDoS attacks against the Prebid Server cluster. One or more managed rule groups can be activated in AWS WAF by default after extended testing including rules in the Baseline Rule Group and the IP Reputation Rule Group. AWS WAF allows the securement of web applications’ API from attacks before reaching to the servers. The customer has the option to activate, purchase, or use existing rule subscriptions, or add regular expression or CIDR matching rules as needed.
## Customer managed AWS KMS keys
AWS Key Management Service (KMS) lets customers create and manage cryptographic keys to activate server-side encryption. This solution creates six KMS keys and uses them in the S3 buckets storing artifacts, CloudFront access log data, CloudTrail events, DataSync log and metric files, and metadata of AWS AWS Glue Data Catalog to secure data at rest.
## Audit trails
CloudTrail is configured in this solution for auditing API calls against resources and used for problem analysis and remediation. CloudWatch alarms are used by compute resources for software failures. All compute resources send logging output to CloudWatch logs. CloudFront standard logs are configured to create log files that contain information about the user requests initiated to the solution’s CloudFront distribution.
# Quotas
Service quotas, also referred to as limits, are the maximum number of service resources or operations for your AWS account.
## Quotas for AWS services in this solution
Make sure you have sufficient quota for each of the [services implemented in this solution](architecture-details.md#aws-services-in-this-solution). For more information, see [AWS service quotas](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html).
Use the following links to go to the page for that service. To view the service quotas for all AWS services in the documentation without switching pages, view the information in the [Service endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/aws-general.pdf#aws-service-information) page in the PDF instead.
## AWS CloudFormation quotas
Your AWS account has AWS CloudFormation quotas that you should be aware of when deploying this solution. By understanding these quotas, you can avoid limitation errors that would prevent you from deploying this solution successfully. For more information, see [AWS CloudFormation quotas](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cloudformation-limits.html) in the in the *AWS CloudFormation User’s Guide*.