The bidder application uses a microservices approach, meaning that the bid request-response event capture and post bid events capture services are decoupled from the bidder service. This enables independent release cycles and scaling mechanisms based on the transactions-per-second (TPS) requirements. For example, the post bid event capture TPS is multiple orders of magnitude less than other components and can be configured independently.
Overview
This Guidance helps ad-tech companies capture Open Real-Time Bidding (OpenRTB) events and establish a foundation for near real-time and batch analytics. During a programmatic advertising transaction, a demand-side platform (DSP) service generates a series of events. Capturing these events helps ad-tech companies keep their budgets updated and understand signals for optimizing the bid response. By tracking events, such as a successful win bid, advertisers can better measure the effectiveness of their campaigns. They can also analyze these events to make informed decisions for future bids.
For an overview of RTB and a description of the events that advertisers generate to win a bid request, visit
Guidance for Building a Real Time Bidder for Advertising on AWS
.
How it works
This architecture enables you to capture OpenRTB bid events for both near real-time and batch analytics.
Step 1
The supply-side platform (SSP) receives an ad request from a publisher and launches an auction.
Step 2
An OpenRTB bid request is sent to a DSP public endpoint that is configured on an Elastic Load Balancer.
Step 3
The bidder application service receives the bid request. This service runs on Amazon Elastic Kubernetes Service (Amazon EKS) within an Amazon Virtual Private Cloud (VPC).
Step 4
The bid request-response event capture service is hosted on a different container pod in the same cluster. To reduce latency of the bid response, combine the publishing of the bid request and response event as a single call per bid asynchronously to the bid request-response event capture service endpoint.
Step 5
Post bid events capture service is hosted on a separate container pod that exposes the service to SSPs. This service is used to receive post bid events.
Step 6
Implement the event capture service in Java to take advantage of Amazon Kinesis Producer Library (KPL). KPL simplifies implementation of an asynchronous producer application and reduces costs for sending data to the Amazon Kinesis Data Streams API.
Step 7
The event messages are routed to Kinesis Data Streams through a dedicated VPC endpoint.
Step 8
Amazon Kinesis Data Firehose consumes these aggregated records and deaggregates and sends individual events to Amazon Simple Storage Service (Amazon S3) for long-term storage and analytics.
Step 9
Amazon CloudWatch captures application logs for traceability.
Step 10
AWS Key Management Service (AWS KMS) stores and manages encryption keys used for securing persisted data in Kinesis Data Streams and Amazon S3.
Well-Architected Pillars
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
Operational Excellence
Security
Data at rest in the Amazon S3 bucket is encrypted with AWS KMS keys. Data in transit is encrypted and transferred over HTTPS. The Amazon EKS clusters run in a VPC, and the connectivity between Amazon EKS and Kinesis is routed through a secure VPC endpoint.
Reliability
The bid request-response and post bid event capture services are decoupled from the code bidder application. The bidder application needs to implement throttling, retry, and backpressure accordingly. Additionally, the architecture promotes a stateless implementation using containers. The KPL implements throttling, retry, and asynchronous publish of events.
Performance Efficiency
All components of this Guidance are co-located in a single Region and Availability Zone, and you can use automated deployments to deploy the architecture into any Region that meets your data residency and latency requirements.
Cost Optimization
With serverless services, you pay only for the resources you consume. The Amazon EKS cluster is the only component in this architecture that does not use serverless services. To meet the high TPS workload, we recommend a hybrid consumption model for the Amazon EKS cluster. Amazon Elastic Compute Cloud (Amazon EC2) Spot instances provide up to a 90% discount compared to on-demand instances and can support additional bursts in workloads. Additionally, the services in the bidder application run within an Amazon VPC and are deployed in a single Availability Zone to minimize data transfer costs.
Sustainability
With Amazon EKS, you can implement scalability and elasticity. By decoupling components through microservices, you can allocate the right instance type and the right number of instances to dynamic workloads, helping you to not overprovision resources.
Related content
Guidance for Building a Real Time Bidder for Advertising on AWS
This Guidance helps you assess ad opportunities at scale using real time bidding (RTB). It shows how demand side platforms (DSPs) bid on ad space available from supply side platforms (SSPs).