# Guidance for Automated Geospatial Insights Engine on AWS

## Overview

This Guidance shows how build an automated geospatial insights engine on AWS to improve supply and demand forecasting and risk management. Many industries depend upon intelligence and insights gained from processed geospatial and Earth-observation data from sources like satellite and aerial imagery and remote sensing. This Guidance couples this data with mechanistic and AI-powered models, so you can enhance forecasting, automate risk mitigation, and meet regulations.

## How it works

### Overview

This architecture diagram shows an overview of the automated geospatial analysis engine, specifically the key modules and their interactions.

[Download the architecture diagram](https://d1.awsstatic.com/solutions/guidance/architecture-diagrams/automated-geospatial-insights-engine-on-aws-generic.pdf?target=_blank)Step 1The UI module is responsible for transforming analysis results into displayable visual assets. It uses a geospatial mapping service to obtain base map data, and it renders interactive maps on the frontend using a map-rendering library.Step 2The region module manages the hierarchical structures of groups, regions, polygons, and states.Step 3The scheduler module manages the scheduling of engine processing tasks based on each region's processing configuration.Step 4The scheduler module also subscribes to new satellite images using an Amazon Simple Notification Service (Amazon SNS) topic hosted in Open Data on AWS.Step 5The executor module manages the execution of the engine analysis, invoked by the schedule module or by new sentinel images in Amazon Simple Storage Service (Amazon S3) that match the created regions.Step 6The results module acts as an intermediary between the region and executor modules, transforming and publishing farm data to a SpatioTemporal Asset Catalog (STAC) server and mapping execution results to their corresponding farms.Step 7The notification module manages your subscriptions to notifications generated by other modules.Step 8Note: The STAC server module is an AWS Cloud Development Kit (AWS CDK) port of stac-server, an execution of the STAC API specification for searching and serving metadata for geospatial data, including but not limited to satellite imagery.### UI Module

This architecture diagram shows how to set up the user interface (UI), manages authentication and authorization, and query analysis results.

[Download the architecture diagram](https://d1.awsstatic.com/solutions/guidance/architecture-diagrams/automated-geospatial-insights-engine-on-aws-generic.pdf#page=2?target=_blank)Step 1Load the sample React application from the S3 bucket, using Amazon CloudFront to maintain low latency and high performance.Step 2The React application renders maps using Amazon Location Service.Step 3The React application overlays the base map with the processed satellite images by querying the tiler API on Amazon API Gateway.Step 4Amazon Cognito delivers a token for your authentication.Step 5Amazon Verified Permissions uses the role encoded in your token to perform fine-grained authorization.Step 6Once you have been authenticated, the AWS Lambda tiler-API function queries the STAC server's metadata of the polygon execution result (based on the current view port of the map).Step 7The Lambda tiler-API function generates signed URLs for all image assets specified in the STAC items and returns them to the React application for rendering.### Region Module

This architecture diagram demonstrates the hierarchical structure of groups, regions, polygons, and states.

[Download the architecture diagram](https://d1.awsstatic.com/solutions/guidance/architecture-diagrams/automated-geospatial-insights-engine-on-aws-generic.pdf#page=3?target=_blank)Step 1Create grower, farm, and field resources using the region API through API Gateway. During the process of setting up a new farm, you'll be able to define the schedule and prioritization for the geospatial data processing tasks associated with it.Step 2Amazon Cognito delivers a token for your authentication.Step 3Verified Permissions uses the role encoded in your token to perform fine-grained authorization. Your role is stored in custom:role, a custom attribute of Amazon Cognito.Step 4Once you have been authenticated, API Gateway forwards the request to Lambda.Step 5The grower, region, and farm data is stored in an Amazon DynamoDB table.Step 6A region's resource changes emit events to an Amazon EventBridge event bus. Events can be tracked to update other components of the framework.Step 7Lambda records a segment with details about invoking and running the function and sends it to AWS X-Ray.### Scheduler Module

This architecture diagram shows how to schedule the engine's processing tasks, based on each region's processing configuration.

[Download the architecture diagram](https://d1.awsstatic.com/solutions/guidance/architecture-diagrams/automated-geospatial-insights-engine-on-aws-generic.pdf)Step 1The scheduler module subscribes to farm events published by the region module.Step 2If a farm is configured with a recurrent schedule, it creates a processing schedule for that farm using an EventBridge scheduler.Step 3When a schedule is invoked, the EventBridge scheduler puts messages into the job queue for the farm to be processed.Step 4The scheduler module also subscribes to new satellite-image notifications through an Amazon SNS topic hosted in Open Data on AWS. Amazon Simple Queue Service (Amazon SQS) queues messages.Step 5When a new satellite image becomes available in the Open Data on AWS bucket, Lambda queries the STAC server module to check if the image includes the geographic region where the farms are located.Step 6In the event of spatial intersection, Lambda puts a processing-task message in the job queue.Step 7Lambda records a segment with details about invoking and running the function and sends it to X-Ray.### Executor Module

This architecture diagram demonstrates the execution of the region analysis invoked by theschedule module.

[Download the architecture diagram](https://d1.awsstatic.com/solutions/guidance/architecture-diagrams/automated-geospatial-insights-engine-on-aws-generic.pdf#page=5?target=_blank)Step 1The executor module subscribes to the job queue of the scheduler module, which queues messages when a farm is scheduled for processing.Step 2The queue invokes the executor module using Lambda, which then places the job in the right-priority AWS Batch queue based on the farm configuration.Step 3AWS Batch will pick up a job from these queues and launch tasks using AWS Fargate for Amazon Elastic Container Service (Amazon ECS) to process the fields associated with the farm in parallel.Step 4The Amazon ECS tasks retrieve the satellite images from the S3 bucket hosted in Open Data on AWS.Step 5These tasks then process the images and store the results in an S3 bucket.Step 6Once the execution finishes, the Amazon ECS tasks publish the results' metadata to an EventBridge event bus.### Results Module

This architecture diagram shows how to transform data into STAC items, publish them to the STAC server, and map execution results to corresponding regions.

[Download the architecture diagram](https://d1.awsstatic.com/solutions/guidance/architecture-diagrams/automated-geospatial-insights-engine-on-aws-generic.pdf#page=6?target=_blank)Step 1Interact with the results module through API Gateway.Step 2Amazon Cognito delivers a token for your authentication.Step 3Verified Permissions uses the role encoded in your token to perform fine-grained authorization.Step 4Once you have been authenticated, API Gateway forwards the request to Lambda.Step 5The data containing the execution result details—such as the unique ID, status, last updated time, and execution specifics—is stored in a DynamoDB table.Step 6The results module subscribes to grower- and farm-change events from the region module and farm-analysis-result events from the executor module.Step 7The results module transforms these events into a STAC item and publishes it to the Amazon SNS STAC server module topic.Step 8Lambda records a segment with details about invoking and running the function and sends it to X-Ray.### Notification Module

This module shows how to manage subscriptions to notifications generated by other modules.

[Download the architecture diagram](https://d1.awsstatic.com/solutions/guidance/architecture-diagrams/automated-geospatial-insights-engine-on-aws-generic.pdf#page=7?target=_blank)Step 1Create a subscription for farm processing events through API Gateway.Step 2Amazon Cognito delivers a token for your authentication.Step 3Verified Permissions uses the role encoded in your token to perform fine-grained authorization.Step 4Once you have been authenticated, API Gateway forwards the request to Lambda.Step 5Your list of subscription data is stored in a DynamoDB table.Step 6Lambda subscribes you to an Amazon SNS farm topic. (It will create the topic if it does not already exist.)Step 7Lambda subscribes you to an EventBridge event bus for farm processing events published by the execution module.Step 8Lambda transforms the events into notification messages and publishes them to you (or other subscribed users) through Amazon SNS.Step 9Lambda records a segment with details about invoking and running the function and sends it to X-Ray.## Deploy with confidence

Everything you need to launch this Guidance in your account is right here.

- **Let's make it happen**: Ready to deploy? Review the sample code on GitHub for detailed deployment instructions to deploy as-is or customize to fit your needs.

[Go to sample code](https://github.com/aws-solutions-library-samples/guidance-for-automated-geospatial-insights-engine-on-aws?target=_blank)


## Well-Architected Pillars

The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.

### Operational Excellence

X-Ray provides complete tracing and monitoring capabilities to help you identify performance bottlenecks and troubleshoot issues. It enables you to visualize and analyze the components of this Guidance, such as API calls, Lambda functions, and AWS Step Functions workflows. Finally, AWS CloudFormation provisions and manages the required resources, enabling automated deployments and changes. [Read the Operational Excellence whitepaper](/wellarchitected/latest/operational-excellence-pillar/welcome.html)


### Security

Amazon Cognito provides secure user authentication mechanisms so that only authorized users can access your applications and resources. AWS Identity and Access Management (IAM) policies and roles let you control access to resources according to the principle of least privilege. Additionally, Verified Permissions provides a centralized and efficient system for managing access to your custom applications. Through a policy-based approach that uses the Cedar policy language for defining fine-grained permissions, it promotes consistency and maintainability in the authorization process. It also separates the authorization process from the application code so that you can easily update and manage permissions. [Read the Security whitepaper](/wellarchitected/latest/security-pillar/welcome.html)


### Reliability

AWS Batch automatically scales the number of compute resources based on the number of jobs in the queue. As a result, this Guidance provisions the right amount of resources for satellite image processing, reducing the risk of job failures due to resource constraints. Amazon SQS and EventBridge maintain durability and persistence by decoupling components through asynchronous messaging and by storing messages redundantly across multiple Availability Zones. This makes applications more resilient: if an individual component fails, messages are temporarily stored in the queue and processed when the component recovers. [Read the Reliability whitepaper](/wellarchitected/latest/reliability-pillar/welcome.html)


### Performance Efficiency

DynamoDB lets you define a partition key (and an optional sort key) for your table to enhance performance efficiency. By using defined keys to distribute your data across servers and partitions in addition to using appropriate indexes, you can optimize data access patterns. Additionally, Amazon S3 offers seamless scalability, enabling your applications to handle high request rates effortlessly. For example, each partitioned Amazon S3 prefix can sustain at least 3,500 requests per second for PUT, COPY, POST, and DELETE operations. Each prefix can also sustain up to 5,500 requests per second for GET and HEAD operations. Because there are no restrictions on the number of prefixes you can create within a bucket, you can partition your data effectively to achieve optimal performance and scalability. And by encoding the region, polygon, or results ID in the file prefix when storing satellite analysis images, you can enable parallel processing and listing. [Read the Performance Efficiency whitepaper](/wellarchitected/latest/performance-efficiency-pillar/welcome.html)


### Cost Optimization

The serverless and event-driven nature of Lambda, EventBridge, Amazon SQS, and AWS Batch means that they automatically scale based on demand and don’t overprovision capacity. This helps you optimize costs, because you only pay for the resources you use. Additionally, EventBridge and Amazon SQS remove the need for you to set up polling-based architectures, helping you further reduce costs. [Read the Cost Optimization whitepaper](/wellarchitected/latest/cost-optimization-pillar/welcome.html)


### Sustainability

This Guidance uses serverless services for compute and storage—including Lambda, AWS Batch, and DynamoDB —so they automatically scale to zero when not in use. This removes the need for always-on infrastructure and reduces the overall environmental impact of your workloads. Additionally, Lambda uses AWS Graviton2 processors, which are designed to be more energy-efficient than traditional x86-based processors. They consume less power while delivering comparable performance, reducing your carbon emissions. [Read the Sustainability whitepaper](/wellarchitected/latest/sustainability-pillar/sustainability-pillar.html)


[Read usage guidelines](/solutions/guidance-disclaimers/)