# Extract sensor data and apply object detection with custom business logic Scene Intelligence with Rosbag on AWS provides an end-to-end solution for extracting sensor data from [rosbag](http://wiki.ros.org/rosbag) files generated from autonomous driving use cases. This solution guides users through a sample use case to demonstrate how the processing occurs and where custom logic can be applied to the extracted data. You can use this solution to: + Stage sample rosbag files + Extract rosbag sensor data such as metadata and images + Apply object detection ([YOLOv5](https://pytorch.org/hub/ultralytics_yolov5/)) and lane detection ([LaneDet](https://github.com/Turoad/lanedet)) models to extracted images + Apply scene detection business logic and store the output in an indexable fashion, using [Amazon DynamoDB](https://aws.amazon.com/dynamodb/) and [Amazon OpenSearch Service](https://aws.amazon.com/opensearch-service/) This implementation guide provides an overview of the Scene Intelligence with Rosbag on AWS solution, its reference architecture and components, considerations for planning the deployment, and configuration steps for deploying the solution to the Amazon Web Services (AWS) Cloud. The intended audience for using this solution’s features and capabilities in their environment includes solution architects, business decision makers, DevOps engineers, data scientists, and cloud professionals. Use this navigation table to quickly find answers to these questions: | If you want to . . . | Read . . . | | --- | --- | | Know the cost for running this solution. The estimated cost for running this solution in the **US East (N. Virginia)** Region is **USD \$1605.13 per month** for AWS resources. | [Cost](cost.md) | | Understand the security considerations for this solution. | [Security](security-1.md) | | Know how to plan for quotas for this solution. | [Quotas](quotas.md) | | Know which AWS Regions support this solution. | [Supported AWS Regions](supported-aws-regions.md) | | View or download the AWS CloudFormation template included in this solution to automatically deploy the infrastructure resources (the "stack") for this solution. | [AWS CloudFormation template](aws-cloudformation-template-for-deployment.md) | | Access the source code and optionally use the AWS Cloud Development Kit (AWS CDK) to deploy the solution. | [GitHub repository](https://github.com/awslabs/autonomous-driving-data-framework/tree/SO0279-v1.0.2) | # Features and benefits The solution provides the following features: **Extract rosbag data** You can use this solution to store rosbag files in [Amazon Simple Storage Service](https://aws.amazon.com/s3/) (Amazon S3) and invoke [Directed Acyclic Graphs](https://docs.aws.amazon.com/mwaa/latest/userguide/working-dags.html) (DAGs) with [Amazon Managed Workflows for Apache Airflow](https://aws.amazon.com/managed-workflows-for-apache-airflow/) (Amazon MWAA) to extract and process the data. With the data extracted and processed, you can more easily review and analyze your data. **Customizable end-to-end** You can determine which sensors to process, change the machine learning (ML) models used for processing, and bring your own DAGs for processing. **Indexable metadata** Business data output that the solution writes to DynamoDB and is automatically available in OpenSearch Service, ready to be queried without user management. **Integration with Service Catalog AppRegistry and Application Manager, a capability of AWS Systems Manager** This solution includes a [Service Catalog AppRegistry](https://docs.aws.amazon.com/servicecatalog/latest/arguide/intro-app-registry.html) resource to register the solution’s CloudFormation template and its underlying resources as an application in both AppRegistry and [Application Manager](https://docs.aws.amazon.com/systems-manager/latest/userguide/application-manager.html). With this integration, you can centrally manage the solution’s resources and enable application search, reporting, and management actions. # Use cases **Extract rosbag data** You can use this solution to extract the structured data (metadata) and unstructured data, such as images, from defined topics in Robot Operating System (ROS)1 rosbag files. The topics in this solution match the same that rosbag provides, and you can change them to fit your custom topics. The solution writes all topic data to Amazon S3, including images as .png files and metadata as Apache Parquet. **Apply business logic** You can apply custom business logic to the extracted data by using an Apache Spark script. Modify the processing DAG to match your rosbag, apply your own logic, or use the provided DAG and scripts. **Search for anomalies** The processing logic output applied to the extracted data is written to DynamoDB and OpenSearch Service. You can query the metadata output from an OpenSearch Service dashboard or with an OpenSearch service application programming interface (API) to locate a section of the rosbag file for inspection. # Concepts and definitions This section describes key concepts and defines terminology specific to this solution: **AV/ADAS** Autonomous Vehicle/Advanced Driver-Assistance System. Developing and deploying AV/ADASs requires scalable compute, storage, networking, analytics, and deep learning frameworks. **DAG** Directed Acyclic Graph. Workflows in Amazon MWAA are authored as DAGs using Python. **drive** Logical grouping of data, such as `Test Car 1 stores its rosbag file on Drive 1`. **lane detection (LaneDet)** A specific object detection model to identify automotive roads within images. **object detection** Refers to identifying objects within an image pulled from the rosbag file. Based on the [COCO dataset](https://cocodataset.org/) for object detection, this term implies the detection of recognized objects such as street lights, stop signs, and people. **ROS** Robot Operating System. The ROS is a set of software libraries and tools that help you build robot applications. **rosbag** A ROS archive file containing sensor data, meant for playback and logging. **scene detection** See **object detection**. **YOLO** You Only Look Once, a PyTorch object detection model. **Note** For a general reference of AWS terms, see the [AWS Glossary](https://docs.aws.amazon.com/general/latest/gr/glos-chap.html).