Verify installed components against reference images and bill of materials data using Amazon Bedrock. Catch missing or incorrect items that manual checks routinely overlook.
Overview
This Guidance shows how to automate equipment detection and installation verification tasks that traditionally require extensive manual effort by applying foundation models to on-site visual inspection workflows. When reference images and bill of materials are uploaded, foundation models generate detailed descriptions of each component and create module-specific detection rules. Field technicians then capture images during on-site inspections, and the system compares what appears in the photo against expected components for that module, identifying detected items and flagging anything missing in real time through a mobile interface. You can reduce inspection time, minimize human error in component validation, and ensure installation completeness without requiring technicians to manually cross-reference complex equipment lists.
Benefits
Reduce inspection errors with generative AI
Accelerate field verification workflows
Enable technicians to photograph assemblies and receive near real-time detection results on mobile devices. Cut inspection cycle times while maintaining consistent quality standards across sites.
Adapt detection rules without retraining models
Generate module-specific detection rules and false positive patterns automatically from your product data. Update inspection criteria as components change without custom model development.
How it works
This architecture diagram illustrates how to effectively support on-site visual inspection and verification on AWS. It shows the key components and their interactions, providing an overview of the architecture's structure and functionality.
Download the architecture diagram.
Step 1
Admin uploads images of products and bill of material (BOM) to Amazon Simple Storage Service bucket.
Item description generation pipeline AWS Lambda is triggered to process each reference image with Amazon Bedrock Claude Sonnet model and generates a reference rich and detailed description of the image.
Descriptions are combined with BOM data to generate module specific detection rules. Additional rules for common false positive patterns are generated for improved accuracy.
Descriptions and specific detection rules are stored in Amazon S3 and synced to Amazon DynamoDB table.
The detection pipeline is triggered when a user uploads the image to Amazon S3 while performing an on-site visual inspection.
A AWS Lambda function is triggered and prepares the context with the uploaded image and target module, retrieving relevant reference descriptions and detection rules from DynamoDB to provide context to the model (RAG approach).
Amazon Bedrock Nova Pro performs the detection task and outputs results to Amazon S3 and DynamoDB.
Detection results are retrieved by the mobile client through Amazon API Gateway, with all the items detected. User can move to next module for verification or restart the detection by taking a different picture to detect the missing items.
Related content
How Amazon uses Amazon Nova models to automate operational readiness testing for new fulfillment centers
Learn how Amazon uses Amazon Nova models to automate visual inspection and operational readiness testing.