# Searching faces in a streaming video **Note** Amazon Rekognition Streaming Video Analysis will no longer be open to new customers starting April 30, 2026. If you would like to use Streaming Video Analysis, sign up prior to that date. Existing customers for accounts that have used this feature within the last 12 months can continue to use the service as normal. For more information, see [Rekognition Streaming Video Analysis availability change](https://docs.aws.amazon.com/rekognition/latest/dg/rekognition-streaming-video-analysis-availability-change.html). Amazon Rekognition Video can search faces in a collection that match faces that are detected in a streaming video. For more information about collections, see [Searching faces in a collection](collections.md). **Topics** + [Creating the Amazon Rekognition Video face search stream processor](#streaming-video-creating-stream-processor) + [Starting the Amazon Rekognition Video face search stream processor](#streaming-video-starting-stream-processor) + [Using stream processors for face searching (Java V2 example)](#using-stream-processors-v2) + [Using stream processors for face searching (Java V1 example)](#using-stream-processors) + [Reading streaming video analysis results](streaming-video-kinesis-output.md) + [Displaying Rekognition results with Kinesis Video Streams locally](displaying-rekognition-results-locally.md) + [Understanding the Kinesis face recognition JSON frame record](streaming-video-kinesis-output-reference.md) The following diagram shows how Amazon Rekognition Video detects and recognizes faces in a streaming video. ![\[Diagram of workflow for using Amazon Rekognition Video to process video streams from Amazon Kinesis.\]](http://docs.aws.amazon.com/rekognition/latest/dg/images/VideoRekognitionStream.png) ## Creating the Amazon Rekognition Video face search stream processor Before you can analyze a streaming video, you create an Amazon Rekognition Video stream processor ([CreateStreamProcessor](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_CreateStreamProcessor.html)). The stream processor contains information about the Kinesis data stream and the Kinesis video stream. It also contains the identifier for the collection that contains the faces you want to recognize in the input streaming video. You also specify a name for the stream processor. The following is a JSON example for the `CreateStreamProcessor` request. ``` { "Name": "streamProcessorForCam", "Input": { "KinesisVideoStream": { "Arn": "arn:aws:kinesisvideo:us-east-1:nnnnnnnnnnnn:stream/inputVideo" } }, "Output": { "KinesisDataStream": { "Arn": "arn:aws:kinesis:us-east-1:nnnnnnnnnnnn:stream/outputData" } }, "RoleArn": "arn:aws:iam::nnnnnnnnnnn:role/roleWithKinesisPermission", "Settings": { "FaceSearch": { "CollectionId": "collection-with-100-faces", "FaceMatchThreshold": 85.5 } } } ``` The following is an example response from `CreateStreamProcessor`. ``` { “StreamProcessorArn”: “arn:aws:rekognition:us-east-1:nnnnnnnnnnnn:streamprocessor/streamProcessorForCam” } ``` ## Starting the Amazon Rekognition Video face search stream processor You start analyzing streaming video by calling [StartStreamProcessor](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartStreamProcessor.html) with the stream processor name that you specified in `CreateStreamProcessor`. The following is a JSON example for the `StartStreamProcessor` request. ``` { "Name": "streamProcessorForCam" } ``` If the stream processor successfully starts, an HTTP 200 response is returned, along with an empty JSON body. ## Using stream processors for face searching (Java V2 example) The following example code shows how to call various stream processor operations, such as [CreateStreamProcessor](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_CreateStreamProcessor.html) and [StartStreamProcessor](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartStreamProcessor.html), using the AWS SDK for Java version 2. This code is taken from the AWS Documentation SDK examples GitHub repository. See the full example [here](https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/javav2/example_code/rekognition/src/main/java/com/example/rekognition/CreateStreamProcessor.java). ``` import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.rekognition.RekognitionClient; import software.amazon.awssdk.services.rekognition.model.CreateStreamProcessorRequest; import software.amazon.awssdk.services.rekognition.model.CreateStreamProcessorResponse; import software.amazon.awssdk.services.rekognition.model.FaceSearchSettings; import software.amazon.awssdk.services.rekognition.model.KinesisDataStream; import software.amazon.awssdk.services.rekognition.model.KinesisVideoStream; import software.amazon.awssdk.services.rekognition.model.ListStreamProcessorsRequest; import software.amazon.awssdk.services.rekognition.model.ListStreamProcessorsResponse; import software.amazon.awssdk.services.rekognition.model.RekognitionException; import software.amazon.awssdk.services.rekognition.model.StreamProcessor; import software.amazon.awssdk.services.rekognition.model.StreamProcessorInput; import software.amazon.awssdk.services.rekognition.model.StreamProcessorSettings; import software.amazon.awssdk.services.rekognition.model.StreamProcessorOutput; import software.amazon.awssdk.services.rekognition.model.StartStreamProcessorRequest; import software.amazon.awssdk.services.rekognition.model.DescribeStreamProcessorRequest; import software.amazon.awssdk.services.rekognition.model.DescribeStreamProcessorResponse; /** * Before running this Java V2 code example, set up your development * environment, including your credentials. *

* For more information, see the following documentation topic: *

* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html */ public class CreateStreamProcessor { public static void main(String[] args) { final String usage = """ Usage: Where: role - The ARN of the AWS Identity and Access Management (IAM) role to use. \s kinInputStream - The ARN of the Kinesis video stream.\s kinOutputStream - The ARN of the Kinesis data stream.\s collectionName - The name of the collection to use that contains content. \s StreamProcessorName - The name of the Stream Processor. \s """; if (args.length != 5) { System.out.println(usage); System.exit(1); } String role = args[0]; String kinInputStream = args[1]; String kinOutputStream = args[2]; String collectionName = args[3]; String streamProcessorName = args[4]; Region region = Region.US_EAST_1; RekognitionClient rekClient = RekognitionClient.builder() .region(region) .build(); processCollection(rekClient, streamProcessorName, kinInputStream, kinOutputStream, collectionName, role); startSpecificStreamProcessor(rekClient, streamProcessorName); listStreamProcessors(rekClient); describeStreamProcessor(rekClient, streamProcessorName); deleteSpecificStreamProcessor(rekClient, streamProcessorName); } public static void listStreamProcessors(RekognitionClient rekClient) { ListStreamProcessorsRequest request = ListStreamProcessorsRequest.builder() .maxResults(15) .build(); ListStreamProcessorsResponse listStreamProcessorsResult = rekClient.listStreamProcessors(request); for (StreamProcessor streamProcessor : listStreamProcessorsResult.streamProcessors()) { System.out.println("StreamProcessor name - " + streamProcessor.name()); System.out.println("Status - " + streamProcessor.status()); } } private static void describeStreamProcessor(RekognitionClient rekClient, String StreamProcessorName) { DescribeStreamProcessorRequest streamProcessorRequest = DescribeStreamProcessorRequest.builder() .name(StreamProcessorName) .build(); DescribeStreamProcessorResponse describeStreamProcessorResult = rekClient .describeStreamProcessor(streamProcessorRequest); System.out.println("Arn - " + describeStreamProcessorResult.streamProcessorArn()); System.out.println("Input kinesisVideo stream - " + describeStreamProcessorResult.input().kinesisVideoStream().arn()); System.out.println("Output kinesisData stream - " + describeStreamProcessorResult.output().kinesisDataStream().arn()); System.out.println("RoleArn - " + describeStreamProcessorResult.roleArn()); System.out.println( "CollectionId - " + describeStreamProcessorResult.settings().faceSearch().collectionId()); System.out.println("Status - " + describeStreamProcessorResult.status()); System.out.println("Status message - " + describeStreamProcessorResult.statusMessage()); System.out.println("Creation timestamp - " + describeStreamProcessorResult.creationTimestamp()); System.out.println("Last update timestamp - " + describeStreamProcessorResult.lastUpdateTimestamp()); } private static void startSpecificStreamProcessor(RekognitionClient rekClient, String StreamProcessorName) { try { StartStreamProcessorRequest streamProcessorRequest = StartStreamProcessorRequest.builder() .name(StreamProcessorName) .build(); rekClient.startStreamProcessor(streamProcessorRequest); System.out.println("Stream Processor " + StreamProcessorName + " started."); } catch (RekognitionException e) { System.out.println(e.getMessage()); System.exit(1); } } private static void processCollection(RekognitionClient rekClient, String StreamProcessorName, String kinInputStream, String kinOutputStream, String collectionName, String role) { try { KinesisVideoStream videoStream = KinesisVideoStream.builder() .arn(kinInputStream) .build(); KinesisDataStream dataStream = KinesisDataStream.builder() .arn(kinOutputStream) .build(); StreamProcessorOutput processorOutput = StreamProcessorOutput.builder() .kinesisDataStream(dataStream) .build(); StreamProcessorInput processorInput = StreamProcessorInput.builder() .kinesisVideoStream(videoStream) .build(); FaceSearchSettings searchSettings = FaceSearchSettings.builder() .faceMatchThreshold(75f) .collectionId(collectionName) .build(); StreamProcessorSettings processorSettings = StreamProcessorSettings.builder() .faceSearch(searchSettings) .build(); CreateStreamProcessorRequest processorRequest = CreateStreamProcessorRequest.builder() .name(StreamProcessorName) .input(processorInput) .output(processorOutput) .roleArn(role) .settings(processorSettings) .build(); CreateStreamProcessorResponse response = rekClient.createStreamProcessor(processorRequest); System.out.println("The ARN for the newly create stream processor is " + response.streamProcessorArn()); } catch (RekognitionException e) { System.out.println(e.getMessage()); System.exit(1); } } private static void deleteSpecificStreamProcessor(RekognitionClient rekClient, String StreamProcessorName) { rekClient.stopStreamProcessor(a -> a.name(StreamProcessorName)); rekClient.deleteStreamProcessor(a -> a.name(StreamProcessorName)); System.out.println("Stream Processor " + StreamProcessorName + " deleted."); } } ``` ## Using stream processors for face searching (Java V1 example) The following example code shows how to call various stream processor operations, such as [CreateStreamProcessor](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_CreateStreamProcessor.html) and [StartStreamProcessor](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartStreamProcessor.html), using Java V1. The example includes a stream processor manager class (StreamManager) that provides methods to call stream processor operations. The starter class (Starter) creates a StreamManager object and calls various operations. **To configure the example:** 1. Set the values of the Starter class member fields to your desired values. 1. In the Starter class function `main`, uncomment the desired function call. ### Starter class ``` //Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. //PDX-License-Identifier: MIT-0 (For details, see https://github.com/awsdocs/amazon-rekognition-developer-guide/blob/master/LICENSE-SAMPLECODE.) // Starter class. Use to create a StreamManager class // and call stream processor operations. package com.amazonaws.samples; import com.amazonaws.samples.*; public class Starter { public static void main(String[] args) { String streamProcessorName="Stream Processor Name"; String kinesisVideoStreamArn="Kinesis Video Stream Arn"; String kinesisDataStreamArn="Kinesis Data Stream Arn"; String roleArn="Role Arn"; String collectionId="Collection ID"; Float matchThreshold=50F; try { StreamManager sm= new StreamManager(streamProcessorName, kinesisVideoStreamArn, kinesisDataStreamArn, roleArn, collectionId, matchThreshold); //sm.createStreamProcessor(); //sm.startStreamProcessor(); //sm.deleteStreamProcessor(); //sm.deleteStreamProcessor(); //sm.stopStreamProcessor(); //sm.listStreamProcessors(); //sm.describeStreamProcessor(); } catch(Exception e){ System.out.println(e.getMessage()); } } } ``` ### StreamManager class ``` //Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. //PDX-License-Identifier: MIT-0 (For details, see https://github.com/awsdocs/amazon-rekognition-developer-guide/blob/master/LICENSE-SAMPLECODE.) // Stream manager class. Provides methods for calling // Stream Processor operations. package com.amazonaws.samples; import com.amazonaws.services.rekognition.AmazonRekognition; import com.amazonaws.services.rekognition.AmazonRekognitionClientBuilder; import com.amazonaws.services.rekognition.model.CreateStreamProcessorRequest; import com.amazonaws.services.rekognition.model.CreateStreamProcessorResult; import com.amazonaws.services.rekognition.model.DeleteStreamProcessorRequest; import com.amazonaws.services.rekognition.model.DeleteStreamProcessorResult; import com.amazonaws.services.rekognition.model.DescribeStreamProcessorRequest; import com.amazonaws.services.rekognition.model.DescribeStreamProcessorResult; import com.amazonaws.services.rekognition.model.FaceSearchSettings; import com.amazonaws.services.rekognition.model.KinesisDataStream; import com.amazonaws.services.rekognition.model.KinesisVideoStream; import com.amazonaws.services.rekognition.model.ListStreamProcessorsRequest; import com.amazonaws.services.rekognition.model.ListStreamProcessorsResult; import com.amazonaws.services.rekognition.model.StartStreamProcessorRequest; import com.amazonaws.services.rekognition.model.StartStreamProcessorResult; import com.amazonaws.services.rekognition.model.StopStreamProcessorRequest; import com.amazonaws.services.rekognition.model.StopStreamProcessorResult; import com.amazonaws.services.rekognition.model.StreamProcessor; import com.amazonaws.services.rekognition.model.StreamProcessorInput; import com.amazonaws.services.rekognition.model.StreamProcessorOutput; import com.amazonaws.services.rekognition.model.StreamProcessorSettings; public class StreamManager { private String streamProcessorName; private String kinesisVideoStreamArn; private String kinesisDataStreamArn; private String roleArn; private String collectionId; private float matchThreshold; private AmazonRekognition rekognitionClient; public StreamManager(String spName, String kvStreamArn, String kdStreamArn, String iamRoleArn, String collId, Float threshold){ streamProcessorName=spName; kinesisVideoStreamArn=kvStreamArn; kinesisDataStreamArn=kdStreamArn; roleArn=iamRoleArn; collectionId=collId; matchThreshold=threshold; rekognitionClient=AmazonRekognitionClientBuilder.defaultClient(); } public void createStreamProcessor() { //Setup input parameters KinesisVideoStream kinesisVideoStream = new KinesisVideoStream().withArn(kinesisVideoStreamArn); StreamProcessorInput streamProcessorInput = new StreamProcessorInput().withKinesisVideoStream(kinesisVideoStream); KinesisDataStream kinesisDataStream = new KinesisDataStream().withArn(kinesisDataStreamArn); StreamProcessorOutput streamProcessorOutput = new StreamProcessorOutput().withKinesisDataStream(kinesisDataStream); FaceSearchSettings faceSearchSettings = new FaceSearchSettings().withCollectionId(collectionId).withFaceMatchThreshold(matchThreshold); StreamProcessorSettings streamProcessorSettings = new StreamProcessorSettings().withFaceSearch(faceSearchSettings); //Create the stream processor CreateStreamProcessorResult createStreamProcessorResult = rekognitionClient.createStreamProcessor( new CreateStreamProcessorRequest().withInput(streamProcessorInput).withOutput(streamProcessorOutput) .withSettings(streamProcessorSettings).withRoleArn(roleArn).withName(streamProcessorName)); //Display result System.out.println("Stream Processor " + streamProcessorName + " created."); System.out.println("StreamProcessorArn - " + createStreamProcessorResult.getStreamProcessorArn()); } public void startStreamProcessor() { StartStreamProcessorResult startStreamProcessorResult = rekognitionClient.startStreamProcessor(new StartStreamProcessorRequest().withName(streamProcessorName)); System.out.println("Stream Processor " + streamProcessorName + " started."); } public void stopStreamProcessor() { StopStreamProcessorResult stopStreamProcessorResult = rekognitionClient.stopStreamProcessor(new StopStreamProcessorRequest().withName(streamProcessorName)); System.out.println("Stream Processor " + streamProcessorName + " stopped."); } public void deleteStreamProcessor() { DeleteStreamProcessorResult deleteStreamProcessorResult = rekognitionClient .deleteStreamProcessor(new DeleteStreamProcessorRequest().withName(streamProcessorName)); System.out.println("Stream Processor " + streamProcessorName + " deleted."); } public void describeStreamProcessor() { DescribeStreamProcessorResult describeStreamProcessorResult = rekognitionClient .describeStreamProcessor(new DescribeStreamProcessorRequest().withName(streamProcessorName)); //Display various stream processor attributes. System.out.println("Arn - " + describeStreamProcessorResult.getStreamProcessorArn()); System.out.println("Input kinesisVideo stream - " + describeStreamProcessorResult.getInput().getKinesisVideoStream().getArn()); System.out.println("Output kinesisData stream - " + describeStreamProcessorResult.getOutput().getKinesisDataStream().getArn()); System.out.println("RoleArn - " + describeStreamProcessorResult.getRoleArn()); System.out.println( "CollectionId - " + describeStreamProcessorResult.getSettings().getFaceSearch().getCollectionId()); System.out.println("Status - " + describeStreamProcessorResult.getStatus()); System.out.println("Status message - " + describeStreamProcessorResult.getStatusMessage()); System.out.println("Creation timestamp - " + describeStreamProcessorResult.getCreationTimestamp()); System.out.println("Last update timestamp - " + describeStreamProcessorResult.getLastUpdateTimestamp()); } public void listStreamProcessors() { ListStreamProcessorsResult listStreamProcessorsResult = rekognitionClient.listStreamProcessors(new ListStreamProcessorsRequest().withMaxResults(100)); //List all stream processors (and state) returned from Rekognition for (StreamProcessor streamProcessor : listStreamProcessorsResult.getStreamProcessors()) { System.out.println("StreamProcessor name - " + streamProcessor.getName()); System.out.println("Status - " + streamProcessor.getStatus()); } } } ``` # Reading streaming video analysis results **Note** Amazon Rekognition Streaming Video Analysis will no longer be open to new customers starting April 30, 2026. If you would like to use Streaming Video Analysis, sign up prior to that date. Existing customers for accounts that have used this feature within the last 12 months can continue to use the service as normal. For more information, see [Rekognition Streaming Video Analysis availability change](https://docs.aws.amazon.com/rekognition/latest/dg/rekognition-streaming-video-analysis-availability-change.html). You can use the Amazon Kinesis Data Streams Client Library to consume analysis results that are sent to the Amazon Kinesis Data Streams output stream. For more information, see [Reading Data from a Kinesis Data Stream](https://docs.aws.amazon.com/streams/latest/dev/building-consumers.html). Amazon Rekognition Video places a JSON frame record for each analyzed frame into the Kinesis output stream. Amazon Rekognition Video doesn't analyze every frame that's passed to it through the Kinesis video stream. A frame record that's sent to a Kinesis data stream contains information about which Kinesis video stream fragment the frame is in, where the frame is in the fragment, and faces that are recognized in the frame. It also includes status information for the stream processor. For more information, see [Understanding the Kinesis face recognition JSON frame record](streaming-video-kinesis-output-reference.md). The Amazon Kinesis Video Streams Parser Library contains example tests that consume Amazon Rekognition Video results and integrates it with the original Kinesis video stream. For more information, see [Displaying Rekognition results with Kinesis Video Streams locally](displaying-rekognition-results-locally.md). Amazon Rekognition Video streams Amazon Rekognition Video analysis information to the Kinesis data stream. The following is a JSON example for a single record. ``` { "InputInformation": { "KinesisVideo": { "StreamArn": "arn:aws:kinesisvideo:us-west-2:nnnnnnnnnnnn:stream/stream-name", "FragmentNumber": "91343852333289682796718532614445757584843717598", "ServerTimestamp": 1510552593.455, "ProducerTimestamp": 1510552593.193, "FrameOffsetInSeconds": 2 } }, "StreamProcessorInformation": { "Status": "RUNNING" }, "FaceSearchResponse": [ { "DetectedFace": { "BoundingBox": { "Height": 0.075, "Width": 0.05625, "Left": 0.428125, "Top": 0.40833333 }, "Confidence": 99.975174, "Landmarks": [ { "X": 0.4452057, "Y": 0.4395594, "Type": "eyeLeft" }, { "X": 0.46340984, "Y": 0.43744427, "Type": "eyeRight" }, { "X": 0.45960626, "Y": 0.4526856, "Type": "nose" }, { "X": 0.44958648, "Y": 0.4696949, "Type": "mouthLeft" }, { "X": 0.46409217, "Y": 0.46704912, "Type": "mouthRight" } ], "Pose": { "Pitch": 2.9691637, "Roll": -6.8904796, "Yaw": 23.84388 }, "Quality": { "Brightness": 40.592964, "Sharpness": 96.09616 } }, "MatchedFaces": [ { "Similarity": 88.863960, "Face": { "BoundingBox": { "Height": 0.557692, "Width": 0.749838, "Left": 0.103426, "Top": 0.206731 }, "FaceId": "ed1b560f-d6af-5158-989a-ff586c931545", "Confidence": 99.999201, "ImageId": "70e09693-2114-57e1-807c-50b6d61fa4dc", "ExternalImageId": "matchedImage.jpeg" } } ] } ] } ``` In the JSON example, note the following: + **InputInformation** – Information about the Kinesis video stream that's used to stream video into Amazon Rekognition Video. For more information, see [InputInformation](streaming-video-kinesis-output-reference.md#streaming-video-kinesis-output-reference-inputinformation). + **StreamProcessorInformation** – Status information for the Amazon Rekognition Video stream processor. The only possible value for the `Status` field is RUNNING. For more information, see [StreamProcessorInformation](streaming-video-kinesis-output-reference-streamprocessorinformation.md). + **FaceSearchResponse** – Contains information about faces in the streaming video that match faces in the input collection. [FaceSearchResponse](streaming-video-kinesis-output-reference.md#streaming-video-kinesis-output-reference-facesearchresponse) contains a [DetectedFace](streaming-video-kinesis-output-reference.md#streaming-video-kinesis-output-reference-detectedface) object, which is a face that was detected in the analyzed video frame. For each detected face, the array `MatchedFaces` contains an array of matching face objects ([MatchedFace](streaming-video-kinesis-output-reference.md#streaming-video-kinesis-output-reference-facematch)) found in the input collection, along with a similarity score. ## Mapping the Kinesis video stream to the Kinesis data stream You might want to map the Kinesis video stream frames to the analyzed frames that are sent to the Kinesis data stream. For example, during the display of a streaming video, you might want to display boxes around the faces of recognized people. The bounding box coordinates are sent as part of the Kinesis Face Recognition Record to the Kinesis data stream. To display the bounding box correctly, you need to map the time information that's sent with the Kinesis Face Recognition Record with the corresponding frames in the source Kinesis video stream. The technique that you use to map the Kinesis video stream to the Kinesis data stream depends on if you're streaming live media (such as a live streaming video), or if you're streaming archived media (such as a stored video). ### Mapping when you're streaming live media **To map a Kinesis video stream frame to a Kinesis data stream frame** 1. Set the input parameter `FragmentTimeCodeType` of the [PutMedia](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/API_dataplane_PutMedia.html) operation to `RELATIVE`. 1. Call `PutMedia` to deliver live media into the Kinesis video stream. 1. When you receive a Kinesis Face Recognition Record from the Kinesis data stream, store the values of `ProducerTimestamp` and `FrameOffsetInSeconds` from the [KinesisVideo](streaming-video-kinesis-output-reference.md#streaming-video-kinesis-output-reference-kinesisvideostreams-kinesisvideo) field. 1. Calculate the time stamp that corresponds to the Kinesis video stream frame by adding the `ProducerTimestamp` and `FrameOffsetInSeconds` field values together. ### Mapping when you're streaming archived media **To map a Kinesis video stream frame to a Kinesis data stream frame** 1. Call [PutMedia](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/API_dataplane_PutMedia.html) to deliver archived media into the Kinesis video stream. 1. When you receive an `Acknowledgement` object from the `PutMedia` operation response, store the `FragmentNumber` field value from the [Payload](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/API_dataplane_PutMedia.html#API_dataplane_PutMedia_ResponseSyntax) field. `FragmentNumber` is the fragment number for the MKV cluster. 1. When you receive a Kinesis Face Recognition Record from the Kinesis data stream, store the `FrameOffsetInSeconds` field value from the [KinesisVideo](streaming-video-kinesis-output-reference.md#streaming-video-kinesis-output-reference-kinesisvideostreams-kinesisvideo) field. 1. Calculate the mapping by using the `FrameOffsetInSeconds` and `FragmentNumber` values that you stored in steps 2 and 3. `FrameOffsetInSeconds` is the offset into the fragment with the specific `FragmentNumber` that's sent to the Amazon Kinesis data stream. For more information about getting the video frames for a given fragment number, see [Amazon Kinesis Video Streams Archived Media](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/API_Operations_Amazon_Kinesis_Video_Streams_Archived_Media.html). # Displaying Rekognition results with Kinesis Video Streams locally **Note** Amazon Rekognition Streaming Video Analysis will no longer be open to new customers starting April 30, 2026. If you would like to use Streaming Video Analysis, sign up prior to that date. Existing customers for accounts that have used this feature within the last 12 months can continue to use the service as normal. For more information, see [Rekognition Streaming Video Analysis availability change](https://docs.aws.amazon.com/rekognition/latest/dg/rekognition-streaming-video-analysis-availability-change.html). You can see the results of Amazon Rekognition Video displayed in your feed from Amazon Kinesis Video Streams using the Amazon Kinesis Video Streams Parser Library’s example tests provided at [KinesisVideo - Rekognition Examples](https://github.com/aws/amazon-kinesis-video-streams-parser-library#kinesisvideo---rekognition-examples). The `KinesisVideoRekognitionIntegrationExample` displays bounding boxes over detected faces and renders the video locally through JFrame. This process assumes you have successfully connected a media input from a device camera to a Kinesis video stream and started an Amazon Rekognition Stream Processor. For more information, see [Streaming using a GStreamer plugin](streaming-using-gstreamer-plugin.md). ## Step 1: Installing Kinesis Video Streams Parser Library To create a directory and download the Github repository, run the following command: ``` $ git clone https://github.com/aws/amazon-kinesis-video-streams-parser-library.git ``` Navigate to the library directory and run the following Maven command to perform a clean installation: ``` $ mvn clean install ``` ## Step 2: Configuring the Kinesis Video Streams and Rekognition integration example test Open the `KinesisVideoRekognitionIntegrationExampleTest.java` file. Remove the `@Ignore` right after the class header. Populate the data fields with the information from your Amazon Kinesis and Amazon Rekognition resources. For more information, see [Setting up your Amazon Rekognition Video and Amazon Kinesis resources](setting-up-your-amazon-rekognition-streaming-video-resources.md). If you are streaming video to your Kinesis video stream, remove the `inputStream` parameter. See the following code example: ``` RekognitionInput rekognitionInput = RekognitionInput.builder() .kinesisVideoStreamArn("arn:aws:kinesisvideo:us-east-1:123456789012:stream/rekognition-test-video-stream") .kinesisDataStreamArn("arn:aws:kinesis:us-east-1:123456789012:stream/AmazonRekognition-rekognition-test-data-stream") .streamingProcessorName("rekognition-test-stream-processor") // Refer how to add face collection : // https://docs.aws.amazon.com/rekognition/latest/dg/add-faces-to-collection-procedure.html .faceCollectionId("rekognition-test-face-collection") .iamRoleArn("rekognition-test-IAM-role") .matchThreshold(0.95f) .build(); KinesisVideoRekognitionIntegrationExample example = KinesisVideoRekognitionIntegrationExample.builder() .region(Regions.US_EAST_1) .kvsStreamName("rekognition-test-video-stream") .kdsStreamName("AmazonRekognition-rekognition-test-data-stream") .rekognitionInput(rekognitionInput) .credentialsProvider(new ProfileCredentialsProvider()) // NOTE: Comment out or delete the inputStream parameter if you are streaming video, otherwise // the test will use a sample video. //.inputStream(TestResourceUtil.getTestInputStream("bezos_vogels.mkv")) .build(); ``` ## Step 3: Running the Kinesis Video Streams and Rekognition integration example test Ensure that your Kinesis video stream is receiving media input if you are streaming to it and start analyzing your stream with an Amazon Rekognition Video Stream Processor running. For more information, see [Overview of Amazon Rekognition Video stream processor operations](streaming-video.md#using-rekognition-video-stream-processor). Run the `KinesisVideoRekognitionIntegrationExampleTest` class as a JUnit test. After a short delay, a new window opens with a video feed from your Kinesis video stream with bounding boxes drawn over detected faces. **Note** The faces in the collection used in this example must have External Image Id (the file name) specified in this format in order for bounding box labels to display meaningful text: PersonName1-Trusted, PersonName2-Intruder, PersonName3-Neutral, etc. The labels can also be color-coded and are customizable in the FaceType.java file. # Understanding the Kinesis face recognition JSON frame record **Note** Amazon Rekognition Streaming Video Analysis will no longer be open to new customers starting April 30, 2026. If you would like to use Streaming Video Analysis, sign up prior to that date. Existing customers for accounts that have used this feature within the last 12 months can continue to use the service as normal. For more information, see [Rekognition Streaming Video Analysis availability change](https://docs.aws.amazon.com/rekognition/latest/dg/rekognition-streaming-video-analysis-availability-change.html). Amazon Rekognition Video can recognize faces in a streaming video. For each analyzed frame, Amazon Rekognition Video outputs a JSON frame record to a Kinesis data stream. Amazon Rekognition Video doesn't analyze every frame that's passed to it through the Kinesis video stream. The JSON frame record contains information about the input and output stream, the status of the stream processor, and information about faces that are recognized in the analyzed frame. This section contains reference information for the JSON frame record. The following is the JSON syntax for a Kinesis data stream record. For more information, see [Working with streaming video events](streaming-video.md). **Note** The Amazon Rekognition Video API works by comparing the faces in your input stream to a collection of faces, and returning the closest found matches, along with a similarity score. ``` { "InputInformation": { "KinesisVideo": { "StreamArn": "string", "FragmentNumber": "string", "ProducerTimestamp": number, "ServerTimestamp": number, "FrameOffsetInSeconds": number } }, "StreamProcessorInformation": { "Status": "RUNNING" }, "FaceSearchResponse": [ { "DetectedFace": { "BoundingBox": { "Width": number, "Top": number, "Height": number, "Left": number }, "Confidence": number, "Landmarks": [ { "Type": "string", "X": number, "Y": number } ], "Pose": { "Pitch": number, "Roll": number, "Yaw": number }, "Quality": { "Brightness": number, "Sharpness": number } }, "MatchedFaces": [ { "Similarity": number, "Face": { "BoundingBox": { "Width": number, "Top": number, "Height": number, "Left": number }, "Confidence": number, "ExternalImageId": "string", "FaceId": "string", "ImageId": "string" } } ] } ] } ``` ## JSON record The JSON record includes information about a frame that's processed by Amazon Rekognition Video. The record includes information about the streaming video, the status for the analyzed frame, and information about faces that are recognized in the frame. **InputInformation** Information about the Kinesis video stream that's used to stream video into Amazon Rekognition Video. Type: [InputInformation](#streaming-video-kinesis-output-reference-inputinformation) object **StreamProcessorInformation** Information about the Amazon Rekognition Video stream processor. This includes status information for the current status of the stream processor. Type: [StreamProcessorInformation](streaming-video-kinesis-output-reference-streamprocessorinformation.md) object **FaceSearchResponse** Information about the faces detected in a streaming video frame and the matching faces found in the input collection. Type: [FaceSearchResponse](#streaming-video-kinesis-output-reference-facesearchresponse) object array ## InputInformation Information about a source video stream that's used by Amazon Rekognition Video. For more information, see [Working with streaming video events](streaming-video.md). **KinesisVideo** Type: [KinesisVideo](#streaming-video-kinesis-output-reference-kinesisvideostreams-kinesisvideo) object ## KinesisVideo Information about the Kinesis video stream that streams the source video into Amazon Rekognition Video. For more information, see [Working with streaming video events](streaming-video.md). **StreamArn** The Amazon Resource Name (ARN) of the Kinesis video stream. Type: String **FragmentNumber** The fragment of streaming video that contains the frame that this record represents. Type: String **ProducerTimestamp** The producer-side Unix time stamp of the fragment. For more information, see [PutMedia](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/API_dataplane_PutMedia.html). Type: Number **ServerTimestamp** The server-side Unix time stamp of the fragment. For more information, see [PutMedia](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/API_dataplane_PutMedia.html). Type: Number **FrameOffsetInSeconds** The offset of the frame (in seconds) inside the fragment. Type: Number # StreamProcessorInformation **Note** Amazon Rekognition Streaming Video Analysis will no longer be open to new customers starting April 30, 2026. If you would like to use Streaming Video Analysis, sign up prior to that date. Existing customers for accounts that have used this feature within the last 12 months can continue to use the service as normal. For more information, see [Rekognition Streaming Video Analysis availability change](https://docs.aws.amazon.com/rekognition/latest/dg/rekognition-streaming-video-analysis-availability-change.html). Status information about the stream processor. **Status** The current status of the stream processor. The one possible value is RUNNING. Type: String ## FaceSearchResponse Information about a face detected in a streaming video frame and the faces in a collection that match the detected face. You specify the collection in a call to [CreateStreamProcessor](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_CreateStreamProcessor.html). For more information, see [Working with streaming video events](streaming-video.md). **DetectedFace** Face details for a face detected in an analyzed video frame. Type: [DetectedFace](#streaming-video-kinesis-output-reference-detectedface) object **MatchedFaces** An array of face details for faces in a collection that matches the face detected in `DetectedFace`. Type: [MatchedFace](#streaming-video-kinesis-output-reference-facematch) object array ## DetectedFace Information about a face that's detected in a streaming video frame. Matching faces in the input collection are available in [MatchedFace](#streaming-video-kinesis-output-reference-facematch) object field. **BoundingBox** The bounding box coordinates for a face that's detected within an analyzed video frame. The BoundingBox object has the same properties as the BoundingBox object that's used for image analysis. Type: [BoundingBox](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_BoundingBox.html) object **Confidence** The confidence level (1-100) that Amazon Rekognition Video has that the detected face is actually a face. 1 is the lowest confidence, 100 is the highest. Type: Number **Landmarks** An array of facial landmarks. Type: [Landmark](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_Landmark.html) object array **Pose** Indicates the pose of the face as determined by its pitch, roll, and yaw. Type: [Pose](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_Pose.html) object **Quality** Identifies face image brightness and sharpness. Type: [ImageQuality](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_ImageQuality.html) object ## MatchedFace Information about a face that matches a face detected in an analyzed video frame. **Face** Face match information for a face in the input collection that matches the face in the [DetectedFace](#streaming-video-kinesis-output-reference-detectedface) object. Type: [Face](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_Face.html) object **Similarity** The level of confidence (1-100) that the faces match. 1 is the lowest confidence, 100 is the highest. Type: Number