# Custom Labels Examples
This section contains examples that show you how you make use Amazon Rekognition Custom Labels's capabilities.
| Example | Description |
| --- | --- |
| [Improving a model with Model feedback](ex-feedback.md) | Shows how to improve a model using human verification to create a new training dataset. |
| [Amazon Rekognition Custom Labels demonstration](ex-custom-labels-demo.md) | Demonstration of a user interface that displays the results of a call to `DetectCustomLabels`. |
| [Detecting Custom Labels in videos](ex-video-extraction.md) | Shows how you can use `DetectCustomLabels` with frames extracted from a video. |
| [Analyzing images with an AWS Lambda function](ex-lambda.md) | Shows how you can use `DetectCustomLabels` with a Lambda function. |
| [Creating a manifest file from a CSV file](ex-csv-manifest.md) | Shows how to use a CSV file to create a manifest file suitable for finding [objects, scenes, and concepts](understanding-custom-labels.md#tm-classification) associated with an entire image (classification). |
# Improving a model with Model feedback
The Model Feedback solution enables you to give feedback on your model's predictions and make improvements by using human verification. Depending on the use case, you can be successful with a training dataset that has only a few images. A larger annotated training set might be required to build a more accurate model. Using the Model Feedback solution, you can create a larger dataset through model assistance.
To install and configure the Model Feedback solution, see [Model Feedback Solution](https://github.com/aws-samples/amazon-rekognition-custom-labels-feedback-solution).
The workflow for continuous model improvement is as follows:
1. Train the first version of your model (possibly with a small training dataset).
1. Provide an unannotated dataset for the Model Feedback solution.
1. The Model Feedback solution uses the current model. It starts human verification jobs to annotate a new dataset.
1. Based on human feedback, the Model Feedback solution generates a manifest file that you use to create a new model.
# Amazon Rekognition Custom Labels demonstration
The Amazon Rekognition Custom Labels Demonstration shows a user interface that analyzes images from your local computer by using the [DetectCustomLabels](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_DetectCustomLabels) API.
The application shows you information about the Amazon Rekognition Custom Labels models in your AWS account. After you select a running model, you can analyze an image from your local computer. If necessary, you can start a model. You can also stop a running model. The application shows integration with other AWS Services such as Amazon Cognito, Amazon S3, and Amazon CloudFront.
For more information, see [Amazon Rekognition Custom Labels Demo](https://github.com/aws-samples/amazon-rekognition-custom-labels-demo).
# Detecting Custom Labels in videos
The following example shows how you can use `DetectCustomLabels` with frames extracted from a video. The code has been tested with video files in *mov* and *mp4* format.
**Using `DetectCustomLabels` with captured frames**
1. If you haven't already done so, install and configure the AWS CLI and the AWS SDKs. For more information, see [Step 4: Set up the AWS CLI and AWS SDKs](su-awscli-sdk.md).
1. Make sure you have `rekognition:DetectCustomLabels` and `AmazonS3ReadOnlyAccess` permissions. For more information, see [Step 4: Set up the AWS CLI and AWS SDKs](su-awscli-sdk.md).
1. Use the following example code. Change the value of `videoFile` to the name of a video file. Change the value of `projectVersionArn` to the Amazon Resource Name (ARN) of your Amazon Rekognition Custom Labels model.
```
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Purpose
Shows how to analyze a local video with an Amazon Rekognition Custom Labels model.
"""
import argparse
import logging
import json
import math
import cv2
import boto3
from botocore.exceptions import ClientError
logger = logging.getLogger(__name__)
def analyze_video(rek_client, project_version_arn, video_file):
"""
Analyzes a local video file with an Amazon Rekognition Custom Labels model.
Creates a results JSON file based on the name of the supplied video file.
:param rek_client: A Boto3 Amazon Rekognition client.
:param project_version_arn: The ARN of the Custom Labels model that you want to use.
:param video_file: The video file that you want to analyze.
"""
custom_labels = []
cap = cv2.VideoCapture(video_file)
frame_rate = cap.get(5) # Frame rate.
while cap.isOpened():
frame_id = cap.get(1) # Current frame number.
print(f"Processing frame id: {frame_id}")
ret, frame = cap.read()
if ret is not True:
break
if frame_id % math.floor(frame_rate) == 0:
has_frame, image_bytes = cv2.imencode(".jpg", frame)
if has_frame:
response = rek_client.detect_custom_labels(
Image={
'Bytes': image_bytes.tobytes(),
},
ProjectVersionArn=project_version_arn
)
for elabel in response["CustomLabels"]:
elabel["Timestamp"] = (frame_id/frame_rate)*1000
custom_labels.append(elabel)
print(custom_labels)
with open(video_file + ".json", "w", encoding="utf-8") as f:
f.write(json.dumps(custom_labels))
cap.release()
def add_arguments(parser):
"""
Adds command line arguments to the parser.
:param parser: The command line parser.
"""
parser.add_argument(
"project_version_arn", help="The ARN of the model that you want to use."
)
parser.add_argument(
"video_file", help="The local path to the video that you want to analyze."
)
def main():
logging.basicConfig(level=logging.INFO,
format="%(levelname)s: %(message)s")
try:
# Get command line arguments.
parser = argparse.ArgumentParser(usage=argparse.SUPPRESS)
add_arguments(parser)
args = parser.parse_args()
session = boto3.Session(profile_name='custom-labels-access')
rekognition_client = session.client("rekognition")
analyze_video(rekognition_client,
args.project_version_arn, args.video_file)
except ClientError as err:
print(f"Couldn't analyze video: {err}")
if __name__ == "__main__":
main()
```
# Analyzing images with an AWS Lambda function
AWS Lambda is a compute service that lets you run code without provisioning or managing servers. For example, you can analyze images submitted from a mobile application without having to create a server to host the application code. The following instructions show how to create a Lambda function in Python that calls [DetectCustomLabels](https://docs.aws.amazon.com/rekognition/latest/APIReference/API_DetectCustomLabels). The function analyzes a supplied image and returns a list of labels found in the image. The instructions include example Python code showing how to call the Lambda function with an image in an Amazon S3 bucket, or an image supplied from a local computer.
**Topics**
+ [
## Step 1: Create an AWS Lambda function (console)
](#example-lambda-create-function)
+ [
## Step 2: (Optional) Create a layer (console)
](#example-lambda-create-layer)
+ [
## Step 3: Add Python code (console)
](#example-lambda-add-code)
+ [
## Step 4: Try your Lambda function
](#example-lambda-test)
## Step 1: Create an AWS Lambda function (console)
In this step, you create an empty AWS function and an IAM execution role that lets your function call the `DetectCustomLabels` operation. It also grants access to the Amazon S3 bucket that stores images for analysis. You also specify environment variables for the following:
+ The Amazon Rekognition Custom Labels model that you want your Lambda function to use.
+ The confidence limit that you want the model to use.
Later you add the source code and optionally a layer to the Lambda function.
**To create an AWS Lambda function (console)**
1. Sign in to the AWS Management Console and open the AWS Lambda console at [https://console.aws.amazon.com/lambda/](https://console.aws.amazon.com/lambda/).
1. Choose **Create function**. For more information, see [Create a Lambda Function with the Console](https://docs.aws.amazon.com/lambda/latest/dg/getting-started-create-function.html).
1. Choose the following options.
+ Choose **Author from scratch**.
+ Enter a value for **Function name**.
+ For **Runtime** choose **Python 3.10**.
1. Choose **Create function** to create the AWS Lambda function.
1. On the function page, Choose the **Configuration** tab.
1. On the **Environment variables** pane, choose **Edit**.
1. Add the following environment variables. For each variable choose **Add enviroment variable** and then enter the variable key and value.
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/rekognition/latest/customlabels-dg/ex-lambda.html)
1. Choose **Save** to save the environment variables.
1. On the **Permissions** pane, Under **Role name**, choose the execution role to open the role in the IAM console.
1. In the **Permissions** tab, choose **Add permissions** and then **Create inline policy**.
1. Choose **JSON** and replace the existing policy with the following policy.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": "rekognition:DetectCustomLabels",
"Resource": "*",
"Effect": "Allow",
"Sid": "DetectCustomLabels"
}
]
}
```
------
1. Choose **Next**.
1. In **Policy details**, enter a name for the policy, such as *DetectCustomLabels-access*.
1. Choose **Create policy**.
1. If you are storing images for analysis in an Amazon S3 bucket, repeat steps 10–14.
1. For step 11, use the following policy. Replace *bucket/folder path* with the Amazon S3 bucket and folder path to the images that you want to analyze.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "S3Access",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucket/folder path/*"
}
]
}
```
------
1. For step 13, choose a different policy name, such as *S3Bucket-access*.
## Step 2: (Optional) Create a layer (console)
To run this example, You don't need to do this step. The `DetectCustomLabels` operation is included in the default Lambda Python environment as part of AWS SDK for Python (Boto3). If other parts of your Lambda function need recent AWS service updates that aren't in the default Lambda Python environment, do this step to add the latest Boto3 SDK release as a layer to your function.
First, you create a .zip file archive that contains the Boto3 SDK. You then create a layer and add the .zip file archive to the layer. For more information, see [Using layers with your Lambda function](https://docs.aws.amazon.com/lambda/latest/dg/invocation-layers.html#invocation-layers-using).
**To create and add a layer (console)**
1. Open a command prompt and enter the following commands.
```
pip install boto3 --target python/.
zip boto3-layer.zip -r python/
```
1. Note the name of the zip file (boto3-layer.zip). You need it in step 6 of this procedure.
1. Open the AWS Lambda console at [https://console.aws.amazon.com/lambda/](https://console.aws.amazon.com/lambda/).
1. In the navigation pane, choose **Layers**.
1. Choose **Create layer**.
1. Enter values for **Name** and **Description**.
1. Choose **Upload a .zip file** and choose **Upload**.
1. In the dialog box, choose the .zip file archive (boto3-layer.zip) that you created in step 1 of this procedure.
1. For compatible runtimes, choose **Python 3.9**.
1. Choose **Create** to create the layer.
1. Choose the navigation pane menu icon.
1. In the navigation pane, choose **Functions**.
1. In the resources list, choose the function that you created in [Step 1: Create an AWS Lambda function (console)](#example-lambda-create-function).
1. Choose the **Code** tab.
1. In the **Layers** section, choose **Add a layer**.
1. Choose **Custom layers**.
1. In **Custom layers**, choose the layer name that you entered in step 6.
1. In **Version** choose the layer version, which should be 1.
1. Choose **Add**.
## Step 3: Add Python code (console)
In this step, you add Python code to your Lambda function by using the Lambda console code editor. The code analyzes a supplied image with `DetectCustomLabels` and returns a list of labels found in the image. The supplied image can be located in an Amazon S3 bucket or provided as byte64 encoded image bytes.
**To add Python code (console)**
1. If you're not in the Lambda console, do the following:
1. Open the AWS Lambda console at [https://console.aws.amazon.com/lambda/](https://console.aws.amazon.com/lambda/).
1. Open the Lambda function you created in [Step 1: Create an AWS Lambda function (console)](#example-lambda-create-function).
1. Choose the **Code** tab.
1. In **Code source**, replace the code in **lambda\$1function.py** with the following:
```
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Purpose
An AWS lambda function that analyzes images with an the Amazon Rekognition
Custom Labels model.
"""
import json
import base64
from os import environ
import logging
import boto3
from botocore.exceptions import ClientError
# Set up logging.
logger = logging.getLogger(__name__)
# Get the model ARN and confidence.
model_arn = environ['MODEL_ARN']
min_confidence = int(environ.get('CONFIDENCE', 50))
# Get the boto3 client.
rek_client = boto3.client('rekognition')
def lambda_handler(event, context):
"""
Lambda handler function
param: event: The event object for the Lambda function.
param: context: The context object for the lambda function.
return: The labels found in the image passed in the event
object.
"""
try:
# Determine image source.
if 'image' in event:
# Decode the image
image_bytes = event['image'].encode('utf-8')
img_b64decoded = base64.b64decode(image_bytes)
image = {'Bytes': img_b64decoded}
elif 'S3Object' in event:
image = {'S3Object':
{'Bucket': event['S3Object']['Bucket'],
'Name': event['S3Object']['Name']}
}
else:
raise ValueError(
'Invalid source. Only image base 64 encoded image bytes or S3Object are supported.')
# Analyze the image.
response = rek_client.detect_custom_labels(Image=image,
MinConfidence=min_confidence,
ProjectVersionArn=model_arn)
# Get the custom labels
labels = response['CustomLabels']
lambda_response = {
"statusCode": 200,
"body": json.dumps(labels)
}
except ClientError as err:
error_message = f"Couldn't analyze image. " + \
err.response['Error']['Message']
lambda_response = {
'statusCode': 400,
'body': {
"Error": err.response['Error']['Code'],
"ErrorMessage": error_message
}
}
logger.error("Error function %s: %s",
context.invoked_function_arn, error_message)
except ValueError as val_error:
lambda_response = {
'statusCode': 400,
'body': {
"Error": "ValueError",
"ErrorMessage": format(val_error)
}
}
logger.error("Error function %s: %s",
context.invoked_function_arn, format(val_error))
return lambda_response
```
1. Choose **Deploy** to deploy your Lambda function.
## Step 4: Try your Lambda function
In this step you use Python code on your computer to pass a local image, or an image in an Amazon S3 bucket, to your Lambda function. Images passed from a local computer must be smaller than 6291456 bytes. If your images are larger, upload the images to an Amazon S3 bucket and call the script with the Amazon S3 path to the image. For information about uploading image files to an Amazon S3 bucket, see [Uploading objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html).
Make sure you run the code in the same AWS Region in which you created the Lambda function. You can view the AWS Region for your Lambda function in the navigation bar of the function details page in the [Lambda console](https://console.aws.amazon.com/lambda/).
If the AWS Lambda function returns a timeout error, extend the timeout period for the Lambda function function, For more information, see [Configuring function timeout (console)](https://docs.aws.amazon.com/lambda/latest/dg/configuration-function-common.html#configuration-timeout-console).
For more information about invoking a Lambda function from your code, see [Invoking AWS Lambda Functions](https://docs.aws.amazon.com/lambda/latest/dg/invoking-lambda-functions.html).
**To try your Lambda function**
1. Make sure you have `lambda:InvokeFunction` permission. You can use the following policy.
You can get the ARN for your Lambda function function from the function overview in the [Lambda console](https://console.aws.amazon.com/lambda/).
To provide access, add permissions to your users, groups, or roles:
+ Users and groups in AWS IAM Identity Center:
Create a permission set. Follow the instructions in [Create a permission set](https://docs.aws.amazon.com//singlesignon/latest/userguide/howtocreatepermissionset.html) in the *AWS IAM Identity Center User Guide*.
+ Users managed in IAM through an identity provider:
Create a role for identity federation. Follow the instructions in [Create a role for a third-party identity provider (federation)](https://docs.aws.amazon.com//IAM/latest/UserGuide/id_roles_create_for-idp.html) in the *IAM User Guide*.
+ IAM users:
+ Create a role that your user can assume. Follow the instructions in [Create a role for an IAM user](https://docs.aws.amazon.com//IAM/latest/UserGuide/id_roles_create_for-user.html) in the *IAM User Guide*.
+ (Not recommended) Attach a policy directly to a user or add a user to a user group. Follow the instructions in [Adding permissions to a user (console)](https://docs.aws.amazon.com//IAM/latest/UserGuide/id_users_change-permissions.html#users_change_permissions-add-console) in the *IAM User Guide*.
1. Install and configure AWS SDK for Python. For more information, see [Step 4: Set up the AWS CLI and AWS SDKs](su-awscli-sdk.md).
1. [Start the model](rm-start.md) that you specified in step 7 of [Step 1: Create an AWS Lambda function (console)](#example-lambda-create-function) .
1. Save the following code to a file named `client.py`.
```
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""
Purpose
Test code for running the Amazon Rekognition Custom Labels Lambda
function example code.
"""
import argparse
import logging
import base64
import json
import boto3
from botocore.exceptions import ClientError
logger = logging.getLogger(__name__)
def analyze_image(function_name, image):
"""Analyzes an image with an AWS Lambda function.
:param image: The image that you want to analyze.
:return The status and classification result for
the image analysis.
"""
lambda_client = boto3.client('lambda')
lambda_payload = {}
if image.startswith('s3://'):
logger.info("Analyzing image from S3 bucket: %s", image)
bucket, key = image.replace("s3://", "").split("/", 1)
s3_object = {
'Bucket': bucket,
'Name': key
}
lambda_payload = {"S3Object": s3_object}
# Call the lambda function with the image.
else:
with open(image, 'rb') as image_file:
logger.info("Analyzing local image image: %s ", image)
image_bytes = image_file.read()
data = base64.b64encode(image_bytes).decode("utf8")
lambda_payload = {"image": data}
response = lambda_client.invoke(FunctionName=function_name,
Payload=json.dumps(lambda_payload))
return json.loads(response['Payload'].read().decode())
def add_arguments(parser):
"""
Adds command line arguments to the parser.
:param parser: The command line parser.
"""
parser.add_argument(
"function", help="The name of the AWS Lambda function that you want " \
"to use to analyze the image.")
parser.add_argument(
"image", help="The local image that you want to analyze.")
def main():
"""
Entrypoint for script.
"""
try:
logging.basicConfig(level=logging.INFO,
format="%(levelname)s: %(message)s")
# Get command line arguments.
parser = argparse.ArgumentParser(usage=argparse.SUPPRESS)
add_arguments(parser)
args = parser.parse_args()
# Get analysis results.
result = analyze_image(args.function, args.image)
status = result['statusCode']
if status == 200:
labels = result['body']
labels = json.loads(labels)
print(f"There are {len(labels)} labels in the image.")
for custom_label in labels:
confidence = int(round(custom_label['Confidence'], 0))
print(
f"Label: {custom_label['Name']}: Confidence: {confidence}%")
else:
print(f"Error: {result['statusCode']}")
print(f"Message: {result['body']}")
except ClientError as error:
logging.error(error)
print(error)
if __name__ == "__main__":
main()
```
1. Run the code. For the command line argument, supply the Lambda function name and the image that you want to analyze. You can supply a path to a local image, or the S3 path to an image stored in an Amazon S3 bucket. For example:
```
python client.py function_name s3://bucket/path/image.jpg
```
If the image is in an Amazon S3 bucket make sure it is the same bucket that you specified in step 15 of [Step 1: Create an AWS Lambda function (console)](#example-lambda-create-function).
If successful, the output is a list of labels found in the image. If no labels are returned, consider lowering the confidence value that you set in step 7 of [Step 1: Create an AWS Lambda function (console)](#example-lambda-create-function).
1. If you have finished with the Lambda function and the model isn't used by other applications, [stop the model](rm-stop.md). Remember to [start the model](rm-start.md) the next time you want use the Lambda function.