# Smart cropping video using Elemental Inference
<a name="elemental-inference-smart-crop"></a>

In an AWS Elemental MediaLive channel, you can enable the smart crop feature in order to set up one or more outputs with an aspect ratio that is different from the source aspect ratio. A typical use case is to create vertical video from a landscape video. 

MediaLive uses AWS Elemental Inference to crop the video frames to an aspect ratio that you specify. 

Elemental Inference analyzes the source content to detect the region of interest. For example, consider the source video of a soccer game. Elemental Inference infers the location of the region of interest when the ball is moving. Elemental Inference will typically infer that the soccer ball and the players around the ball are that region of interest. 

MediaLive obtains information about the region of interest from Elemental Inference and crops and scales the video. 

**Topics**
+ [Get ready](smart-crop-get-ready.md)
+ [Setting up with the MediaLive console](smart-crop-console-create.md)
+ [Viewing the smart crop setup](smart-crop-view.md)
+ [Modifying smart crop using the MediaLive console](smart-crop-modify-console.md)
+ [Disabling smart crop using the MediaLive console](smart-crop-disable-console.md)
+ [Monitoring smart crop activity](smart-crop-monitor.md)

# Get ready
<a name="smart-crop-get-ready"></a>

## Pricing
<a name="smart-crop-pricing"></a>

There is a charge for running a channel that has the smart crop feature enabled. The charge is applied to each pipeline, which means that it applies once in single-pipeline channels and twice in standard channels. The charge isn't applied to each output in the channel, it is applied only to the pipelines.

To stop this charge, you must disable the feature [in all outputs in the channel](smart-crop-disable-console.md). For information about the current rates for using this feature, see [https://aws.amazon.com/elemental-inference/pricing/](https://aws.amazon.com/elemental-inference/pricing/).

## Source requirements
<a name="smart-crop-source-requirements"></a>
+ Input type: All supported types. The input must be live input, not a file input.
+ Input codec: All supported codecs 
+ Input resolution: All supported resolutions.
+ Aspect ratio: Any aspect ratio 
+ Dynamic image overlays: We recommend that the source doesn’t include image overlays because movement in the overlay might include movement that Elemental Inference will incorrectly start to track. 
+ Static image overlays and burned-in captions: We recommend that the source doesn’t include static image overlays or burned-in captions because the smart crop might cut them off awkwardly. 
+ Smart crop is supported in channels that implement input switching and/or input failover. 
+ Smart crop isn't supported in MediaLive Anywhere channels. 

## Output specifications
<a name="smart-crop-output-requirements"></a>
+ Output types: All supported types.
+ Output codec: All supported codecs.
+ Aspect ratio: Any aspect ratio, and any orientation (portrait or landscape).
+ Resolution: All supported resolutions.
+ Shared encodes: You might choose not to share encodes in a channel that you set up for smart crop because if you enable smart crop in one of the outputs that shares an encode, MediaLive automatically enables it in the other output that shares the encode. If you are setting up smart crop in an existing channel with shared encodes, see [Sharing a video encode](create-video-share.md) for information about how to uncouple the outputs.
+ You can't enable AFD in any video outputs where smart crop is enabled.
+ You can't insert dynamic image overlays in a channel where smart crop is enabled.

# Setting up with the MediaLive console
<a name="smart-crop-console-create"></a>

You set up the smart crop feature in one or more video outputs of a MediaLive channel. In each video output where you want the feature, you set the video scaling to SMART\$1CROP. When you save the channel, MediaLive automatically creates one *feed* resource in Elemental Inference, in your AWS account. Even if you enable more than one Elemental Inference feature, MediaLive creates only one feed.

You can set up smart crop in a new MediaLive channel that you are creating. Or you can update an existing channel to include smart crop.

This section describes how to set up smart crop using the MediaLive console. For information about setting up using an AWS API, see [Elemental Inference features using AWS CLI](elemental-inference-cli.md).

**Note**  
The information in this section assumes that you are familiar with the general steps for creating a channel, as described in [Creating a channel from scratch](creating-channel-scratch.md). 

**Topics**
+ [Step A: Enable smart crop](#smart-crop-procedure-a)
+ [Step B: Review the setup](#smart-crop-procedure-b)
+ [Step C: Save or update the channel](#smart-crop-procedure-c)

## Step A: Enable smart crop
<a name="smart-crop-procedure-a"></a>

You must enable smart crop in the applicable video outputs. 

1. If you plan to update an existing feed, make sure that you have room in the [enabled outputs quota](https://console.aws.amazon.com/servicequotas/home?region=us-east-1#!/services/elemental-inference/quotas) for Elemental Inference. The list of quotas is sorted alphabetically. Look for quotas that don't start with "Request rate for".

   Keep in mind that each feature that you enable in a channel results in one Elemental Inference output.

1. On the **Create channel** or **Edit channel page**, choose **AWS Elemental Inference settings**. 

1. In **State**, choose **ENABLED**. Sections for each Elemental Inference feature appear.

1. In the **Output groups** section of the channel, find an output group and one of the outputs that contains the video. Display the **Stream settings** section, and choose the **Video** section. 
   + Complete the **Width** and **Height** fields to specify the resolution. Set the resolution to the crop that you want MediaLive to apply to the video. For example, 720 x 1600.
   + Open **Scaling settings**, then set **Scaling behavior** to **SMART\$1CROP**. 

1. Repeat the previous step in more video outputs, in this output group and other output groups. Remember that you don't have to set up smart crop in every output group, or in every video output in one output group.

## Step B: Review the setup
<a name="smart-crop-procedure-b"></a>

1. On the **Create channel** or **Edit channel page**, choose **AWS Elemental Inference settings**. The **Smart crop** section automatically expands to show a list of output groups and their video outputs. 
   + Video outputs that are set up for smart crop (smart crop is enabled) appear with the slider enabled. The output is *smart-crop-enabled*
   + Output groups that have *all* their video outputs with smart crop enabled appear with the slider enabled. This output group is smart-crop-enabled.
   + Output groups don't appear with the slider enabled if that contain a mix of enabled and disabled video outputs.

1. You can adjust the configuration:

   Move the slider to enable any output or output group. In the applicable video outputs, MediaLive automatically sets the **Scaling behavior** setting to **Smart crop**. However, you probably still need to view the video output in the Streams settings to change the video width and height, to set the aspect ratio and resolution. If you don't set the width and height, the output might be pillar boxed or letter boxed.

   You can move the slider to disable any output or output group. In the applicable video outputs, MediaLive automatically sets **Scaling behavior** setting to **Default**. However, you probably still need to view the video output in the Streams settings to change the video width and height (the aspect ratio).

When you change the configuration, MediaLive performs the appropriate changes in Elemental Inference. 

## Step C: Save or update the channel
<a name="smart-crop-procedure-c"></a>

Save the channel. MediaLive calls the Elemental Inference `AssociateFeed` endpoint to perform these actions:
+ To create a smart crop output in the feed. 
+ To associate the channel (the resource) with the feed, if this is the first Elemental Inference feature that you are setting up.

You now have a usable feed: resource - feed - output.

You can start the channel. When the channel is running, MediaLive delivers the source stream to Elemental Inference and then retrieves metadata from Elemental Inference that describes the region of interest in each video frame. In each video output where smart crop is set up, MediaLive crops the video to the new region of interest. MediaLive then continues with regular processing to encode the video. 

# Viewing the smart crop setup
<a name="smart-crop-view"></a>

You can view information about the Elemental Inference features in a channel. You can view information on the MediaLive console or the Elemental Inference console. 

## Viewing the setup on the MediaLive console
<a name="smart-crop-eml-console-view"></a>

On the MediaLive console, in the left navigation bar, choose **Channels**, then choose the channel to view. In the tabs in the middle of the page, choose **AWS Elemental Inference**. Information appears, including the following:

**ID and ARN**

The ID and ARN of the Elemental Inference feed that is associated with the channel. 

The ID is identical to the last portion of the ARN of the feed. If you are interested in working with the feed using Elemental Inference directly, make a note of the ARN.

**Status of the feed**

These statuses are listed in lifetime order, from **CREATING** to **ARCHIVED**. Note the following:
+ A newly created feed typically transitions immediately from **CREATING** to **AVAILABLE** to **ACTIVE**. **ACTIVE** means that the feed is associated with the channel.
+ When MediaLive deletes a feed, its status changes to **DELETED**, then after a short period, it changes to **ARCHIVED**. There is no way to change the status of a feed that is **DELETED** or **ARCHIVED**. If you re-enable Elemental Inference features in a channel, MediaLive will create a new feed that has a new ID.

**Feature panels**

One panel appears for each Elemental Inference that you have enabled in the channel. Each panel includes the following information:
+ The type of feature.
+ The status of the feature, which will always be **ENABLED**. (If you disable the feature, the entire panel disappears, so you won't see **DISABLED**.)
+ Settings, which lists the names of all of the video encodes (video description) that are set up for smart crop. These video encodes might be in the same or different outputs.

## Viewing the setup on the Elemental Inference console
<a name="smart-crop-inference-view"></a>

To view information for a feed on the Elemental Inference console, see [ Creating an Elemental Inference workflow](https://docs.aws.amazon.com/elemental-inference/latest/userguide/monitoring-inference-via-console) in the *AWS Elemental Inference user guide*.

## Viewing the setup using the Elemental Inference CLI
<a name="smart-crop-inference-view-cli"></a>

To view information using an AWS API, use the `GetFeed` operation of Elemental Inference. For more information, see [GetFeed](https://docs.aws.amazon.com/elemental-inference/latest/APIReference/API_GetFeed) in the *AWS Elemental Inference API Reference*. 

# Modifying smart crop using the MediaLive console
<a name="smart-crop-modify-console"></a>

**Note**  
This section describes how to modify the smart crop configuration using the MediaLive console. For information about modifying using an AWS API, see [ Work with existing Elemental Inference feeds](https://docs.aws.amazon.com/elemental-inference/latest/userguide/elemental-inference-modify-delete) in the *AWS Elemental Inference user guide*. 

You can use the MediaLive console to modify the existing smart crop configuration in a channel as follows:

In the channel, you can adjust the resolution (width and height) in one or more video outputs that are set up for smart crop. That is the only change you can make. For more information, see [Creating a channel from scratch](creating-channel-scratch.md). 

# Disabling smart crop using the MediaLive console
<a name="smart-crop-disable-console"></a>

You can use the MediaLive console to modify the disable smart crop in a channel.

**Note**  
The information in this section assumes that you are familiar with the general steps for creating a channel, as described in [Creating a channel from scratch](creating-channel-scratch.md). 

**To disable smart crop in all outputs**

1. On the **Create channel** or **Edit channel page**, choose **AWS Elemental Inference settings**. The **Smart crop** section is automatically expanded to show a list of output groups and their video outputs. 

1. Choose the appropriate action:
   + If smart crop is the only Elemental Inference feature that is enabled on this page: in **State**, choose **DISABLED**.
   + Otherwise, in the **Smart crop** section, move the slider for every output group to disabled (gray).

**To disable smart crop in individual outputs**

1. On the **Create channel** or **Edit channel page**, in the **Output groups** section, select the output that contains the video. 

1. Display the **Stream settings** section, and choose the **Video** section. 
   + Adjust the values in the **Width** and **Height** fields.
   + Open **Scaling settings**, then set **Scaling behavior** to a value other than **SMART\$1CROP**.

When you disable Elemental Inference features in a channel, MediaLive handles the resources as follows:
+ If you disable one feature among several features, MediaLive deletes the output for that feature. 
+ If you disable all the features, MediaLive deletes the feed, including its outputs. When the feed is deleted, its status changes to Archived. The feed can't become Active again.

# Monitoring smart crop activity
<a name="smart-crop-monitor"></a>

You can monitor smart crop activity using these AWS services:
+ Metrics that Amazon CloudWatch produces, using data sent by Elemental Inference. See [ Monitoring AWS Elemental Inference with Amazon CloudWatch ](https://docs.aws.amazon.com/elemental-inference/latest/userguide/monitoring-cloudwatch) in the *AWS Elemental Inference user guide*. 
+ Events that Elemental Inference produces. You can use Amazon EventBridge to work with these events. See [ Monitoring AWS Elemental Inference events with Amazon EventBridge ](https://docs.aws.amazon.com/elemental-inference/latest/userguide/monitoring-events) in the *AWS Elemental Inference user guide*. 
+ Actions that Amazon EventBridge takes and that are collected by AWS CloudTrail. See [ Monitoring Elemental Inference API calls with AWS CloudTrail](https://docs.aws.amazon.com/elemental-inference/latest/userguide/logging-using-cloudtrail) in the *AWS Elemental Inference user guide*.