# Generating insights from calls using call analytics for the Amazon Chime SDK
The topics in this section explain how to use Amazon Chime SDK call analytics to generate insights from your call data.
Amazon Chime SDK call analytics gives developers low-code solutions for generating cost-effective insights from real-time audio, including audio ingestion, analysis, alerting, and data lake integration. Call analytics enables you to generate insights through integration with Amazon Transcribe and Transcribe Call Analytics (TCA), and natively through Amazon Chime SDK voice analytics. Call analytics can also record calls to your Amazon S3 Bucket.
You can use the following methods to configure and run call analytics.
+ Use the Amazon Chime SDK console to create a call analytics configuration and associate it with an Amazon Chime SDK Voice Connector. During that process, you can enable call recording and analytics. You don't need to write code to complete the process.
+ Use a set of Amazon Chime SDK APIs [Amazon Chime SDK](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/welcome.html) APIs to programmatically create and run a configuration.
For more information, refer to [Creating call analytics configurations for the Amazon Chime SDK](creating-ca-configuration.md) and [Using call analytics configurations for the Amazon Chime SDK](using-call-analytics-configurations.md), later in this section.
**Topics**
+ [What is Amazon Chime SDK call analytics](what-is-amazon-chime-sdk-call-analytics.md)
+ [Understanding call analytics terminology for the Amazon Chime SDK](ca-terms-concepts.md)
+ [Creating call analytics configurations for the Amazon Chime SDK](creating-ca-configuration.md)
+ [Using call analytics configurations for the Amazon Chime SDK](using-call-analytics-configurations.md)
+ [Managing call analytics pipelines for the Amazon Chime SDK](managing-call-analytics-pipelines.md)
+ [Pausing and resuming call analytics pipelines for the Amazon Chime SDK](pausing-and-resuming-call-analytics-pipelines.md)
+ [Using the call analytics resource access role for the Amazon Chime SDK](call-analytics-resource-access-role.md)
+ [Understanding the call analytics statuses for the Amazon Chime SDK](call-analytics-statuses.md)
+ [Monitoring call analytics pipelines for the Amazon Chime SDK with Amazon CloudWatch](monitoring-with-cloudwatch.md)
+ [Call analytics processor and output destinations for the Amazon Chime SDK](call-analytics-processor-and-output-destinations.md)
+ [Call analytics data model for the Amazon Chime SDK](ca-data-model.md)
+ [Using Amazon Chime SDK voice analytics](voice-analytics.md)
+ [Call analytics service quotas for the Amazon Chime SDK](ca-regions.md)
# What is Amazon Chime SDK call analytics
Amazon Chime SDK call analytics is a low-code solution for generating cost-effective insights from real-time audio, including capabilities for audio ingestion, recording, voice analytics, alerting, and a data lake. You can generate machine learning powered insights using call analytics by creating a reusable call analytics configuration that determines which AWS machine learning integrations and audio processing features to enable for a workflow. You then use the call analytics configuration with various media sources such as Voice Connectors or Amazon Kinesis Video Streams. Call analytics generates insights through integrations with Amazon Transcribe and Transcribe call analytics (TCA), and natively through [Amazon Chime SDK voice analytics](voice-analytics.md), a service that runs under call analytics.
You follow these steps to use call analytics:
![\[Image showing the process of setting up Amazon Chime SDK call analytics.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/CallAnalyticsOverview.png)
In the diagram:
1. You start by creating a call analytics configuration.
1. You set your output destinations and an optional data lake.
1. You create workflows that associate your configuration with a Voice Connector and Amazon Kinesis Video Streams.
1. You analyze and optionally visualize your insights.
You can use the Amazon Chime SDK console to create a call analytics configuration and enable call analytics to start automatically. If you need to control the configurations that apply to a given type of call, you use APIs to create a configuration. Either way, the configuration contains details about the AWS machine learning services to invoke for the call audio, enable call recording, and the destinations for the insights, metadata, and recordings. Call analytics provides the following destinations:
+ An Amazon Kinesis Data Stream (KDS). You can use KDS to receive live call insights that you can then integrate into your application. For instance, you can integrate the live insights to help a sales or customer-support agent during a customer call, or use the insights to augment generative AI prompts and summaries.
+ An Amazon S3 bucket configured as a data warehouse. The bucket stores data in Parquet format. Parquet is an open-source file format designed to compress and store large volumes of data. You can then use Amazon Athena to query that data using simple query language (SQL), or move the data to your existing data warehouse to pair up with your business data. For example, you can perform post-call aggregate analytics to understand effectiveness of the customer calls, problems areas of a product, or opportunities to train employees to achieve better customer outcomes.
In addition to those destinations, call analytics also supports real-time alerts that you can preconfigure, based on the insights. The alerts are sent to Amazon EventBridge.
**Note**
When you create a call analytics configuration, you don't select a specific audio source. That allows you to reuse configurations across multiple audio sources. For example, a configuration can enable call recording and provide call transcription. You can then use the configuration with a Chime SDK Voice Connector and an audio stream via a Kinesis Video Stream. You can also share the configuration among multiple Voice Connectors. Each call analytics configuration is unique and identified by an ARN.
# Understanding call analytics terminology for the Amazon Chime SDK
The following terminology and concepts are central to understanding how to use Amazon Chime SDK call analytics.
**Amazon Athena**
An interactive query service that enables you to analyze data in Amazon S3 using standard SQL. Athena is serverless, so you have no infrastructure to manage, and you pay only for the queries that you run. To use Athena, point to your data in Amazon S3, define the schema, and use standard SQL queries. You can also use workgroups to group users and control the resources they have access to when running queries. Workgroups enable you to manage query concurrency and prioritize query execution across different groups of users and workloads. For more information, refer to [What is Amazon Athena](https://docs.aws.amazon.com/athena/latest/ug/what-is.html).
**Amazon Kinesis Data Firehose**
An extract, transform, and load (ETL) service that reliably captures, transforms, and delivers streaming data to data lakes, data stores, and analytics services. For more information, refer to [What Is Amazon Kinesis Data Firehose](https://aws.amazon.com/kinesis/data-firehose/).
**Call analytics data warehouse**
Optional storage for call analytics data. The warehouse stores data in a parquet-based data file format in an Amazon S3 bucket. You can use standard SQL to query the data. You enable the warehouse in a call analytics configuration.
**Glue Data Catalog**
A centralized metadata repository for data assets across various data sources. The catalog consists of databases and tables. For call analytics, the metadata in the table tells Athena the location of your Amazon S3 bucket. It also specifies the data structure, such as column names, data types, and the table's name. Databases only hold the metadata and schema information for a dataset. For more information, refer to [Understanding the AWS Glue data catalog table structure for the Amazon Chime SDK](ca-data-model-diagram.md), later in this section.
**Media insights pipeline**
A temporary resource identified by a unique `MediaPipelineId`. Created by using a call analytics pipeline configuration and runtime parameters. The runtime parameters specify the data source for the pipeline.
**Media insights pipeline configuration**
A static configuration used to create media insights pipelines. You can use a configuration to instantiate one or more pipelines.
**Media insights pipeline configuration element**
The media insights pipeline configuration element includes instructions for either processing media using a processor element or delivering generated insights using a sink element.
**Media insights pipeline task**
A temporary sub-resource of a media insights pipeline. Tasks hold metadata about the status of a process for a specific stream ARN and channel ID. Identified by a unique ID. Created by starting voice analytics on a media insights pipeline.
**Speaker search**
A voice analytics feature that helps you to recognize call participants.
**Voice analytics**
An Amazon Chime SDK feature that includes speaker search and voice tone analysis.
**Voice embedding**
A vector representation of a caller's voice, plus a unique ID.
**Voice enhancement**
A system that enhances the audio quality of phone calls.
**Voice profile**
The combination of a voice embedding, its ID, and its expiration date.
**Voice profile domain**
A collection of voice profiles.
**Voice tone analysis**
A voice analytics feature that enables you to analyze caller voices for a `positive`, `negative`, or `neutral` sentiment.
For more information about the APIs used to create call insights configurations, initiate pipelines, and run voice analytics, refer to [ Amazon Chime SDK Media Pipelines](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_Operations_Amazon_Chime_SDK_Media_Pipelines.html), in the *Amazon Chime SDK API Reference*.
**Note**
We strongly recommend using the media insights pipeline APIs to run call analytics, because only those APIs provide new features. For more information about the differences between the media pipeline and voice namespaces, refer to [Using voice APIs to run voice analytics for the Amazon Chime SDK](va-in-voice-namespace.md), later in this section.
# Creating call analytics configurations for the Amazon Chime SDK
To use call analytics, you start by creating a *configuration*, a static structure that holds the information needed to create a call analytics pipeline. You can use the Amazon Chime SDK console to create a configuration, or call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API.
A call analytics configuration includes details about audio processors, such as recording, voice analytics, or Amazon Transcribe. It also includes insight destinations and alert event configurations. Optionally, you can save your call data to an Amazon S3 bucket for further analysis.
However, *configurations do not include specific audio sources*. That allows you reuse the configuration across multiple call analytics workflows. For example, you can use the same call analytics configuration with different Voice Connectors or across different Amazon Kinesis Video Stream (KVS) sources.
You use the configurations to create pipelines when SIP calls occur through a Voice Connector, or when new media is sent to an Amazon Kinesis Video Stream (KVS). The pipelines, in turn, process the media according to the specifications in the configuration.
You can stop a pipeline programmatically at any time. Pipelines also stop processing media when a Voice Connector call ends. In addition, you can pause a pipeline. Doing so disables calls to the underlying Amazon machine learning services and resumes them when desired. However, call recording runs while you pause a pipeline.
The following sections explain the prerequisites for creating a call analytics configuration, and how to create one.
**Topics**
+ [Understanding the Amazon Chime SDK call analytics prerequisites](ca-prerequisites.md)
+ [Using the Amazon Chime SDK console to create call analytics configurations](create-config-console.md)
+ [Using APIs to create call analytics configurations for the Amazon Chime SDK](create-config-apis.md)
+ [Associating a configuration with a Voice Connector for the Amazon Chime SDK](ca-associate-vc-steps.md)
# Understanding the Amazon Chime SDK call analytics prerequisites
Before you create a call analytics configuration, you must have the following items. You can use the AWS console to create them:
+ An Amazon Chime SDK Voice Connector. If not, refer to [Creating Amazon Chime SDK Voice Connectors](https://docs.aws.amazon.com/chime-sdk/latest/ag/ca-prerequisites.html). You must also:
+ Enable streaming for the Voice Connector. For more information, refer to [Automating the Amazon Chime SDK with EventBridge](https://docs.aws.amazon.com/chime-sdk/latest/ag/automating-chime-with-cloudwatch-events.html), in the *Amazon Chime SDK Administrator Guide*
+ Configure the Voice Connector to use call analytics. For more information, refer to [Configuring Voice Connectors to use call analytics](https://docs.aws.amazon.com/chime-sdk/latest/ag/configure-voicecon.html), in the *Amazon Chime SDK Administrator Guide*.
+ Amazon EventBridge targets. If not, see [Monitoring the Amazon Chime SDK with EventBridge](https://docs.aws.amazon.com/chime-sdk/latest/ag/automating-chime-with-cloudwatch-events.html), *Amazon Chime SDK Administrator Guide*.
+ A service-linked role that allows the Voice Connector to access actions on the EventBridge targets. For more information, refer to [Using the Amazon Chime SDK Voice Connector service linked role policy](https://docs.aws.amazon.com/chime-sdk/latest/ag/using-service-linked-roles-stream.html), in the *Amazon Chime SDK Administrator Guide*.
+ An Amazon Kinesis Data Stream. If not, refer to [ Creating and Managing Streams](https://docs.aws.amazon.com/streams/latest/dev/working-with-streams.html), in the *Amazon Kinesis Streams Developer Guide*. Voice analytics and transcription require a Kinesis Data Stream.
+ To analyze calls offline, you must create an Amazon Chime SDK data lake. To do that, refer to [Creating an Amazon Chime SDK data lake](ca-data-lake.md), later in this guide.
# Using the Amazon Chime SDK console to create call analytics configurations
After you create the prerequisites listed in the previous section, you can use the Amazon Chime SDK console to create one or more call analytics configurations. You can also use the console to associate one or more Voice Connectors with your configurations. When you complete that process, call analytics begins running with the features that you enable when you create the configuration.
You follow these steps to create a call analytics configuration:
1. Specify the configuration details, including a name and optional tags.
1. Configure your recording settings. Create a call analytics configuration that includes recording and machine-learning powered insights.
1. Configure your analytics services.
1. Select output destinations for consuming real-time insights. Create an optional data lake to perform post-call analytics.
1. Create a new service role or use an existing role.
1. Set up real-time alerts that send notifications via Amazon EventBridge when certain conditions are met.
1. Review your settings and create the configuration
After you create the configuration, you enable call analytics by associating a Voice Connector with the configuration. Once you do that, call analytics starts automatically when a call comes in to that Voice Connector. For more information, refer to [Associating a configuration with a Voice Connector for the Amazon Chime SDK](ca-associate-vc-steps.md), later in this section.
The following sections explain how to complete each step of the process. Expand them in the order listed.
## Specify configuration details
**To specify configuration details**
1. Open the Amazon Chime console at [https://console.aws.amazon.com/chime-sdk/home](https://console.aws.amazon.com/chime-sdk/home).
1. In the navigation pane, under **Call Analytics**, choose **Configurations**, then choose **Create configuration**.
1. Under **Basic information**, do the following:
1. Enter a name for the configuration. The name should reflect your use case and any tags.
1. (Optional) Under **Tags**, choose **Add new tag**, then enter your tag keys and optional values. You define the keys and values. Tags can help you query the configuration.
1. Choose **Next**.
## Configuring recording
**To configure recording**
+ On the **Configure recording** page, do the following:
1. Choose the **Activate call recording** checkbox. This enables recording for Voice Connector calls or KVS streams and sending the data to your Amazon S3 bucket.
1. Under **File format**, choose **WAV with PCM** for the best audio quality.
—or—
Choose **OGG with OPUS** to compress the audio and optimize storage.
1. (Optional) As needed, choose the **Create an Amazon S3 bucket** link and follow those steps to create an Amazon S3 bucket.
1. Enter the URI of your Amazon S3 bucket, or choose **Browse** to locate a bucket.
1. (Optional) Choose **Activate voice enhancement** to help improve the audio quality of your recordings.
1. Choose **Next**.
## Understanding voice enhancement
When you create a call analytics configuration, you can enable call recording and store the recorded calls in an Amazon S3 bucket. As part of that, you can also enable voice enhancement and improve the audio quality of your stored calls. Voice enhancement only applies to recordings generated after the feature is enabled. When the voice enhancement capability is active, an enhanced recording is created in addition to the original recording, and is stored in the same Amazon S3 bucket and format. Voice enhancement will generate enhanced recordings for calls that are up to 30 minutes long. Enhanced recordings will not be generated for calls that are longer than 30 minutes.
Phone calls are narrowband-filtered and sampled at 8 KHz. Voice enhancement boosts the sampling rate from 8kHz to 16kHz and uses a machine learning model to expand the frequency content from narrowband to wideband to make the speech more natural-sounding. Voice enhancement also uses a noise reduction model called Amazon Voice Focus to help reduce background noise in the enhanced audio.
Voice enhancement also uses a noise reduction model called Voice Focus. The model helps reduce background noise in the enhanced audio. Voice enhancement applies the model to the upgraded 16 KHz audio.
**Note**
The voice enhancement feature is only supported in US East (N. Virginia) Region and US West (Oregon) Region.
Voice enhancement recordings metadata are published through your configured KDS into the existing AWS Glue data catalog table *call\$1analytics\$1recording\$1metadata*. To identify the original call recording record from the voice enhanced call recording, a new field called *detail-subtype* with value *VoiceEnhancement* is added to KDS notification and the glue table *call\$1analytics\$1recording\$1metadata*. For more information on the data warehouse schema, see [Call analytics data model for the Amazon Chime SDK](ca-data-model.md).
### Voice enhancement file format
Note the following about enhanced recording files.
+ Enhanced recordings are written to the same Amazon S3 bucket as regular recordings. You configure the destination by calling the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_S3RecordingSinkConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_S3RecordingSinkConfiguration.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_S3RecordingSinkRuntimeConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_S3RecordingSinkRuntimeConfiguration.html)APIs, or by using the Amazon Chime SDK console.
+ Enhanced recordings have **\$1enhanced** appended to the base file name. name.
+ Enhanced recordings keep the same file format as the original recording. You configure the file format by calling the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_S3RecordingSinkConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_S3RecordingSinkConfiguration.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_S3RecordingSinkRuntimeConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_S3RecordingSinkRuntimeConfiguration.html) APIs, or by using the Amazon Chime SDK console.
The following example shows a typical file name format.
```
s3://original_file_name_enhanced.wav
```
or
```
s3://original_file_name_enhanced.ogg
```
## Configure analytics services
Amazon Transcribe provides text transcriptions of calls. You can then use the transcripts to augment other machine learning services such as Amazon Comprehend or your own machine learning models.
**Note**
Amazon Transcribe also provides automatic language recognition. However, You can't use that feature with custom language models or content redaction. Also, if you use language identification with other features, you can only use the languages that those features support. For more information, refer to [Language identification with streaming transcriptions](https://docs.aws.amazon.com/transcribe/latest/dg/lang-id-stream.html), in the *Amazon Transcribe Developer Guide*.
Amazon Transcribe Call Analytics is a machine-learning powered API that provides call transcripts, sentiment, and real-time conversation insights. The service eliminates the need for note-taking, and it can enable immediate action on detected issues. The service also provides post-call analytics, such as caller sentiment, call drivers, non-talk time, interruptions, talk speed, and conversation characteristics.
**Note**
By default, post-call analytics streams call recordings to your Amazon S3 bucket. To avoid creating duplicate recordings, do not enable call recording and post-call analytics at the same time.
Finally, Transcribe Call Analytics can automatically tag conversations based on specific phrases and help redact sensitive information from audio and text. For more information on the call analytics media processors, insights generated by these processors, and output destinations see [Call analytics processor and output destinations for the Amazon Chime SDK](call-analytics-processor-and-output-destinations.md), later in this section.
**To configure analytics services**
1. On the **Configure analytics services** page, select the check boxes next to **Voice analytics** or **Transcription services**. You can select both items.
Select the **Voice analytics**, checkbox to enable any combination of **Speaker search** and **Voice tone analysis**.
Select the **Transcription services** checkbox to enable Amazon Transcribe or Transcribe Call Analytics.
1. **To enable Speaker search**
+ Select the **Yes, I agree to the Consent Acknowledgement for Amazon Chime SDK voice analytics** checkbox, then choose **Accept**.
1. To enable Voice tone analysis
+ Select the **Voice tone analysis** checkbox.
1. To enable Amazon Transcribe
1. Choose the **Amazon Transcribe** button.
1. Under **Language settings**, do either of the following:
1. If your callers speak a single language, choose **Specific language**, then open the **Language** list and select the language.
1. If your callers speak multiple languages, you can automatically identify them. Choose **Automatic language detection**.
1. Open the **Language options for automatic language identification** list and select at least two languages.
1. (Optional) Open the **Preferred language** list and specify a preferred language. When the languages you selected in the previous step have matching confidence scores, the service transcribes the preferred language.
1. (Optional) Expand **Content removal settings**, select one or more options, then choose one or more of the additional options that appear. Helper text explains each option.
1. (Optional) Expand **Additional settings**, select one or more options, then choose one or more of the additional options that appear. Helper text explains each option.
1. To enable Amazon Transcribe Call Analytics
1. Choose the **Amazon Transcribe Call Analytics** button.
1. Open the **Language** list and select a language.
1. (Optional) Expand **Content removal settings**, select one or more options, then choose one or more of the additional options that appear. Helper text explains each option.
1. (Optional) Expand **Additional settings**, select one or more options, then choose one or more of the additional options that appear. Helper text explains each option.
1. (Optional) Expand **Post-call analytics settings** and do the following:
1. Choose the **Post-call analysis** checkbox.
1. Enter the URI of your Amazon S3 bucket.
1. Select a content redaction type.
1. When you finish making your selections, choose **Next**.
## Configure output details
After you finish the media processing steps, you select a destination for the analytics output. Call analytics provides live insights via Amazon Kinesis Data Streams, and optionally through a data warehouse in an Amazon S3 bucket of your choice. To create the data warehouse, you use a CloudFormation Template. The template helps you create the infrastructure that delivers the call metadata and insights to your Amazon S3 bucket. For more information on the data warehouse creation, refer to [Creating an Amazon Chime SDK data lake](ca-data-lake.md), later in this section. For more information on the data warehouse schema, refer to [Call analytics data model for the Amazon Chime SDK](ca-data-model.md), also later in this section.
If you enabled voice analytics in the previous section, you can also add voice analytics notification destinations such as AWS Lambda, Amazon Simple Queue Service, or Amazon Simple Notification Service. The following steps explain how.
**To configure output details**
1. Open the **Kinesis data stream** list and select your data stream.
**Note**
If you want to visualize your data, you must select the Kinesis data stream used by the Amazon S3 bucket and Amazon Kinesis Data Firehose.
1. (Optional) Expand **Additional voice analytics notification destinations** and select any combination of AWS Lambda, Amazon SNS, and Amazon SQS destinations.
1. (Optional) Under **Analyze and visualize insights**, select the **Perform historical analysis with data lake** checkbox. For more information about data lakes, refer to [Creating an Amazon Chime SDK data lake](ca-data-lake.md), later in this section.
1. When finished, choose **Next**.
## Configure access permissions
To enable call analytics, the machine learning service and other resources must have permissions to access data media and deliver insights. You can use an existing service role or use the console to create a new role. For more information about roles, refer to [Using the call analytics resource access role for the Amazon Chime SDK](call-analytics-resource-access-role.md), later in this section.
**To configure access permissions**
1. On the **Configure access permissions** page, do one of the following:
1. Select **Create and use a new service role**.
1. In the **Service role name suffix** box, enter a descriptive suffix for the role.
—or—
1. Select **Use an existing service role**.
1. Open the **Service role** list and select a role.
1. Choose **Next**.
## (Optional) Configure real-time alerts
**Important**
To use real-time alerts, you must first enable Amazon Transcribe or Amazon Transcribe Analytics.
You can create a set of rules that send real-time alerts to Amazon EventBridge. When an insight generated by Amazon Transcribe or Amazon Transcribe Call Analytics matches your specified rule during an analytics session, an alert is sent. Alerts have the detail type `Media Insights Rules Matched`. EventBridge supports integration with downstream services such as Amazon Lambda, Amazon SQS, and Amazon SNS to trigger notifications for the end user or initiate other custom business logic. For more information, refer to [Using Amazon EventBridge notifications for the Amazon Chime SDK](using-eventbridge-notifications.md), later in this section.
**To configure alerts**
1. Under **Real-time alerts**, choose **Active real-time alerts**.
1. Under **Rules**, select **Create rule**.
1. In the **Rule name** box, enter a name for the rule.
1. Open the **Rule type** list and select the type of rule you want to use.
1. Use the controls that appear to add keywords to the rule and apply logic, such as **mentioned** or **not mentioned**.
1. Choose **Next**.
## Review and create
**To create the configuration**
1. Review the settings in each section. As needed choose **Edit** to change a setting.
1. Choose **Create configuration**.
Your configuration appears on the **Configurations** page of the Amazon Chime SDK console.
# Using APIs to create call analytics configurations for the Amazon Chime SDK
You can programmatically create Voice Connectors and call analytics configurations, and then associate them in order to start a call analytics workflow. This guide assumes that you know how to write the code.
The APIs that you use vary, depending on the type of workflow. For example, to record audio, you first call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API to create a call analytics configuration. You then call the [CreateVoiceConnector](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceConnector.html) to create a Voice Connector. Finally, you associate the configuration with a Voice Connector by using the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_PutVoiceConnectorStreamingConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_PutVoiceConnectorStreamingConfiguration.html) API.
In contrast, to record audio with a Kinesis video stream producer, you call [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html), and then call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API.
For more information about using call analytics configurations to enable different workflows, refer to the workflows in [Using call analytics configurations for the Amazon Chime SDK](using-call-analytics-configurations.md), later in this section.
# Associating a configuration with a Voice Connector for the Amazon Chime SDK
After you use the console to create a call analytics configuration, you use the configuration by associating a Voice Connector with it. The Voice Connector then automatically invokes call the analytics services specified in your configuration. The Voice Connector invokes call analytics for each call.
**To associate a Voice Connector**
1. Open the Amazon Chime console at [https://console.aws.amazon.com/chime-sdk/home](https://console.aws.amazon.com/chime-sdk/home).
1. In the navigation pane, under **SIP Trunking**, choose **Voice Connectors**.
1. Choose the name of the Voice Connector that you want to associate with a configuration, then choose the **Streaming** tab.
1. If it isn't already selected, choose **Start** to begin streaming to Kinesis Video Streams.
1. Under **Call Analytics**, select **Activate**, and on the menu that appears, choose your call analytics configuration ARN.
1. Choose **Save**.
**Note**
After enabling, disabling, or modifying a configuration associated with a Voice Connector, allow 5 minutes for the new settings to propagate through the service and take effect.
For more information about call analytics configurations, refer to [Managing call analytics](https://docs.aws.amazon.com/chime-sdk/latest/ag/ag-call-analytics.html) in the *Amazon Chime SDK Administrator Guide*.
For more information about using call analytics configurations to enable different workflows, refer to [Using call analytics configurations for the Amazon Chime SDK](using-call-analytics-configurations.md), later in this section.
# Using call analytics configurations for the Amazon Chime SDK
To process audio using a call analytics configuration, you must create a call analytics pipeline, also known as a media insights pipeline. The pipeline is created during a call to handle the audio and is terminated at the end of the call. Call analytics pipelines require the ARN of a call analytics configuration, and information about the audio source. The call analytics configuration includes details about audio processors, insight destinations, and alert event configurations, *but not the audio source*. This allows you to reuse the configuration across different call analytics workflows, such as with different Voice Connectors or KVS sources. The call analytics pipeline invokes the machine learning services specified in the configuration and records the audio. You can manually or automatically stop the pipeline when the call ends.
You can use call analytics pipelines in a wide variety use cases. The following workflows show potential ways to use a call analytics configuration and pipeline.
**Topics**
+ [Understanding workflows for recording calls for the Amazon Chime SDK](recording-workflows.md)
+ [Understanding workflows for machine-learning based analytics for the Amazon Chime SDK](ml-based-analytics.md)
# Understanding workflows for recording calls for the Amazon Chime SDK
The topics in this section list and describe the workflows for recording calls and Kinesis Video Streams.
# Recording Voice Connector calls
Use this workflow when:
+ You already use, or plan to use, a Voice Connector to bring SIP media into call analytics.
**Note**
Voice Connectors support SIP and SIPREC. For more information, refer to [ Managing Amazon Chime SDK Voice Connectors](https://docs.aws.amazon.com/chime-sdk/latest/ag/voice-connectors.html), in the *Amazon Chime SDK Administrator Guide*.
+ You want to automatically record SIP or SIPREC calls with low latency to your choice of Amazon Simple Storage Service destinations.
+ You want to use the Amazon Chime SDK console to create the configuration and associate it with a Voice Connector.
+ You want to apply the same recording configuration to every Voice Connector call. If you want to apply multiple configurations to one or more Voice Connectors, refer to the next section.
To enable calling programmatically, use the following Amazon Chime SDK APIs.
+ Use the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API to create a call analytics configuration
+ Use the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceConnector.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceConnector.html) to create a Voice Connector.
+ Use the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_PutVoiceConnectorStreamingConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_PutVoiceConnectorStreamingConfiguration.html) API to associate the configuration with a Voice Connector.
For more information, see [Configuring Voice Connectors to use call analytics](https://docs.aws.amazon.com/chime-sdk/latest/ag/configure-voicecon.html) in the *Amazon Chime SDK Administrator Guide*.
The following diagram shows the flow of data when a Voice Connector initiates a call recording session. Numbers in the diagram correspond to the numbered text below.
![\[Image showing the flow of data when a Voice Connector starts recording a call.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/call-analytics-workflow-1.png)
In the diagram:
1. Use the Amazon Chime SDK console or the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API to create a call analytics configuration. During the configuration creation process, you simply activate call recording, choose the desired recording file format, and specify the Amazon S3 destination for storing the recording files. For more information, refer to [Creating call analytics configurations](https://docs.aws.amazon.com/chime-sdk/latest/ag/create-ca-config.html), in the *Amazon Chime SDK Administrator Guide*.
1. You use the Amazon Chime SDK console or the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_PutVoiceConnectorStreamingConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_PutVoiceConnectorStreamingConfiguration.html) API to associate the configuration with a Voice Connector. To use the console, refer to [ Configuring Voice Connectors to use call analytics](https://docs.aws.amazon.com/chime-sdk/latest/ag/configure-voicecon.html).
1. During an outgoing call, the Voice Connector receives each call participant's audio.
1. If a call analytics recording configuration is attached to the Voice Connector, the Voice Connector service uses the media pipeline service to initiate a call analytics recording session.
1. The media pipeline service initiates the call recording processor that monitors the ongoing call.
1. When the call ends, the media pipeline service delivers the call recording file to the designated Amazon S3 bucket and provides the recording metadata through the Amazon Kinesis Data Stream. If a data warehouse is enabled, the call metadata also goes to the Amazon Simple Storage Service data warehouse. In cases where SIPREC is utilized to incorporate SIP audio into call analytics, the call metadata includes SIPREC metadata in a table format. For more information on the recording tables, refer to [Understanding the AWS Glue data catalog tables for the Amazon Chime SDK](glue-tables.md), later in this section.
1. The media pipeline service sends the pipeline status events to the default Amazon EventBridge. For more information see, [Using EventBridge notifications](https://docs.aws.amazon.com/chime-sdk/latest/dg/ca-eventbridge-notifications.html) in this guide.
**Note**
Please note, you must enable Voice Connector streaming to enable recording with a Voice Connector. This feature enables streaming of call data to the Voice Connector managed Kinesis Video Streams in your account. For more information, refer to [Streaming Amazon Chime SDK Voice Connector media to Kinesis Video Streams](https://docs.aws.amazon.com/chime-sdk/latest/ag/start-kinesis-vc.html) in the *Amazon Chime SDK Administrator Guide*.
You can also to store Voice Connector created call data in Kinesis Video Streams for varying durations, ranging from hours to days or even years. Opting for no data retention limits the usability of the call data for immediate consumption. The cost of Kinesis Video Streams is determined based on the bandwidth and total storage utilized. You can adjust the data retention period at any time within the Voice Connector streaming configuration. To enable call analytics recording, you must ensure that the Kinesis Video Stream retains the data long enough to conduct the call analytics. You do that by specifying a suitable data retention period.
You can associate a call insights pipeline configuration with as many Voice Connectors as you want. You can also create a different configuration for each Voice Connector. Voice Connectors use the AWSServiceRoleForAmazonChimeVoiceConnector to call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API on your behalf once per transaction ID. For information about the role, see [ Using the Amazon Chime SDK service-linked role for Amazon Chime SDK Voice Connectors](https://docs.aws.amazon.com/chime-sdk/latest/ag/using-service-linked-roles-stream.html#service-linked-role-permissions-stream) in the *Amazon Chime SDK Administrator Guide*.
# Recording with Amazon Kinesis Video stream producers
You record Amazon Kinesis Video streams when:
+ You need to apply different configurations to a call instead of using the same configuration for every Voice Connector call.
+ You want to record SIP or non-SIP audio that isn't processed by a Voice Connector.
To use this call recording option, you need to publish audio to Kinesis Video Streams (KVS) and then call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API with KVS stream channel information and a call analytics configuration ARN.
**Note**
The call analytics APIs support a maximum of two audio channels. You can also enable Voice Connector streaming, then use the KVS information published in the Voice Connector's EventBridge notifications to initiate a call recording.
When calling the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API, you can choose whether or not to specify fragment numbers for each KVS stream channel definition. If you supply a fragment number, call analytics will begin processing the stream at that fragment. If you don't specify a fragment ID, call analytics begins processing the stream from the latest available fragment.
The following diagram shows the flow of data when a Voice Connector initiates a call recording session. Numbers in the diagram correspond to the numbered text below.
![\[Image showing the flow of data when a Voice Connector starts recording a call.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/call-analytics-workflow-2.png)
In the diagram:
1. You can use the Amazon Chime SDK console or the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API to the create a call recording configuration.
1. Use the AWS SDK to create an application that pushes external audio into KVS, or enable Voice Connector streaming to publish call audio automatically to a KVS. For more information, see [Streaming Amazon Chime SDK Voice Connector media to Kinesis Video Streams](https://docs.aws.amazon.com/chime-sdk/latest/ag/start-kinesis-vc.html) in the *Amazon Chime SDK Administrator Guide*.
1. If Voice Connector streaming is enabled, the Voice Connector service sends notifications to the default EventBridge.
1. In case of Voice Connector streaming, your application can use the Amazon Chime Voice Connector streaming `STARTED` events from EventBridge to gather KVS stream information about the legs of a call.
1. Once your application has the audio information from Voice Connector streaming events or an external source, your application invokes the Amazon Chime SDK [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API.
1. The media pipeline service initiates the call recording processor that monitors the ongoing call.
1. The media pipeline service sends the pipeline status events to the default Amazon EventBridge. For more information, refer to [Using EventBridge notifications](https://docs.aws.amazon.com/chime-sdk/latest/dg/ca-eventbridge-notifications.html).
1. Once a call is completed, the media pipeline service will deliver the call recording file to the designated Amazon S3 bucket and provide the recording metadata through Amazon Kinesis Data Stream. If a data warehouse is enabled, the call metadata will also be sent to the Amazon S3 data warehouse. In cases where SIPREC is utilized to incorporate SIP audio into call analytics, the call metadata will include SIPREC metadata in a convenient table format. For more information on the recording tables, refer to [Understanding the AWS Glue data catalog tables for the Amazon Chime SDK](glue-tables.md), later in this section.
1. Your application can monitor the pipeline, and in case of a Voice Connector, the call status using events published to the Amazon EventBridge. For more information see, [Using EventBridge notifications](https://docs.aws.amazon.com/chime-sdk/latest/dg/ca-eventbridge-notifications.html) in this guide.
1. To terminate recording, call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_DeleteMediaPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_DeleteMediaPipeline.html) API to terminate the call recording.
For API based recording and examples see, [Amazon S3 recording sink](https://docs.aws.amazon.com/chime-sdk/latest/dg/ca-processors-sinks.html#ca-s3-recording-sink) in this guide.
# Using the CLI to start recording
The examples in this section explain how to do the following:
+ Use the CLI to run a call analytics configuration and invoke the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html).
+ Use the CLI to specify recording destinations audio file formats, and audio file names.
**Topics**
+ [Running a configuration and starting a pipeline](#cli-run-config)
+ [Setting destinations, names, and formats](#cli-set-buckets-formats)
## Running a configuration and starting a pipeline
Use the following command to run a configuration and start a media insights pipeline. The pipeline.json file contains the configuration settings.
```
aws chime-sdk-media-pipeline create-media-insights-pipeline --cli-input-json file://pipeline.json
```
The following example shows a typical `pipeline.json` file.
```
{
"MediaInsightsPipelineConfigurationArn": arn:aws:chime:region;account_id:media-insights-pipeline-configuration/MyConfiguration,
"KinesisVideoStreamRecordingSourceRuntimeConfiguration": {
"Streams": [
{
"StreamArn": kinesis_video_stream_arn_1
},
{
"StreamArn": kinesis_video_stream_arn_2
}
],
"FragmentSelector": {
"FragmentSelectorType": "selector_type", // Specify "server_timestamp" or "producer_timestamp" as the fragment selector type
"TimestampRange": {
"StartTimestamp": epoch_time_seconds,
"EndTimestamp": epoch_time_seconds
}
}
},
"S3RecordingSinkRuntimeConfiguration": {
"Destination": arn:aws:s3:::bucket_name/prefix/optional_file_name,
"RecordingFileFormat": file_format // Specify "Opus" or "WAV" as the recording file format, if you want to override the configuration
}
}
```
The `MediaInsightsPipelineConfigurationArn` is the configuration ARN that you receive after you create a call analytics configuration.
## Setting destinations, names, and formats
The following example uses a folder named `MyRecordingBucket` as the `S3SinkConfiguration.Destination` value, and `Opus` as the `RecordingFileFormat` value.
```
arn:aws:s3:::MyRecordingBucket/voice-connector-id/transaction-id_year-month-date-hour-minute-second-millisecond.ogg
```
The following example uses `MyRecordingBucket` as the `S3SinkConfiguration.Destination` value, and `Wav` as the `RecordingFileFormat` value.
```
arn:aws:s3:::MyRecordingBucket/voice-connector-id/transaction-id_year-month-date-hour-minute-second-millisecond.wav
```
# Understanding workflows for machine-learning based analytics for the Amazon Chime SDK
The following sections describe how to use the machine-learning analytics features provide by Amazon Chime SDK call analytics.
**Note**
If you plan to run multiple machine-learning analytics on the same Kinesis Video Stream, you may need to increase the connection-level limit for `GetMedia` and `GetMediaForFragmentList` for the video stream. For more information, refer to [Kinesis Video Streams limits](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/limits.html) in the *Kinesis Video Streams Developer Guide*.
# Using Voice Connectors to initiate call analytics automatically
Use this workflow when:
+ You want console-driven setup.
+ You already use or plan to use a Voice Connector to bring SIP media into call analytics. Voice Connectors support SIP as well as SIPREC. For more information on configuring Voice Connectors, refer to [Managing Amazon Chime SDK Voice Connector](https://docs.aws.amazon.com/chime-sdk/latest/ag/voice-connectors.html).
+ You want to apply the same media insights configuration to every Voice Connector call.
+ You need to use Amazon Chime SDK voice analytics, which requires a Voice Connector or a media insights pipeline.
To enable this workflow in the Amazon Chime SDK console, follow the steps for creating a recording configuration in [Configuring Voice Connectors to use call analytics.](https://docs.aws.amazon.com/chime-sdk/latest/ag/configure-voicecon.html)
To enable this workflow programmatically, use the following APIs: [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API to create a call analytics configuration and then associated the configuration to a Voice Connector using the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_PutVoiceConnectorStreamingConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_PutVoiceConnectorStreamingConfiguration.html) API. For more information, see [Configuring Voice Connectors to use voice analytics](https://docs.aws.amazon.com/chime-sdk/latest/ag/configure-voicecon.html) in the *Amazon Chime SDK Administrator Guide*.
The following diagram shows the flow of data when a Voice Connector initiates a call analytics session. Numbers in the diagram correspond to the numbered text below.
![\[Image showing the flow of data when a Voice Connector initiates a call.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/call-analytics-workflow-1.png)
In the diagram:
1. You use the Amazon Chime SDK console or the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API to the create a media insights pipeline configuration.
1. You use the Amazon Chime SDK console or the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_PutVoiceConnectorStreamingConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_PutVoiceConnectorStreamingConfiguration.html) API to associate the configuration with a Voice Connector. To associate an existing configuration with a Voice Connector, refer to [ Configuring Voice Connectors to use call analytics](https://docs.aws.amazon.com/chime-sdk/latest/ag/configure-voicecon.html), in the *Amazon Chime SDK Administrator Guide*.
1. During an outgoing call, the Voice Connector receives each call participant's audio.
1. Because of built-in integration with call analytics, if a call analytics configuration is attached to a Voice Connector, the Voice Connector service initiate a call analytics session using the media pipeline service.
1. The media pipeline service invokes one or more media processors as specified in the configuration.
1. The media pipeline service sends the output data to one or more destinations based on the configuration. For example, you can send real-time analytics via an Amazon Kinesis Data Stream, and if configured, you can send the call metadata and analytics to an Amazon S3 data warehouse.
1. The media pipeline service sends the pipeline status events to the default Amazon EventBridge. If you have configured rules then the notifications for them will be sent to the Amazon EventBridge as well. For more information see, [ Using EventBridge notifications](https://docs.aws.amazon.com/chime-sdk/latest/dg/ca-eventbridge-notifications.html).
**Note**
A voice analytics processor only starts automatically when you call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartVoiceToneAnalysisTask.html) APIs.
You must enable Voice Connector streaming to use call analytics with Voice Connector. This feature enables streaming of call data to Voice Connector managed Kinesis Video Streams in your account. For more information, refer to [ Streaming Amazon Chime SDK Voice Connector media to Kinesis Video Streams](https://docs.aws.amazon.com/chime-sdk/latest/ag/start-kinesis-vc.html) in the *Amazon Chime SDK Administrator Guide*.
You can store Voice Connector call data in Kinesis Video Streams for varying amounts of time, ranging from hours to years. Opting for no data retention limits the usability of the call data for immediate consumption. The cost of Kinesis Video Streams is determined based on the bandwidth and total storage utilized. It is possible to adjust the data retention period at any time by editing your Voice Connector's streaming configuration. To enable call analytics recording, you must ensure that the Kinesis Video Stream retains data until call analytics finishes. You do that by specifying a suitable data retention period.
You can associate a media insights pipeline configuration with as many Voice Connectors as you want. You can also create a different configuration for each Voice Connector. Voice Connectors use the AWSServiceRoleForAmazonChimeVoiceConnector to call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API on your behalf once per transaction ID. For information about the role, see [ Using the Amazon Chime SDK service-linked role for Amazon Chime SDK Voice Connectors](https://docs.aws.amazon.com/chime-sdk/latest/ag/using-service-linked-roles-stream.html#service-linked-role-permissions-stream) in the *Amazon Chime SDK Administrator Guide*.
# Using call analytics APIs with Voice Connectors
Use this workflow if you use a Voice Connector but need to control when you apply a call analytics configuration and which call to apply the configuration to.
To use this method, you need to create an EventBridge target for events that the Voice Connector publishes, and then use the events to trigger the call analytics pipeline APIs. For more information, see [ Automating the Amazon Chime SDK with EventBridge](https://docs.aws.amazon.com/chime-sdk/latest/ag/automating-chime-with-cloudwatch-events.html) in the *Amazon Chime SDK Administrator Guide*.
The following diagram shows how to implement more granular control when using call analytics with Voice Connector. Numbers in the diagram correspond to numbers in the text below.
![\[Image showing the flow of data when using API calls with Voice Connectors.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/analytics-vc-with-apis.png)
In the diagram:
1. You use the Amazon Chime SDK console or the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API to the create a media insights pipeline configuration.
1. During an outgoing call the Voice Connector will receive participant audio.
1. The Voice Connector sends call audio to Kinesis Video Stream and corresponding events to the EventBridge. These events have stream and call metadata.
1. Your application is subscribed to EventBridge via an EventBridge Target.
1. Your application invokes the Amazon Chime SDK [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API.
1. The media pipeline service invokes one or more media processors based on the processor elements in the media insights pipeline configuration.
1. The media pipeline service sends the output data to one or more destinations based on the configuration. Amazon Chime SDK call analytics will provide real-time analytics via Amazon Kinesis Data Stream and if configured call metadata analytics to an Amazon S3 data warehouse.
1. The media pipeline service sends the events to the Amazon EventBridge. If you have configured rules then the notifications for them will be sent to the Amazon EventBridge as well.
1. You can pause or resume the call analytics session by invoking the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineStatus.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineStatus.html) API.
**Note**
Call recording does not support pausing and resuming calls. Also, voice analytics tasks started for the call also stop when you pause a session. To restart them, you must call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html) APIs.
1. If you select voice tone analytics during configuration, you start voice analytics by calling the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html) APIs.
# Using call analytics with Kinesis Video Streams producers
To use this option, you need to publish audio data to Kinesis Video Streams (KVS) and then call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API with KVS stream channel information.
**Note**
The call analytics APIs support a maximum of two audio channels.
When calling the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API, you can specify fragment numbers for each KVS stream channel definition. If you supply a fragment number, call analytics begins processing the stream at that fragment. Otherwise, call analytics begins processing the stream from the latest available fragment.
Call analytics supports PCM audio (only signed 16-bit little-endian audio formats, which does not include WAV) with an audio sample rate between 8kHz and 48kHz. Low-quality audio, such as telephony audio, is typically around 8,000 Hz. High-quality audio typically ranges from 16,000 Hz to 48,000 Hz. The sample rate that you specify must match that of your audio. For more information, see [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_KinesisVideoStreamSourceRuntimeConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_KinesisVideoStreamSourceRuntimeConfiguration.html) in the *Amazon Chime SDK API Reference*.
The Kinesis Video Streams Producer SDK provides a set of libraries that you can use to stream audio data to a Kinesis Video Stream. For more information, refer to [ Kinesis Video Streams Producer Libraries](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/producer-sdk.html), in the *Amazon Kinesis Video Streams Developer Guide*.
The following diagram shows the flow of data when using call analytics with a custom Kinesis Video Stream producer. Numbers in the diagram correspond to the numbered text below.
![\[Image showing the flow of data when using call analytics with a Kinesis Video Stream producer.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/analytics-kvs-with-apis.png)
1. You use the AWS console or the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API to create a media insights pipeline configuration.
1. You use a Kinesis Video Stream Producer to write audio to Kinesis Video Streams.
1. Your application invokes the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API.
1. The media pipeline service reads audio from the customer's Kinesis Video Streams.
1. The media pipeline service sends the events to the Amazon EventBridge. If you have configured rules then the notifications for them will be sent to the Amazon EventBridge as well.
1. The media pipeline service invokes one or more processor elements.
1. The media pipeline service sends output data to one or more sink elements.
1. You can pause or resume the call analytics session by invoking the [ UpdateMediaInsightsPipelineStatus](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineStatus.html) API.
**Note**
Call recording does not support pause and resume.
1. Your application can process the Amazon EventBridge events to trigger custom business workflows.
1. If you select voice analytics when you create a configuration, your application can start voice analytics by calling the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html) APIs.
# Managing call analytics pipelines for the Amazon Chime SDK
You can read, list, and delete media insights pipelines by calling the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetMediaPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetMediaPipeline.html), [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_ListMediaPipelines.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_ListMediaPipelines.html), and [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_DeleteMediaPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_DeleteMediaPipeline.html) APIs.
Media insights pipelines stop if any of the following conditions are met:
+ Any of the Kinesis Video streams send no new fragments to an `InProgress` pipeline for 15 seconds.
+ The [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_DeleteMediaPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_DeleteMediaPipeline.html) API is called.
+ The media insights pipeline was created more than 8 hours ago. The system stops the pipeline automatically.
+ The media insights pipeline is paused for more than 2 hours. The system stops the pipeline automatically.
# Pausing and resuming call analytics pipelines for the Amazon Chime SDK
To pause and resume a media insights pipeline, invoke the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineStatus.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineStatus.html) API with a `Pause` or `Resume` action. To do so, you pass either the pipeline's ID or ARN in the `Identifier` field.
**Warning**
Warning: The `UpdateMediaInsightsPipelineStatus` API *stops* all voice analytics tasks started on a media insights pipeline when a `Pause` status is provided. When the `Resume` status is provided, tasks are not resumed and must be started again. You must provide all necessary notices and obtain all necessary consents from the speakers prior to re-starting the tasks. For more information, refer to [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html), in the *Amazon Chime SDK API Reference*.
While paused, the pipeline stops sending media to processors and writing data to Kinesis Data Streams and data warehouses. When you `Resume` the pipeline, the service sends the latest chunk available on the stream. Media insights pipelines stop automatically when paused for more than 2 hours. **Please note**, call recording does not support pause and resume.
For more, see the following topics:
+ [Using EventBridge notifications](https://docs.aws.amazon.com/chime-sdk/latest/dg/ca-eventbridge-notifications.html).
+ [https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/API_dataplane_StartSelector.html#KinesisVideo-Type-dataplane_StartSelector-StartSelectorType](https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/API_dataplane_StartSelector.html#KinesisVideo-Type-dataplane_StartSelector-StartSelectorType) in the *Amazon Kinesis Video Streams Developer Guide*.
+ [Amazon Transcribe call analytics processor](https://docs.aws.amazon.com/chime-sdk/latest/dg/ca-processors-sinks.html#ca-transcribe-analytics-processor).
**Note**
You are billed for call analytics usage while a pipeline is paused. However, you aren't billed for AWS services accessed via the resource access role, such as Amazon Transcribe and Amazon Kinesis.
You can read, update, and delete existing call analytics configurations using [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetMediaInsightsPipelineConfiguration.html), [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineConfiguration.html), and [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_DeleteMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_DeleteMediaInsightsPipelineConfiguration.html) APIs by passing the configuration name or ARN in the Identifier field.
You can list configurations by calling the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_media-pipelines-chime_ListMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_media-pipelines-chime_ListMediaInsightsPipelineConfiguration.html) API.
# Using the call analytics resource access role for the Amazon Chime SDK
The calling account must create the resource access role used by a media insights pipeline configuration. You can't use cross-account roles.
Depending on the features that you enable when you create a call analytics configuration, you must use additional resource policies. Expand the following sections to learn more.
## Minimum required policy
The role requires the following policy, at a minimum:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"transcribe:StartCallAnalyticsStreamTranscription",
"transcribe:StartStreamTranscription"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"kinesisvideo:GetDataEndpoint",
"kinesisvideo:GetMedia"
],
"Resource": "arn:aws:kinesisvideo:us-east-1:111122223333:stream/Chime*"
},
{
"Effect": "Allow",
"Action": [
"kinesisvideo:GetDataEndpoint",
"kinesisvideo:GetMedia"
],
"Resource": "arn:aws:kinesisvideo:us-east-1:111122223333:stream/*",
"Condition": {
"StringLike": {
"aws:ResourceTag/AWSServiceName": "ChimeSDK"
}
}
},
{
"Effect": "Allow",
"Action": ["kms:Decrypt"],
"Resource": "arn:aws:kms:us-east-1:111122223333:key/*",
"Condition": {
"StringLike": {
"aws:ResourceTag/AWSServiceName": "ChimeSDK"
}
}
}
]
}
```
------
You must also use the following trust policy:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "mediapipelines.chime.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "111122223333"
},
"ArnLike": {
"aws:SourceARN": "arn:aws:chime:*:111122223333:*"
}
}
}
]
}
```
------
## KinesisDataStreamSink policy
If you use the `KinesisDataStreamSink`, add the following policy:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"kinesis:PutRecord"
],
"Resource": [
"arn:aws:kinesis:us-east-1:111122223333:stream/output_stream_name"
]
},
{
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey"
],
"Resource": [
"arn:aws:kms:us-east-1:111122223333:key/*"
],
"Condition": {
"StringLike": {
"aws:ResourceTag/AWSServiceName": "ChimeSDK"
}
}
}
]
}
```
------
## S3RecordingSink policy
If you use the `S3RecordingSink`, add the following policy:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:PutObjectAcl",
"s3:PutObjectTagging"
],
"Resource": [
"arn:aws:s3:::input_bucket_path/*"
]
},
{
"Effect": "Allow",
"Action": [
"kinesisvideo:GetDataEndpoint",
"kinesisvideo:ListFragments",
"kinesisvideo:GetMediaForFragmentList"
],
"Resource": [
"arn:aws:kinesisvideo:us-east-1:111122223333:stream/*"
],
"Condition": {
"StringLike": {
"aws:ResourceTag/AWSServiceName": "ChimeSDK"
}
}
},
{
"Effect": "Allow",
"Action": [
"kinesisvideo:ListFragments",
"kinesisvideo:GetMediaForFragmentList"
],
"Resource": [
"arn:aws:kinesisvideo:us-east-1:111122223333:stream/Chime*"
]
},
{
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey"
],
"Resource": [
"arn:aws:kms:us-east-1:111122223333:key/*"
],
"Condition": {
"StringLike": {
"aws:ResourceTag/AWSServiceName": "ChimeSDK"
}
}
}
]
}
```
------
## Post Call Analytics policy
If you use the Post Call Analytics feature of the `AmazonTranscribeCallAnalyticsProcessor`, add the following policy:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws:iam::111122223333:role/transcribe_role_name"
],
"Condition": {
"StringEquals": {
"iam:PassedToService": "transcribe.streaming.amazonaws.com"
}
}
}
]
}
```
------
## VoiceEnhancementSinkConfiguration policy
If you use the `VoiceEnhancementSinkConfiguration` element, add the following policy:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement":[
{
"Effect":"Allow",
"Action":[
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl",
"s3:PutObjectTagging"
],
"Resource":[
"arn:aws:s3:::input_bucket_path/*"
]
},
{
"Effect":"Allow",
"Action":[
"kinesisvideo:GetDataEndpoint",
"kinesisvideo:ListFragments",
"kinesisvideo:GetMediaForFragmentList"
],
"Resource":[
"arn:aws:kinesisvideo:us-east-1:111122223333:stream/*"
],
"Condition":{
"StringLike":{
"aws:ResourceTag/AWSServiceName":"ChimeSDK"
}
}
},
{
"Effect":"Allow",
"Action":[
"kinesisvideo:ListFragments",
"kinesisvideo:GetMediaForFragmentList"
],
"Resource":[
"arn:aws:kinesisvideo:us-east-1:111122223333:stream/Chime*"
]
},
{
"Effect":"Allow",
"Action":[
"kms:GenerateDataKey"
],
"Resource":[
"arn:aws:kms:us-east-1:111122223333:key/*"
],
"Condition":{
"StringLike":{
"aws:ResourceTag/AWSServiceName":"ChimeSDK"
}
}
}
]
}
```
------
## VoiceAnalyticsProcessor policy
If you use the `VoiceAnalyticsProcessor`, add the policies for `LambdaFunctionSink`, `SqsQueueSink`, and `SnsTopicSink` depending on which sinks you have defined.
`LambdaFunctionSink` policy:
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"lambda:InvokeFunction",
"lambda:GetPolicy"
],
"Resource": [
"arn:aws:lambda:us-east-1:111122223333:function:function_name"
],
"Effect": "Allow"
}
]
}
```
`SqsQueueSink` policy
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"sqs:SendMessage",
"sqs:GetQueueAttributes"
],
"Resource": [
"arn:aws:sqs:us-east-1:111122223333:queue_name"
],
"Effect": "Allow"
},
{
"Effect": "Allow",
"Action": ["kms:GenerateDataKey", "kms:Decrypt"],
"Resource": "arn:aws:kms:us-east-1:111122223333:key/*",
"Condition": {
"StringLike": {
"aws:ResourceTag/AWSServiceName": "ChimeSDK"
}
}
}
]
}
```
`SnsTopicSink` policy:
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"sns:Publish",
"sns:GetTopicAttributes"
],
"Resource": [
"arn:aws:sns:us-east-1:111122223333:topic_name"
],
"Effect": "Allow"
},
{
"Effect": "Allow",
"Action": ["kms:GenerateDataKey", "kms:Decrypt"],
"Resource": "arn:aws:kms:us-east-1:111122223333:key/*",
"Condition": {
"StringLike": {
"aws:ResourceTag/AWSServiceName": "ChimeSDK"
}
}
}
]
}
```
# Understanding the call analytics statuses for the Amazon Chime SDK
Media insights pipelines track a set of statuses when you do one or both of the following:
+ Use multiple machine-learning processing elements, such as Amazon Transcribe and voice analytics.
+ Enable call recording with or without machine learning processing.
To get pipeline and element statuses, use the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetMediaPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetMediaPipeline.html) API and [EventBridge notifications](using-eventbridge-notifications.md).
To get statuses for voice analytics tasks, use the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetSpeakerSearchTask.html) and [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetVoiceToneAnalysisTask.html) APIs, and [voice analytics notification targets](va-notification-targets.md).
Media insights pipelines track the following statuses.
+ **Pipeline status** – The overall status of a call analytics pipeline, also known as a media insights pipeline. This is determined by the element statuses.
+ **Element status** – The processing status for the individual media insights pipeline configuration elements.
+ **Task status** – The processing status for a media insights pipeline task started for voice analytics. The `VoiceAnalyticsProcessor` element status is determined by the task statuses. No other element in a call analytics pipeline has a task status.
For more information about media insights pipeline tasks, refer to [Understanding call analytics terminology for the Amazon Chime SDK](ca-terms-concepts.md), earlier in this guide.
Not all media insights configuration element types have element statuses. In general, only media insights configuration elements of the “processor” type have an element status. Also, the Amazon S3 recording and voice enhancement sinks have processor statuses. Specifically, element statuses exist for the following media insights configuration element types:
+ `AmazonTranscribeProcessor`
+ `AmazonTranscribeCallAnalyticsProcessor`
+ `S3RecordingSink`
+ `VoiceAnalyticsProcessor`
+ `VoiceEnhancementSink`
The pipeline status is determined by the element statuses as follows:
| Pipeline status | Condition |
| --- | --- |
| NotStarted | All element statuses are not started. |
| Initializing | At least one element is initializing and the rest are not started. |
| InProgress | At least one element is in progress. |
| Failed | At least one element has failed and the remaining elements are stopped. |
| Stopping | Refer to [Managing call analytics pipelines for the Amazon Chime SDK](managing-call-analytics-pipelines.md) for a complete list of stopping conditions. |
| Stopped | All elements are stopped. |
| Paused | All elements are paused. |
Unlike other element statuses, the `VoiceAnalyticsProcessor` element has a few nuances. As mentioned earlier, the `VoiceAnalyticsProcessor` element status, corresponding to the Amazon Chime SDK voice analytics feature, is determined by the task statuses created from [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html) and [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html).
+ The `VoiceAnalyticsProcessor`’s element status begins in a `NotStarted` state, because `StartSpeakerSearchTask` and `StartVoiceToneAnalysisTask` must be called before the element can change the status to `Initializing`, and then `InProgress`.
+ The `VoiceAnalyticsProcessor` stays `InProgress` as long as one task is started and a [stop condition](managing-call-analytics-pipelines.md) is not met while the task is running.
+ Even though the `VoiceAnalyticsProcessor` may be `InProgress`, you will only be charged for the duration that the tasks process.
+ To clean-up media insights pipelines that had at least one voice analytics task started and no more tasks running, you must call `DeleteMediaPipeline`.
+ As long as a task runs or completes successfully, the `VoiceAnalyticsProcessor` element status remains at `InProgress`.
# Monitoring call analytics pipelines for the Amazon Chime SDK with Amazon CloudWatch
You can use Amazon CloudWatch to monitor Amazon Chime SDK call analytics pipelines. You can also set alarms that watch for certain thresholds, and send notifications or take actions when those thresholds are met. For more information about CloudWatch, see the [Amazon CloudWatch User Guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/).
**Topics**
+ [Prerequisites](#monitoring-prereqs)
+ [Call analytics metrics](#monitoring-metrics)
+ [CloudWatch dimensions for pipeline metrics](#monitoring-dimensions)
## Prerequisites
To use CloudWatch metrics, you must first create a media pipelines service-linked role that grants permissions to publish service metrics to Amazon CloudWatch. For more information about the service-linked role, see [Creating a service-linked role for Amazon Chime SDK media pipelines](create-pipeline-role.md), in this guide.
## Call analytics metrics
Amazon Chime SDK call analytics publishes the following metrics to the `AWS/ChimeSDK` namespace for media insights pipelines that you create by using a media insights configuration.
| Metric | Description |
| --- | --- |
| `MediaInsightsPipelineCreated` | The media insights pipeline was successfully created. Unit: Count |
| `MediaInsightsPipelineStopped` | The media insights pipeline was successfully stopped. Unit: Count |
| `MediaInsightsPipelineFailed` | The media insights pipeline failed. Unit: Count |
| `MediaInsightsPipelineDuration` | The time between pipeline creation and Stopped/Failed. Unit: Seconds |
| `MediaInsightsPipelineBillingDuration` | The billing duration of the media insights pipeline. Unit: Count |
| `RecordingFileSize` | The size of the recording file. Unit: Bytes |
| `RecordingDuration ` | The duration of the recording. Unit: Seconds |
## CloudWatch dimensions for pipeline metrics
The following table lists the CloudWatch dimensions that you can use to monitor call analytics pipelines.
| Dimension | Description |
| --- | --- |
| `MediaInsightsPipelineConfigurationId` | The ID of the media insights pipeline configuration. |
| `MediaInsightsPipelineConfigurationName` | The name of the media insights pipeline configuration. |
# Call analytics processor and output destinations for the Amazon Chime SDK
You can only specify unique elements once per media insights pipeline configuration. All processors and sinks must reside in the same AWS account, and you must create them in the same AWS Region as the endpoint that you call. For example, if you use the `us-east-1` endpoint for Amazon Chime SDK Media Pipelines, you can't pass a Kinesis Data Stream from the `us-west-2` region.
Expand each section for information about each destination.
## Amazon Transcribe Call Analytics processor destinations
Supported sinks: `KinesisDataStreamSink`.
You can't combine this processor with an Amazon Transcribe processor. For more information about Amazon Transcribe Call Analytics, refer to [Real-time call analytics](https://docs.aws.amazon.com/transcribe/latest/dg/call-analytics-streaming.html), in the* Amazon Transcribe Developer Guide*. If you enable [Post call analytics](https://docs.aws.amazon.com/transcribe/latest/dg/tca-post-call.html) by including `PostCallAnalyticsSettings` in the `AmazonTranscribeCallAnalyticsProcessorConfiguration` API call, you receive artifacts in the specified Amazon S3 location when the media insights pipeline stops and processing finishes.
**Note**
If you pause the pipeline for more than 35 seconds and then resume it, post-call artifacts are generated in separate files with different session IDs in the Amazon S3 bucket.
Post-call artifacts include an analytics JSON file and audio recording WAV or Opus file. The Amazon S3 bucket URL for redacted (if you enable content redaction) and non-redacted recording files is sent to the Kinesis Data Stream once for each Amazon Transcribe call analytics Post Call session as part of `onetimeMetadata` in the metadata section.
Call analytics with Amazon Transcribe call analytics takes audio data input from the Kinesis Video Stream.
+ Supported media encoding: PCM signed 16-bit little-endian audio.
+ Supported media sample rates: Between 8,000 Hz and 48,000 Hz.
`StreamConfiguration` input for an Amazon Transcribe Analytics process:
+ You must specify the `KinesisVideoStreamArn` for each stream.
+ (Optional) The KVS `FragmentNumber` starts a call analytics job with the chunk after a specified fragment. If not provided, it uses the latest chunk on the Kinesis video stream.
+ The `StreamChannelDefinition` defines who's talking. Amazon Transcribe call analytics requires two-channel audio. You must specify which speaker is on which channel when you call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API. For example, if your agent speaks first, you set the `ChannelId` to `0` to indicate the first channel, and `ParticipantRole` to `AGENT` to indicate that the agent is speaking.
**Note**
When you use a Voice Connector to create a `MediaInsightsPipeline` with an Amazon Transcribe call analytics processor, the Voice Connector account leg audio is `AGENT` and the PSTN leg audio is `CUSTOMER` for the `ParticipantRole`.
For Voice Connector SIPREC, we rely on the SIPREC metadata. In most cases, the stream label with the lowest lexicographic value is considered to be the `AGENT`.
The following example shows Kinesis Video Stream input for one dual-channel audio stream.
```
"StreamChannelDefinition" : {
"NumberOfChannels" : 2
"ChannelDefinitions": [
{
"ChannelId": 0,
"ParticipantRole": "AGENT"
},
{
"ChannelId": 1,
"ParticipantRole": "CUSTOMER"
}
]
}
```
In contrast, the following example shows two mono inputs from two different Kinesis Video streams.
```
KVS-1:
"StreamChannelDefinition" : {
"NumberOfChannels" : 1
"ChannelDefinitions": [
{
"ChannelId": 0,
"ParticipantRole": "AGENT"
}
]
}
KVS-2:
"StreamChannelDefinition" : {
"NumberOfChannels" : 1
"ChannelDefinitions": [
{
"ChannelId": 1,
"ParticipantRole": "CUSTOMER"
}
]
}
```
## Amazon Transcribe call analytics output
Each Amazon Transcribe record contains an `UtteranceEvent` or a `CategoryEvent`, but not both. `CategoryEvents` have a `detail-type` of `TranscribeCallAnalyticsCategoryEvent`.
The following example shows one-time metadata output format for Amazon Transcribe.
```
{
"time": "string", // ISO8601 format
"service-type": "CallAnalytics",
"detail-type": "CallAnalyticsMetadata",
"mediaInsightsPipelineId": "string",
"metadata": "string" // JSON encoded string of the metadata object
}
// metadata object
{
"voiceConnectorId": "string",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"direction": "string",
"oneTimeMetadata": "string" // JSON encoded string of oneTimeMetadata object
}
// onetimeMetadata object
{
"inviteHeaders": "string", // JSON encoded string of SIP Invite headers key-value pair
"siprecMetadata": "string", // siprec metadata in XML
"siprecMetadataJson": "string", // siprec metadata in JSON (converted from above XML)
// If PostcallSettings are enabled for Amazon Transcribe Call Analytics
"s3RecordingUrl": "string",
"s3RecordingUrlRedacted": "string"
}
// inviteHeaders object
{
"string": "string"
}
```
The following example shows the Amazon Transcribe Call Analytics output format.
```
{
"time": "string", // ISO8601 format
"service-type": "CallAnalytics",
"detail-type": "TranscribeCallAnalytics",
"mediaInsightsPipelineId": "string",
"metadata": {
"voiceConnectorId": "string",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"direction": "string"
},
"UtteranceEvent": {
"UtteranceId": "string",
"ParticipantRole": "string",
"IsPartial": boolean,
"BeginOffsetMillis": number,
"EndOffsetMillis": number,
"Transcript": "string",
"Sentiment": "string",
"Items": [{
"Content": "string",
"Confidence": number,
"VocabularyFilterMatch": boolean,
"Stable": boolean,
"ItemType": "string",
"BeginOffsetMillis": number,
"EndOffsetMillis": number,
}, ]
"Entities": [{
"Content": "string",
"Confidence": number,
"Category": "string", // Only PII is supported currently
"Type": "string",
"BeginOffset": number,
"EndOffset": number,
}, ],
"IssuesDetected": [{
"CharacterOffsets": {
"Begin": number,
"End": number
}
}]
},
"CategoryEvent": {
"MatchedCategories": ["string"],
"MatchedDetails": {
"string": {
"TimestampRanges": [{
"BeginOffsetMillis": number,
"EndOffsetMillis": number
}]
}
}
}
}
```
## Amazon Chime SDK Voice Connector streaming updates metadata
If the call analytics configuration is associated with an Amazon Chime SDK Voice Connector, the following Voice Connector Update payload will be sent when there is a [Voice Connector streaming update](https://docs.aws.amazon.com/chime-sdk/latest/ag/automating-chime-with-cloudwatch-events.html).
The following example shows an update metadata format for Amazon Transcribe processor and Transcribe Call Analytics processor.
```
{
"time": "string", // ISO8601 format
"service-type": "CallAnalytics",
"detail-type": "CallAnalyticsMetadata",
"callevent-type": "Update",
"metadata": "string" // JSON encoded string of the metadata object
}
// metadata object
{
"voiceConnectorId": "string",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"direction": "string",
"oneTimeMetadata": "string" // JSON encoded string of oneTimeMetadata object
}
// onetimeMetadata object
{
"sipHeaders": "string", // JSON encoded string of SIP Invite headers key-value pair
"siprecMetadata": "string", // siprec metadata in XML
"siprecMetadataJson": "string" // siprec metadata in JSON (converted from above XML)
}
// sipHeaders object
{
"string": "string"
}
```
The following example shows an update metadata format for Call Analytics Amazon S3 Recording.
```
{
"time": "string", // ISO8601 format
"service-type": "CallAnalytics",
"detail-type": "Recording",
"callevent-type": "Update",
"metadata": "string" // JSON encoded string of the metadata object
}
// metadata object
{
"voiceConnectorId": "string",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"direction": "string",
"oneTimeMetadata": "string" // JSON encoded in string of oneTimeMetadata object
}
// onetimeMetadata object
{
"sipHeaders": "string", // JSON encoded string of SIP Invite headers key-value pair
"siprecMetadata": "string", // siprec metadata in XML
"siprecMetadataJson": "string" // siprec metadata in JSON (converted from above XML)
}
// sipHeaders object
{
"string": "string"
}
```
## SIP call recording metadata
The following examples show the metadata for recording a SIP call between two people, Alice and Bob. Both participants send and receive audio and video. For simplicity, the example only has snippets of SIP and SDP, and SRC records the streams of each participant to SRS without mixing.
```
INVITE sip:recorder@example.com SIP/2.0
Via: SIP/2.0/TCP src.example.com;branch=z9hG4bKdf6b622b648d9
From: ;tag=35e195d2-947d-4585-946f-09839247
To:
Call-ID: d253c800-b0d1ea39-4a7dd-3f0e20a
Session-ID: ab30317f1a784dc48ff824d0d3715d86
;remote=00000000000000000000000000000000
CSeq: 101 INVITE
Max-Forwards: 70
Require: siprec
Accept: application/sdp, application/rs-metadata,
application/rs-metadata-request
Contact: ;+sip.src
Content-Type: multipart/mixed;boundary=boundary
Content-Length: [length]
Content-Type: application/SDP
...
m=audio 49170 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=label:96
a=sendonly
...
m=video 49174 RTP/AVPF 96
a=rtpmap:96 H.264/90000
a=label:97
a=sendonly
...
m=audio 51372 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=label:98
a=sendonly
...
m=video 49176 RTP/AVPF 96
a=rtpmap:96 H.264/90000
a=label:99
a=sendonly
....
Content-Type: application/rs-metadata
Content-Disposition: recording-session
complete2010-12-16T23:41:07Zsip:alice@atlanta.comstructure!structureab30317f1a784dc48ff824d0d3715d86;
remote=47755a9de7794ba387653f2099600ef27+OTCyoxTmqmqyA/1weDAg==
FOO!barAlice
FOO!barBobFOO!bar2010-12-16T23:41:07Z2010-12-16T23:41:07Z2010-12-16T23:41:07Zi1Pz3to5hGk8fuXl+PbwCw==UAAMm5GRQKSCMVvLyl4rFw==8zc6e0lYTlWIINA6GR+3ag==EiXGlc+4TruqqoDaNE76ag==8zc6e0lYTlWIINA6GR+3ag==EiXGlc+4TruqqoDaNE76ag==UAAMm5GRQKSCMVvLyl4rFw==i1Pz3to5hGk8fuXl+PbwCw==
```
The following example shows the updated metadata when one call participant puts the other on hold. In this case, `participant_id srfBElmCRp2QB23b7Mpk0w==` only receives media streams and does not send any media, so the `send` XML element is omitted. In contrast, `participant_id zSfPoSvdSDCmU3A3TRDxAw==` sends media to, but does not receive media from, the other participant, so the `recv` XML element is omitted.
```
INVITE sip:recorder@example.com SIP/2.0
Via: SIP/2.0/TCP src.example.com;branch=z9hG4bKdf6b622b648d9
From: ;tag=35e195d2-947d-4585-946f-09839247
To:
Call-ID: d253c800-b0d1ea39-4a7dd-3f0e20a
Session-ID: ab30317f1a784dc48ff824d0d3715d86
;remote=f81d4fae7dec11d0a76500a0c91e6bf6
CSeq: 101 INVITE
Max-Forwards: 70
Require: siprec
Accept: application/sdp, application/rs-metadata,
application/rs-metadata-request
Contact: ;+sip.src
Content-Type: multipart/mixed;boundary=foobar
Content-Length: [length]
Content-Type: application/SDP
...
m=audio 49170 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=label:96
a=sendonly
...
m=video 49174 RTP/AVPF 96
a=rtpmap:96 H.264/90000
a=label:97
a=sendonly
...
m=audio 51372 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=label:98
a=sendonly
...
m=video 49176 RTP/AVPF 96
a=rtpmap:96 H.264/90000
a=label:99
a=sendonly
....
Content-Type: application/rs-metadata
Content-Disposition: recording-session
partial8zc6e0lYTlWIINA6GR+3ag==EiXGlc+4TruqqoDaNE76ag==8zc6e0lYTlWIINA6GR+3ag==EiXGlc+4TruqqoDaNE76ag==
```
The following example shows the metadata update when the call resumes. The payload now has the `send` and `recv` XML elements.
```
INVITE sip:recorder@example.com SIP/2.0
Via: SIP/2.0/TCP src.example.com;branch=z9hG4bKdf6b622b648d9
From: ;tag=35e195d2-947d-4585-946f-09839247
To:
Call-ID: d253c800-b0d1ea39-4a7dd-3f0e20a
Session-ID: ab30317f1a784dc48ff824d0d3715d86
;remote=f81d4fae7dec11d0a76500a0c91e6bf6
CSeq: 101 INVITE
Max-Forwards: 70
Require: siprec
Accept: application/sdp, application/rs-metadata,
application/rs-metadata-request
Contact: ;+sip.src
Content-Type: multipart/mixed;boundary=foobar
Content-Length: [length]
Content-Type: application/SDP
...
m=audio 49170 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=label:96
a=sendonly
...
m=video 49174 RTP/AVPF 96
a=rtpmap:96 H.264/90000
a=label:97
a=sendonly
...
m=audio 51372 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=label:98
a=sendonly
...
m=video 49176 RTP/AVPF 96
a=rtpmap:96 H.264/90000
a=label:99
a=sendonly
....
Content-Type: application/rs-metadata
Content-Disposition: recording-session
partiali1Pz3to5hGk8fuXl+PbwCw==UAAMm5GRQKSCMVvLyl4rFw==8zc6e0lYTlWIINA6GR+3ag==EiXGlc+4TruqqoDaNE76ag==8zc6e0lYTlWIINA6GR+3ag==EiXGlc+4TruqqoDaNE76ag==i1Pz3to5hGk8fuXl+PbwCw==UAAMm5GRQKSCMVvLyl4rFw==
```
## Amazon Transcribe processor destinations
Supported sinks: `KinesisDataStreamSink`.
You can't combine this processor with the Amazon Transcribe call analytics. For more information on the input to, and output of, Amazon Transcribe, refer to [ Transcribe streaming audio](https://docs.aws.amazon.com/transcribe/latest/dg/streaming.html) in the *Amazon Transcribe Developer Guide*.
Call analytics session with Amazon Transcribe takes audio data input from Kinesis Video Stream.
+ Supported MediaEncoding: PCM signed 16-bit little-endian audio.
+ Supported MediaSampleRate sample rates: Between 8,000 Hz and 48,000 Hz.
`StreamConfiguration` input for Amazon Transcribe processors:
+ You must specify the `KinesisVideoStreamArn` for each stream.
+ (Optional) KVS `FragmentNumber` - Starts a call analytics job with the chunk after a specific fragment. If not provided, it will use the latest available chunk on the Kinesis Video Stream.
+ `StreamChannelDefinition` Amazon Transcribe currently supports audio with two channels. You have to specify the `NumberOfChannels` in the runtime `StreamChannelDefinition`. Additionally, you must pass the `ChannelId` if you send mono audio in two separate channels. In your transcript, channels are assigned the labels `ch_0` and `ch_1`. The following example shows the KVS input for one mono audio channel stream.
```
"StreamChannelDefinition" : {"
NumberOfChannels" : 1
}
```
The following example shows the KVS input for two mono audio inputs in two different streams.
```
KVS-1:
"StreamChannelDefinition" : {
"NumberOfChannels" : 1
"ChannelDefinitions": [
{
"ChannelId": 0
}
]
}
KVS-2:
"StreamChannelDefinition" : {
"NumberOfChannels" : 1
"ChannelDefinitions": [
{
"ChannelId": 1
}
]
}
```
**Note**
For the Voice Connector created `MediaInsightsPipeline` with an Amazon Transcribe processor, the Voice Connector account leg audio is assigned to `channel-0` and the PSTN leg audio to `channel-1`.
For Voice Connector SIPREC, we rely on the SIPREC metadata. In most cases, the stream label with the lowest lexicographic value is assigned to `channel-0`.
For the Amazon Transcribe and Amazon Transcribe call analytics processors, if you pass two Kinesis Video streams, and each stream contains a mono audio channel, we interleave both channels to a single audio stream before processing Transcribe or Transcribe call analytics data.
## Amazon Transcribe output
The following example shows a one-time metadata output format for Amazon Transcribe.
```
{
"time": "string", // ISO8601 format
"service-type": "CallAnalytics",
"detail-type": "CallAnalyticsMetadata",
"mediaInsightsPipelineId": "string",
"metadata": "string" // JSON encoded string of the metadata object
}
// metadata object
{
"voiceConnectorId": "string",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"direction": "string",
"oneTimeMetadata": "string" // JSON encoded string of oneTimeMetadata object
}
// onetimeMetadata object
{
"inviteHeaders": "string", // JSON encoded string of SIP Invite headers key-value pair
"siprecMetadata": "string", // siprec metadata in XML
"siprecMetadataJson": "string" // siprec metadata in JSON (converted from above XML)
}
// inviteHeaders object
{
"string": "string"
}
```
The following example shows the Amazon Transcribe output format.
```
{
"time": "string", // ISO8601 format
"service-type": "CallAnalytics",
"detail-type": "Transcribe",
"mediaInsightsPipelineId": "string",
"metadata": {
"voiceconnectorId": "string",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"direction": "string"
}
"TranscriptEvent": {
"Transcript": {
"Results": [{
"Alternatives": [{
"Entities": [{
"Category": "string",
"Confidence": number,
"Content": "string",
"EndTime": number,
"StartTime": number,
"Type": "string"
}],
"Items": [{
"Confidence": number,
"Content": "string",
"EndTime": number,
"Speaker": "string",
"Stable": boolean,
"StartTime": number,
"Type": "string",
"VocabularyFilterMatch": boolean
}],
"Transcript": "string"
}],
"ChannelId": "string",
"EndTime": number,
"IsPartial": boolean,
"LanguageCode": "string",
"LanguageIdentification": [{
"LanguageCode": "string",
"Score": number
}],
"ResultId": "string",
"StartTime": number
}]
}
}
}
```
## Voice analytics processor destinations
Supported sinks: `KinesisDataStreamSink`, `SqsQueueSink`, `SnsTopicSink`, and `LambdaFunctionSink`.
You can combine this processor with the Amazon Transcribe call analytics processor, the Amazon Transcribe processor, or call recording. You must use the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html) APIs to invoke a voice analytics processor. For more information about using voice analytics, refer to [Using Amazon Chime SDK voice analytics](https://docs.aws.amazon.com/chime-sdk/latest/dg/voice-analytics.html).
## Using Kinesis Data Stream as a sink
The Kinesis Data Stream (KDS) records generated by call analytics include the media pipeline ID, detail type, metadata, and processor-specific sections. For information about consuming data from a Kinesis Data Stream, refer to [Reading Data from Amazon Kinesis Data Streams](https://docs.aws.amazon.com/streams/latest/dev/building-consumers.html), in the *Amazon Kinesis Streams Developer guide*. To create a configuration with this sink, you must have `kinesis:DescribeStream` permission on the specified stream.
**Metadata**
The `metadata` section of the generated KDS records contains any key-value pairs specified in `CallAnalyticsRuntimeMetadata` during the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API call. If a call analytics session was initiated by a Voice Connector, the metadata section is automatically populated with the following parameters:
+ `transactionId`
+ `fromNumber`
+ `toNumber`
+ `callId`
+ `voiceConnectorId`
+ `direction`
In addition to the parameters shown above, the metadata section for Voice Connector initiated call analytics sessions will be populated with a `oneTimeMetadata` field that contains:
+ `inviteHeaders`
+ `siprecMetadata`
This is published to Kinesis Data Streams only once at the beginning of the session and has a `detail-type` of `CallAnalyticsMetadata`.
You can pass unique identifiers in the `MediaInsightsRuntimeMetadata` for each [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API call so that you can uniquely identify the source of each record delivered to your Kinesis Data Stream.
## Amazon S3 call recording
Call analytics recording reads audio from a KVS stream, records it as an audio file, and uploads the file to the specified Amazon S3 Bucket. After recording call analytics also sends the call metadata along with the file location to KDS. If you enable a data warehouse, the call metadata (including SIPREC metadata if SIPREC was used) is delivered to the data warehouse in a set of Parquet tables that you can query.
Like any other call analytics processor, you need to first create a configuration for the pipeline. You can use the Amazon Chime SDK Console or the CLI to create the configuration. You then use the CLI to create the pipeline. For more information on using the console to create recording configurations, refer to [Creating call analytics configurations for the Amazon Chime SDK](creating-ca-configuration.md), earlier in this section. For more information about using the recording workflows, refer to [Understanding workflows for recording calls for the Amazon Chime SDK](recording-workflows.md), earlier in this section.
**To use the CLI to create a configuration**
Run the following command:
```
aws chime-sdk-media-pipeline create-media-insights-pipeline-configuration --cli-input-json file://configuration.json
```
The following example shows a configuration JSON file with only recording enabled:
```
{
"MediaInsightsPipelineConfigurationName": configuration_name,
"ResourceAccessRoleArn": role_arn,
"Elements": [
{
"KinesisDataStreamSinkConfiguration": {
"InsightsTarget": KDS_arn //Where recording live metadata will be delivered.
},
"Type": "KinesisDataStreamSink"
},
{
"S3RecordingSinkConfiguration": {
"Destination": "arn:aws:s3:::kvs-recording-testing",
"RecordingFileFormat": file_format // Specify "Opus" or "WAV" as the recording file format.
},
"Type": "S3RecordingSink"
}
]
}
```
Remember the following:
+ To enable call recording through Kinesis Video Streams, audio should be PCM signed 16-bit little-endian. The sample rate must be 8KHz.
+ Builders must set a long enough data retention period for the Kinesis Video Stream to ensure the fragments are retained and consumable by call analytics.
+ If you enable call recording, either by itself or in combination with other processors, you must supply two Kinesis Video Stream ARNs for recording. Call recording does not support a single stereo audio input.
## Amazon S3 call recording metadata output
The following example shows metadata output format for call analytics Amazon S3 recording.
```
{
"time": "string", // ISO8601 format
"service-type": "CallAnalytics",
"detail-type": "Recording",
"mediaInsightsPipelineId": "string",
"s3MediaObjectConsoleUrl": "string",
"recordingDurationSeconds": "number",
"metadata": "string" // JSON encoded string of the metadata object
}
// metadata object
{
"voiceConnectorId": "string",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"direction": "string",
"startTime": "string", // ISO8601 format
"endTime": "string", // ISO8601 format
"oneTimeMetadata": "string" // JSON encoded in string of oneTimeMetadata object
}
// onetimeMetadata object
{
"sipHeaders": "string", // JSON encoded string of SIP Invite headers key-value pair
"siprecMetadata": "string", // siprec metadata in XML
"siprecMetadataJson": "string" // siprec metadata in JSON (converted from above XML)
}
// sipHeaders object
{
"string": "string"
}
```
## Enable voice enhancement
To enable voice enhancement, include a `VoiceEnhancementSinkConfiguration` element in a [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API call.
This example shows a typical element.
```
{
"Type":"VoiceEnhancementSink",
"VoiceEnhancementSinkConfiguration": {
"Disabled": Boolean (string) // FALSE ==> Voice Enhancement will be performed
}
```
To update a configuration, add the `VoiceEnhancementSinkConfiguration` element to a [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineConfiguration.html) API call. When you do, the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetMediaInsightsPipelineConfiguration.html) API includes the `VoiceEnhancementSinkConfiguration` element in the results.
This example request shows how to enable Voice Enhancement and Amazon S3 recording.
```
POST /media-insights-pipeline-configurations HTTP/1.1
Content-type: application/json
{
"MediaInsightsPipelineConfigurationName":"media_insights_configuration_name",
"ResourceAccessRoleArn":"arn:aws:iam::account_id:role/resource_access_role",
"Elements":[
{
"Type":"S3RecordingSink",
"S3RecordingSinkConfiguration":{
"Destination":"arn:aws:s3:::input_bucket_path",
"RecordingFileFormat":"Wav"
}
},
{
"Type":"VoiceEnhancementSink",
"VoiceEnhancementSinkConfiguration": {
"disabled":"false"
}
}
],
"ClientRequestToken":"client_request_token"
}
```
**Note**
The `VoiceEnhancementSink` element always requires an `S3RecordingSink` element in a call analytics configuration.
# Combining transcription with recording sinks for the Amazon Chime SDK
You can combine Amazon Transcribe and Amazon Transcribe Call Analytics processors with an Amazon S3 recording sink. Builders can pass an S3RecordingSinkConfiguration in addition to the Amazon Transcribe processors in a [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) API call, or by using the console.
In conjunction with the Amazon S3 recording sink, you can use an Amazon Transcribe or an Amazon Transcribe Call Analytics processor, but never both. You can also add voice analytics to the same configuration in addition to a recording sink, with or without a transcribe processor.
**Note**
You can enable recording with any of the processors listed above. However, if you enable Amazon Transcribe Call Analytics along with Amazon S3 call recording, you must provide two Kinesis video streams, and you'll receive duplicate recording files, one from Amazon Transcribe Call Analytics and one from the Amazon S3 call recording.
Remember the following:
+ You must use a unique `MediaInsightsPipelineConfigurationName`.
+ For information about the `ResourceAccessRoleArn`, refer to [Using the call analytics resource access role for the Amazon Chime SDK](call-analytics-resource-access-role.md) in this guide.
+ The `Destination` value must be an S3 path ARN. The Amazon S3 bucket must be owned by the same account.
+ If you use a configuration with Transcribe and recording to create a pipeline, pauses and resumes only appear in the insights generated by a Kinesis Data stream. All the data in the KVS streams is recorded and uploaded to Amazon S3.
+ If a configuration uses Amazon transcribe or transcribe call analytics (TCA) in addition to recording, the media insights pipeline provides transcription or Transcribe Call Analytics insights in real time, followed by Amazon S3 recording at the end of the call. If transcribe services fail during the call analytics, the S3 recording job still tries to run. Conversely, an Amazon S3 recording failure doesn't affect the transcribe insights, since it runs after transcribe completes.
This example shows a configuration with an Amazon Transcribe processor and an Amazon S3 recording sink. The example also enables partial results stabilization, which can reduce latency in output, but may impact accuracy. For more information, refer to [ Partial-result stabilization](https://docs.aws.amazon.com/transcribe/latest/dg/streaming-partial-results.html#streaming-partial-result-stabilization), in the *Amazon Transcribe Developer Guide*.
```
{
"MediaInsightsPipelineConfigurationName": unique_configuration_name,
"ResourceAccessRoleArn": role_arn,
"Elements": [{
"AmazonTranscribeProcessorConfiguration": {
"ContentIdentificationType": "string",
"ContentRedactionType": "string",
"EnablePartialResultsStabilization": boolean, //Enables partial result stabilization. Can reduce latency. May impact accuracy.
"FilterPartialResults": boolean, //To control partial utterance events
"LanguageCode": "string",
"LanguageModelName": "string",
"PartialResultsStability": "string",
"PiiEntityTypes": "string",
"ShowSpeakerLabel": boolean,
"VocabularyFilterMethod": "string",
"VocabularyFilterName": "string",
"VocabularyName": "string"
},
"Type": "AmazonTranscribeProcessor"
},
{
"KinesisDataStreamSinkConfiguration": {
"InsightsTarget": KDS_arn //Where recording and insights live metadata will be delivered.
},
"Type": "KinesisDataStreamSink"
},
{
"S3RecordingSinkConfiguration": {
"Destination": S3_Arn,
"RecordingFileFormat": file_format // Specify "Opus" or "WAV" as the recording file format.
},
"Type": "S3RecordingSink"
}
]
}
```
# Using Amazon EventBridge notifications for the Amazon Chime SDK
Amazon Chime SDK Call Analytics supports sending events to the default EventBridge bus when the state of the media insights pipeline changes, or when call analytics real-time alert conditions are met. For the media insights pipeline error status updates, we recommend that you configure an EventBridge target to notify you if your resources fail asynchronously. Call analytics notifications have a aws.chime source and various detail types, which are shared in the following sections. For more information, see the [Amazon EventBridge User Guide](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-what-is.html).
**Topics**
+ [Status updates](#status-updates)
+ [Real-time alerts](#realtime-alerts)
## Status updates
Media insights pipelines send EventBridge notifications as a call analytics session progresses and either ends successfully or encounters errors. You receive an EventBridge notification with the “Media Insights State Change" detail type when:
+ The status of a media insights pipeline changes.
+ The status of a media insights pipeline element changes.
+ Any pipeline element is stopped.
+ Any pipeline element fails.
The detail section always includes the following fields:
+ `version`
+ `mediaInsightsPipelineArn`
+ `eventType`
The detail section also includes a `mediaInsightsPipelineElementStatuses` field if the media insights pipeline contains multiple elements, such as analytics processors and data sinks. This field indicates the statuses of each element in the pipeline. The possible status for each pipeline element could be:
+ `NotStarted`
+ `InProgress`
+ `Stopped`
+ `Failed`
The detail section also includes any key-value pairs specified in `MediaInsightsRuntimeMetadata` during the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API call. If a call analytics session was initiated by a Voice Connector, the metadata section is populated with the following parameters automatically:
+ `transactionId`
+ `fromNumber`
+ `toNumber`
+ `callId`
+ `voiceConnectorId`
+ `direction`
The following event types may appear whenever a media insights pipeline contains a single element. Expand each section to learn more.
### Amazon Chime SDK media insights in progress
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsInProgress",
"version": "0",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string"
}
}
```
### Amazon Chime SDK media insights paused
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsPaused",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string"
}
}
```
### Amazon Chime SDK media insights stopped
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsStopped",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string"
}
}
```
### Amazon Chime SDK media insights temporary failure
Indicates that the service encountered a temporary failure and will attempt to retry. No action is required from you.
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsTemporaryFailure",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string"
}
}
```
### Amazon Chime SDK media insights permanent failure
Indicates a failure that requires action from you. Use the `failureReason` to troubleshoot the problem. Typical failures could include the following:
+ Insufficient permissions on the resource access role
+ Missing or deleted resources
+ Throttling from an AWS service that call analytics invokes on your behalf, such as Amazon Transcribe or Amazon Kinesis.
+ Incompatible media formats on KVS streams
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsPermanentFailure",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string",
"failureReason": "string"
}
}
```
**Note**
The `failureReason` field is optional. For example, a typical reason could be `Access denied when assuming resource access role`.
The following event types may appear whenever a media insights pipeline is created, or the creation attempt fails, for a call analytics session initiated by an Amazon Chime SDK Voice Connector. Expand each section to learn more.
### Amazon Chime SDK media insights created
This example shows a typical success event.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail":
{
"version": "0",
"mediaInsightsPipelineConfigurationArn": "string",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsCreated",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string",
}
}
```
### Amazon Chime SDK media insights create failed
This example shows a typical failure event.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail":
{
"version": "0",
"mediaInsightsPipelineConfigurationArn": "string",
"eventType": "chime:MediaInsightsCreateFailed",
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string",
"failureOrigin": "Voice Connector",
"httpStatusCode": "string",
"failureReason": "string"
}
}
```
The following event types may appear when a media insights pipeline contains multiple elements. The example notifications are for `AmazonTranscribeProcessor` combined with `S3RecordingSink`. Expand each section to learn more.
### AmazonTranscribeProcessor is in progress and S3RecordingSink has not started
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": [],
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsInProgress",
"mediaInsightsPipelineElementStatuses": [
{
"type": "AmazonTranscribeProcessor",
"status": "InProgress",
"updatedOn": 1686184070655
},
{
"type": "S3RecordingSink",
"status": "NotStarted",
"updatedOn": 1686184070655
}
]
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string"
}
}
```
### AmazonTranscribeProcessor has succeeded and S3RecordingSink is in progress
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": [],
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsInProgress",
"mediaInsightsPipelineElementStatuses": [
{
"type": "AmazonTranscribeProcessor",
"status": "Stopped",
"updatedOn": 1686184070655
},
{
"type": "S3RecordingSink",
"status": "InProgress",
"updatedOn": 1686184070655
}
]
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string"
}
}
```
### AmazonTranscribeProcessor has failed and S3RecordingSink is in progress
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": [],
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsInProgress",
"mediaInsightsPipelineElementStatuses": [
{
"type": "AmazonTranscribeProcessor",
"status": "Failed",
"updatedOn": 1686184070655
},
{
"type": "S3RecordingSink",
"status": "InProgress",
"updatedOn": 1686184070655
}
]
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string"
}
}
```
### AmazonTranscribeProcessor has failed and S3RecordingSink has succeeded
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": [],
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsPermanentFailure",
"mediaInsightsPipelineElementStatuses": [
{
"type": "AmazonTranscribeProcessor",
"status": "Failed",
"updatedOn": 1686184070655
},
{
"type": "S3RecordingSink",
"status": "Stopped",
"updatedOn": 1686184070655
}
]
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string",
"failureReason": "string"
}
}
```
### AmazonTranscribeProcessor has succeeded and S3RecordingSink has failed
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": [],
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsPermanentFailure",
"mediaInsightsPipelineElementStatuses": [
{
"type": "AmazonTranscribeProcessor",
"status": "Stopped",
"updatedOn": 1686184070655
},
{
"type": "S3RecordingSink",
"status": "Failed",
"updatedOn": 1686184070655
}
]
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string",
"failureReason": "string"
}
}
```
### AmazonTranscribeProcessor is paused and S3RecordingSink has not started
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": [],
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsPaused",
"mediaInsightsPipelineElementStatuses": [
{
"type": "AmazonTranscribeProcessor",
"status": "Paused",
"updatedOn": 1686184070655
},
{
"type": "S3RecordingSink",
"status": "NotStarted",
"updatedOn": 1686184070655
}
]
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string"
}
}
```
### AmazonTranscribeProcessor has temporarily failed and S3RecordingSink has not started
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": [],
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsTemporaryFailure",
"mediaInsightsPipelineElementStatuses": [
{
"type": "AmazonTranscribeProcessor",
"status": "TemporarilyFailed",
"updatedOn": 1686184070655
},
{
"type": "S3RecordingSink",
"status": "NotStarted",
"updatedOn": 1686184070655
}
]
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string"
}
}
```
### AmazonTranscribeProcessor and S3RecordingSink succeeded
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"region": "string",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": [],
"detail": {
"version": "0",
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsStopped",
"mediaInsightsPipelineElementStatuses": [
{
"type": "AmazonTranscribeProcessor",
"status": "Stopped",
"updatedOn": 1686184070655
},
{
"type": "S3RecordingSink",
"status": "Stopped",
"updatedOn": 1686184070655
}
]
"callId": "string",
"transactionId": "string",
"fromNumber": "string",
"toNumber": "string",
"voiceConnectorId": "string",
"direction": "string"
}
}
```
### S3RecordingSink succeeded and VoiceEnhancement in progress
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"time": "yyyy-mm-ddThh:mm:ssZ",
"region": "string",
"detail": {
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsInProgress",
"version": "0",
"mediaInsightsPipelineElementStatuses": [
{
"type": "VoiceEnhancementSink",
"status": "InProgress",
"updatedOn": 1686184070655
},
{
"type": "S3RecordingSink",
"status": "Stopped",
"updatedOn": 1686184070655
}
]
}
}
```
### S3RecordingSink succeeded and VoiceEnhancement failed due to calls longer than 30 minutes
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"time": "yyyy-mm-ddThh:mm:ssZ",
"region": "string",
"detail": {
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsStopped",
"version": "0",
"mediaInsightsPipelineElementStatuses": [
{
"type": "VoiceEnhancement",
"status": "NotSupported",
"updatedOn": 1686184070655,
"statusDetail": "Unsupported recording length"
},
{
"type": "S3RecordingSink",
"status": "Stopped",
"updatedOn": 1686184070655
}
]
}
}
```
### S3RecordingSink succeeded and VoiceEnhancement failed due to calls less than 30 minutes
This example shows a typical event structure.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights State Change",
"source": "aws.chime",
"account": number,
"time": "yyyy-mm-ddThh:mm:ssZ",
"region": "string",
"detail": {
"mediaInsightsPipelineArn": "string",
"eventType": "chime:MediaInsightsPermanentFailure",
"version": "0",
"mediaInsightsPipelineElementStatuses": [
{
"type": "VoiceEnhancement",
"status": "Failed",
"updatedOn": 1686184070655
},
{
"type": "S3RecordingSink",
"status": "Stopped",
"updatedOn": 1686184070655
}
]
}
}
```
## Real-time alerts
**Note**
Only Amazon Transcribe and Amazon Transcribe Call Analytics processors support real-time alerts.
Amazon Chime SDK call analytics allows developers to set up rules for sending real-time alerts through a processor during an analytics session. Alerts are sent to Amazon EventBridge with the detail type `Media Insights Rules Matched`. EventBridge supports integration with downstream services such as Lambda, Amazon SQS, and Amazon SNS to trigger notifications for the end user or initiate other custom business logic.
Real-time alerts are set up as a part of the `RealTimeAlertConfiguration` field for the `MediaInsightsPipelineConfiguration`. You can use the Amazon Chime SDK console to configure the field, or you can call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipelineConfiguration.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineConfiguration.html) APIs.
This example shows how to create or update a real-time alert configuration through the API.
```
{
"MediaInsightsPipelineConfigurationName": "config_name",
"ResourceAccessRoleArn": "arn:aws:iam::account_id:role/role_name",
"RealTimeAlertConfiguration": {
"Disabled": false,
"Rules": [{
"Type": "KeywordMatch",
"KeywordMatchConfiguration": {
"RuleName": "rule_name_1",
"Keywords": [
"hello",
"thank you"
],
"Negate": false
}
},
{
"Type": "Sentiment",
"RuleName": "rule_name_2",
"SentimentType": "NEGATIVE",
"TimePeriod": 60
},
{
"Type": "IssueDetection",
"RuleName": "rule_name_3"
}
]
},
"Elements": [{
"Type": "AmazonTranscribeCallAnalyticsProcessor",
"AmazonTranscribeCallAnalyticsProcessorConfiguration": {
"LanguageCode": "en-US"
}
},
{
"Type": "KinesisDataStreamSink",
"KinesisDataStreamSinkConfiguration": {
"InsightsTarget": "arn:aws:kinesis:us-east-1:account_id:stream/stream_name"
}
}
]
}
```
Each rule in a real-time alert configuration is triggered independently. You may receive multiple EventBridge notifications if multiple rule conditions are met at the same time. To create a list of rules for your alerts, you can select among the following rule types:
Keyword Match
Alerts when a specified set of keyword or phrases are matched in an utterance or transcript event. You can configure the alert to emit an event if:
+ Any specified keywords are spoken, and `Negate` is set to `false`.
+ All specified keywords are unspoken for the entirety of the call, if `Negate` is set to `true`.
Amazon Transcribe and Amazon Transcribe Analytics support this rule type.
Sentiment Analysis
Alerts when a particular sentiment type is ongoing for a rolling window period. Only Transcribe Call Analytics support this rule.
Issue Detection
Alerts when an issue is detected in an utterance event. Only Transcribe Call Analytics supports this rule type.
The following example shows a real-time alert event for a `KeywordMatch` rule.
```
{
"version": "0",
"id": "string",
"detail-type": "Media Insights Rules Matched",
"source": "aws.chime",
"account": number,
"region": "us-east-1",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": [],
"detail": {
"version": "0",
"sourceMetadata": {}
"ruleName": "string"
"utteranceId": "string",
"beginTimestamp": "yyyy-mm-ddThh:mm:ssZ",
}
}
```
Some EventBridge fields are specific to the rule type that is matched:
**Keyword match fields**
`utteranceId`: ID of the transcript that contains a matched keyword if you use Amazon Transcribe Call Analytics. For spoken keyword match only.
`resultId`: ID of the transcript that contains a matched keyword if you use Amazon Transcribe. For spoken keyword match only.
`beginTimestamp`: Start time of the transcript that contains a matched keyword. For spoken keyword match only.
**Sentiment analysis fields**
`beginTimestamp`: Start time of the rolling window for the matched sentiment.
`endTimestamp`: End time of the rolling window for the matched sentiment.
# Creating an Amazon Chime SDK data lake
The Amazon Chime SDK call analytics data lake allows you to stream your machine learning powered insights and any metadata from Amazon Kinesis Data Stream to your Amazon S3 bucket. For example, using the data lake to access URLs to recordings. To create the data lake, you deploy a set of AWS CloudFormation templates from either the Amazon Chime SDK console or programmatically using the AWS CLI. The data lake enables you to query your call metadata and voice analytics data by referencing AWS Glue data tables in Amazon Athena.
**Topics**
+ [Prerequisites](#data-lake-prereqs)
+ [Data lake terminology and concepts](#data-lake-terms)
+ [Creating multiple data lakes](#creating-multiple-data-lakes)
+ [Data lake regional availability](#data-lake-regions)
+ [Data lake architecture](#data-lake-architecture)
+ [Data lake setup](#data-lake-setup)
## Prerequisites
You must have the following items in order to create an Amazon Chime SDK lake:
+ An Amazon Kinesis data stream. For more information, refer to [Creating a Stream via the AWS Management Console](https://docs.aws.amazon.com/streams/latest/dev/how-do-i-create-a-stream.html) in the *Amazon Kinesis Streams Developer Guide*.
+ An S3 bucket. For more information, refer to [Create your first Amazon S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html) in the *Amazon S3 User Guide*.
## Data lake terminology and concepts
Use the following terms and concepts to understand how the data lake works.
**Amazon Kinesis Data Firehose**
An extract, transform, and load (ETL) service that reliably captures, transforms, and delivers streaming data to data lakes, data stores, and analytics services. For more information, see What Is Amazon Kinesis Data Firehose?
**Amazon Athena**
Amazon Athena is an interactive query service that enables you to analyze data in Amazon S3 using standard SQL. Athena is serverless, so you have no infrastructure to manage, and you pay only for the queries that you run. To use Athena, point to your data in Amazon S3, define the schema, and use standard SQL queries. You can also use workgroups to group users and control the resources they have access to when running queries. Workgroups enable you to manage query concurrency and prioritize query execution across different groups of users and workloads.
**Glue Data Catalog**
In Amazon Athena, tables and databases contain the metadata that details a schema for underlying source data. For each dataset, a table must exist in Athena. The metadata in the table tells Athena the location of your Amazon S3 bucket. It also specifies the data structure, such as column names, data types, and the table's name. Databases only hold the metadata and schema information for a dataset.
## Creating multiple data lakes
Multiple data lakes can be created by providing a unique Glue database name to specify where to store call insights. For a given AWS account, there can be several call analytics configurations, each with a corresponding data lake. This means that data separation can be applied for certain use cases, such as customizing retention policy, and access policy on how the data is stored. There can be different security policies applied for access of insights, recordings, and metadata.
## Data lake regional availability
The Amazon Chime SDK data lake is available in the following Regions.
| Region | Glue table | Quick |
| --- | --- | --- |
| us-east-1 | Available | Available |
| us-west-2 | Available | Available |
| eu-central-1 | Available | Available |
## Data lake architecture
The following diagram shows the data lake architecture. Numbers in the drawing correspond to the numbered text below.
![\[The program flow through a data lake.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/call-analytics-data-lake-architecture.png)
In the diagram, once you use the AWS console to deploy the CloudFormation template from the media insights pipeline configuration setup workflow, the following data flows to the Amazon S3 bucket:
1. The Amazon Chime SDK call analytics will start streaming real-time data to the customer's Kinesis Data Stream.
1. The Amazon Kinesis Firehose buffers this real-time data until it accumulates 128 MB, or 60 seconds elapses, whichever is first. Firehose then uses the `amazon_chime_sdk_call_analytics_firehose_schema` in the Glue Data Catalog to compress the data and transforms the JSON records to a parquet file.
1. The parquet file resides in your Amazon S3 bucket, in a partitioned format.
1. In addition to real-time data, post-call Amazon Transcribe Call Analytics summary .wav files (redacted and non-redacted, if specified in the configuration), and call recording .wav files are also sent to your Amazon S3 Bucket.
1. You can use Amazon Athena and standard SQL to query the data in the Amazon S3 bucket.
1. The CloudFormation template also creates a Glue Data Catalog to query this post-call summary data through Athena.
1. All the data on Amazon S3 bucket can also be visualized using Quick. QuickSight builds up a connection with an Amazon S3 bucket using Amazon Athena.
The Amazon Athena table uses the following features to optimize query performance:
**Data partitioning**
Partitioning divides your table into parts and keeps the related data together based on column values such as date, country, and region. Partitions act as virtual columns. In this case, the CloudFormation template defines partitions at table creation, which helps reduce the amount of data scanned per query and improves performance. You can also filter by partition to restrict the amount of data scanned by a query. For more information, refer to [Partitioning data in Athena](https://docs.aws.amazon.com/athena/latest/ug/partitions.html) in the *Amazon Athena User Guide*.
This example shows partitioning structure with a date of January 1, 2023:
1.
```
s3://example-bucket/amazon_chime_sdk_data_lake
/serviceType=CallAnalytics/detailType={DETAIL_TYPE}/year=2023
/month=01/day=01/example-file.parquet
```
1. where `DETAIL_TYPE` is one of the following:
1. `CallAnalyticsMetadata`
1. `TranscribeCallAnalytics`
1. `TranscribeCallAnalyticsCategoryEvents`
1. `Transcribe`
1. `Recording`
1. `VoiceAnalyticsStatus`
1. `SpeakerSearchStatus`
1. `VoiceToneAnalysisStatus`
**Optimize columnar data store generation**
Apache Parquet uses column-wise compression, compression based on data type, and predicate pushdown to store data. Better compression ratios or skipping blocks of data means reading fewer bytes from your Amazon S3 bucket. That leads to better query performance and reduced cost. For this optimization, data conversion from JSON to parquet is enabled in the Amazon Kinesis Data Firehose.
**Partition Projection**
This Athena feature automatically creates partitions for each day to improve date-based query performance.
## Data lake setup
Use the Amazon Chime SDK console to complete the following steps.
1. Start the Amazon Chime SDK console ([ https://console.aws.amazon.com/chime-sdk/home](https://console.aws.amazon.com/chime-sdk/home)) and in the navigation pane, under **Call Analytics**, choose **Configurations**.
1. Complete Step 1, choose **Next** and on the Step 2 page, choose the **Voice Analytics** check box.
1. Under **Output details**, select the **Data warehouse to perform historical analysis** checkbox, then choose the **Deploy CloudFormation stack** link.
The system sends you to the **Quick create stack** page in the CloudFormation console.
1. Enter a name for the stack, then enter the following parameters:
1. `DataLakeType` – Choose **Create Call Analytics DataLake**.
1. `KinesisDataStreamName` – Choose your stream. It should be the stream used for call analytics streaming.
1. `S3BucketURI` – Choose your Amazon S3 bucket. The URI must have the prefix `s3://bucket-name`
1. `GlueDatabaseName` – Choose a unique AWS Glue Database name. You cannot reuse an existing database in AWS account.
1. Choose the acknowledgment checkbox, then choose **Create data lake**. Allow 10 minutes for the system to create the lake.
### Data lake setup using AWS CLI
Use AWS CLI to create a role with permissions to call CloudFormation’s create stack. Follow the procedure below to create and setup the IAM roles. For more information, see [Creating a stack](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-cli-creating-stack.html) in the *AWS CloudFormation User Guide*.
1. Create a role called *AmazonChimeSdkCallAnalytics-Datalake-Provisioning-Role* and attach a trust policy to the role allowing CloudFormation to assume the role.
1. Create an IAM trust policy using the following template and save the file in .json format.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "cloudformation.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
```
------
1. Run the **aws iam create-role** command and pass the trust policy as a parameter.
```
aws iam create-role \
--role-name AmazonChimeSdkCallAnalytics-Datalake-Provisioning-Role
--assume-role-policy-document file://role-trust-policy.json
```
1. Note down the *role arn* that is returned from the response. *role arn* is required in the next step.
1. Create a policy with permission to create a CloudFormation stack.
1. Create an IAM policy using the following template and save the file in .json format. This file is required when calling create-policy.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "DeployCloudFormationStack",
"Effect": "Allow",
"Action": [
"cloudformation:CreateStack"
],
"Resource": "*"
}
]
}
```
------
1. Run **aws iam create-policy** and pass create stack policy as a parameter.
```
aws iam create-policy --policy-name testCreateStackPolicy
--policy-document file://create-cloudformation-stack-policy.json
```
1. Note down the *role arn* that is returned from the response. *role arn* is required in the next step.
1. Attach the policy to the role **aws iam attach-role-policy**.
```
aws iam attach-role-policy --role-name {Role name created above}
--policy-arn {Policy ARN created above}
```
1. Create a CloudFormation stack and enter the required parameters: **aws cloudformation create-stack**.
Provide parameter values for each ParameterKey using ParameterValue.
```
aws cloudformation create-stack --capabilities CAPABILITY_NAMED_IAM
--stack-name testDeploymentStack
--template-url https://chime-sdk-assets.s3.amazonaws.com/public_templates/AmazonChimeSDKDataLake.yaml
--parameters ParameterKey=S3BucketURI,ParameterValue={S3 URI}
ParameterKey=DataLakeType,ParameterValue="Create call analytics datalake"
ParameterKey=KinesisDataStreamName,ParameterValue={Name of Kinesis Data Stream}
--role-arn {Role ARN created above}
```
#### Resources created by data lake setup
The following table lists the resources created when you create a data lake.
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/ca-data-lake.html)
# Configuring an Quick dashboard for the Amazon Chime SDK
Once you set up the data lake, you can configure an Quick dashboard with pre-defined metrics that visualize your data. You can use the following dashboards:
+ **Transcribe Call Analytics \$1 Voice Analytics**. Metrics include summary and detailed visuals for turn-by-turn transcripts, issues detected, outcomes, entity detection, and voice profile ID matches.
+ **Transcribe \$1 Voice Analytics**. Metrics include summary and detailed visuals for turn-by-turn transcripts, vocabulary matches, voice tone, and voice profile ID matches.
The following topics explain how to set up an Quick account if you don't already have one, and how to configure a dashboard.
**Topics**
+ [Creating a QuickSight account](#create-quicksight-account)
+ [Configuring your QuickSight account](#configure-qs-acct)
+ [Creating a QuickSight dashboard](#create-qs-dashboard)
## Creating a QuickSight account
The steps in this section explain how to create an Quick account. If you already have an account, you can skip to [Creating a QuickSight dashboard](#create-qs-dashboard).
You can create a QuickSight account by:
+ Using Amazon CloudFormation templates.
+ Using the Amazon Chime SDK console.
### Prerequisites
Gather the following information before you start:
+ The name of your call analytics Amazon S3 bucket.
+ A notification email address. The system delivers QuickSight notifications to this address.
### Using CloudFormation templates to create an account
The following steps explain how to create an Quick account by deploying an Amazon CloudFormation template. The process only subscribes you to an Enterprise account. For information about pricing, refer to [Quick Pricing](https://aws.amazon.com/quicksight/pricing/).
**To deploy the template**
1. Start the AWS console and log in to your AWS account.
1. Paste the following URL into your browser's address bar. Make sure to enter your Region as indicated.
`https://region.console.aws.amazon.com/cloudformation/home?region=region#/stacks/quickcreate?templateURL=https://chime-sdk-assets.s3.amazonaws.com/public_templates/AmazonChimeSDKQuickSightSubscription.yaml`.
1. On the **Quick create stack** page, enter the following:
1. Under **Stack name** enter a name for your account.
1. Under **QuickSightNotificationEmail** the email address that you gathered earlier.
1. Under **QuickSightSubscriptionForDataVisualization**, choose **Create new AWS QuickSight account**.
1. Under **S3BucketName**, enter the name of your Amazon S3 bucket.
1. Select the **I acknowledge that AWS CloudFormation might create IAM resources.** checkbox.
1. Choose **Create stack**.
The system takes approximately 10 minutes to create the stack.
1. When the build finishes, choose **Go to Quick** and enter your email address to sign in to your account.
### Using the console to create an account
The following steps explain how to use the Amazon Chime SDK console to create an Quick account. You must use an Enterprise or Enterprise \$1 Q account.
**To use the console**
1. Start the Amazon Chime SDK console at [ https://console.aws.amazon.com/chime-sdk/home](https://console.aws.amazon.com/chime-sdk/home), search for **QuickSight**, and in the search results choose **QuickSight**.
![\[A search result that links to Quick.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/quicksightsetup-1.png)
1. Choose **Sign up for QuickSight**.
1. Choose **Enterprise** or **Enterprise \$1 Q**, then choose **Continue**.
1. Enter your first name, last name, phone number, and the email address that you gathered previously, then choose **Continue**.
1. Do the following:
1. Under **Authentication method**, choose an option.
**Note**
If you choose the option with federated users, you need the correct IAM permissions. For more information, refer to [ Signing up for an Quick subscription](https://docs.aws.amazon.com/quicksight/latest/user/signing-up.html) in the *Quick User Guide*.
1. Under **QuickSight Region**, select a Region.
1. Under **Account Info**, enter a name for the account and the email address that you gathered earlier.
1. Under **QuickSight access to AWS Services**, use the default role, or choose **Use an existing role** and select a role from the list.
1. (Optional) as needed, under **Allow access and autodiscovery for these resources**, choose additional resources.
1. When done, choose **Finish**.
1. When the build finishes, choose **Go to Quick** and enter your email address to sign in to your account.
## Configuring your QuickSight account
After you log in to your QuickSight account, you need to configure security and add yourself to a group created by the setup process.
**To configure security**
1. Choose the profile icon in the upper-right corner, then choose **Manage QuickSight** from the resulting menu.
![\[A menu with the Manage QuickSight command.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/quicksightsetup-1-a.png)
1. In the navigation pane, choose **Security & permissions**.
1. Under **QuickSight access to AWS services**, choose **Manage**, and ensure the following services are selected.
+ Amazon Redshift
+ Amazon RDS
+ Amazon S3
+ Amazon Athena
+ IAM
1. Choose the **Select Amazon S3 buckets** link.
1. Select the check box next to your Amazon S3 bucket, then select the checkbox to the right, in the **Write permission for Athena Workgroup** column.
1. Choose **Finish**.
1. Choose **Save**.
**To add yourself to the group**
1. In the navigation pane, choose **Manage groups**, then choose the group with **Admins** in the name. For example, *S3BucketName*–**Admins**.
1. Choose **Add user**, then enter your email alias in the box that appears.
Your name appears as **Admin – ***your-alias*.
1. Choose **Add**.
## Creating a QuickSight dashboard
After you create a data lake, you can create a QuickSight dashboard that visualizes your data. You can use an Amazon CloudFormation template or the Amazon Chime SDK console to create the dashboard. The following steps explain both methods.
**To use a template**
1. Start the Amazon CloudFormation console.
1. Paste the following link into your browser's address bar: `https://region.console.aws.amazon.com/cloudformation/home?region=region#/stacks/quickcreate?templateURL=https://chime-sdk-assets.s3.amazonaws.com/public_templates/AmazonChimeSDKQuickSightDashboards.yaml`
1. On the **Quick create stack** page, under **Stack name**, enter a name for the account.
1. Under **ActiveQuickSightAccount**, choose **True**.
1. Under **QuicksightDashboardSelection**, choose **Call Analytics – Transcribe Call Analytics and Voice Analytics dashboard** or **Call Analytics – Transcribe and Voice Analytics dashboard**.
1. Under **Amazon S3BucketName**, enter the URI of your Amazon S3 bucket.
1. Under **GlueDatabaseName**, enter the Glue database on which you want the QuickSight dashboard deployed.
1. Choose the **I acknowledge that AWS CloudFormation might create IAM resources** check box, then choose **Create stack**.
**To configure a QuickSight dashboard manually**
1. Navigate to your QuickSight account.
1. In the top-right corner choose the profile icon, then choose **Manage QuickSight**.
![\[The QuickSight account dialog box and Manage QuickSight command.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/quicksightsetup-1-a.png)
1. In the navigation pane, choose **Manage groups**, then choose the group created by the setup process.
1. Choose **Add User**, enter your email address, then choose **Add**.
The system takes 10 minutes to deploy the page.
1. Use the Amazon Chime SDK console to log in to your QuickSight account and use the dashboard.
# Call analytics data model for the Amazon Chime SDK
The information in this section lists and describes Amazon Chime SDK call analytics data model, a set of tables in an AWS Glue data catalog.
**Topics**
+ [Understanding the AWS Glue data catalog table structure for the Amazon Chime SDK](ca-data-model-diagram.md)
+ [Understanding the AWS Glue data catalog tables for the Amazon Chime SDK](glue-tables.md)
+ [Extracting data in your AWS Glue data catalog for Amazon Chime SDK call analytics](ca-data-model-queries.md)
# Understanding the AWS Glue data catalog table structure for the Amazon Chime SDK
The following diagram shows the table structure of the AWS Glue data catalog created for Amazon Chime SDK call analytics and voice analytics sessions.
![\[The tables in the call analytics Glue data catalog.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/glue-cat-diag-12-03.jpg)
The next section lists and describes the tables and fields in the catalog.
# Understanding the AWS Glue data catalog tables for the Amazon Chime SDK
The following tables list and describe the columns, data types, and elements in an Amazon Chime SDK call analytics Glue data catalog.
**Topics**
+ [call\$1analytics\$1metadata](#ca-glue-metadata)
+ [call\$1analytics\$1recording\$1metadata](#ca-glue-analytics-recording)
+ [transcribe\$1call\$1analytics](#ca-glue-transcribe-ca)
+ [transcribe\$1call\$1analytics\$1category\$1events](#ca-glue-transcribe-ca-events)
+ [transcribe\$1call\$1analytics\$1post\$1call](#ca-glue-transcribe)
+ [transcribe](#ca-glue-transcribe)
+ [voice\$1analytics\$1status](#ca-glue-va-status)
+ [speaker\$1search\$1status](#ca-glue-speaker-status)
+ [voice\$1tone\$1analysis\$1status](#ca-glue-tone-status)
## call\$1analytics\$1metadata
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/glue-tables.html)
## call\$1analytics\$1recording\$1metadata
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/glue-tables.html)
## transcribe\$1call\$1analytics
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/glue-tables.html)
## transcribe\$1call\$1analytics\$1category\$1events
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/glue-tables.html)
## transcribe\$1call\$1analytics\$1post\$1call
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/glue-tables.html)
## transcribe
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/glue-tables.html)
## voice\$1analytics\$1status
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/glue-tables.html)
## speaker\$1search\$1status
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/glue-tables.html)
## voice\$1tone\$1analysis\$1status
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/glue-tables.html)
# Extracting data in your AWS Glue data catalog for Amazon Chime SDK call analytics
Use these sample queries to extract and organize the data in your Amazon Chime SDK call analytics Glue data catalog.
**Note**
For information about connecting to Amazon Athena and querying your Glue data catalog, see [Connecting to Amazon Athena with ODBC](https://docs.aws.amazon.com/athena/latest/ug/connect-with-odbc.html).
Expand each section as needed.
## Extracting values from metadata (STRING datatype) in the call\$1analytics\$1metadata table
`call_analytics_metadata` has the `metadata` field in a JSON string format. Use the [json\$1extract\$1scalar function](https://docs.aws.amazon.com/athena/latest/ug/extracting-data-from-JSON.html) in Athena to query the elements in this string.
```
SELECT
json_extract_scalar(metadata,'$.voiceConnectorId') AS "VoiceConnector ID",
json_extract_scalar(metadata,'$.fromNumber') AS "From Number",
json_extract_scalar(metadata,'$.toNumber') AS "To Number",
json_extract_scalar(metadata,'$.callId') AS "Call ID",
json_extract_scalar(metadata,'$.direction') AS Direction,
json_extract_scalar(metadata,'$.transactionId') AS "Transaction ID"
FROM
"GlueDatabaseName"."call_analytics_metadata"
```
## Querying SIPRECMetadata updates in the call\$1analytics\$1metadata table
The `call_analytics_metadata` field has the metadata field in a JSON string format. `metadata` has another nested object called `oneTimeMetadata`, this object contains SIPRec Metadata in original XML and transformed JSON formats. Use the `json_extract_scalar` function in Athena to query the elements in this string.
```
SELECT
json_extract_scalar(metadata,'$.voiceConnectorId') AS "VoiceConnector ID",
json_extract_scalar(metadata,'$.fromNumber') AS "From Number",
json_extract_scalar(metadata,'$.toNumber') AS "To Number",
json_extract_scalar(metadata,'$.callId') AS "Call ID",
json_extract_scalar(metadata,'$.direction') AS Direction,
json_extract_scalar(metadata,'$.transactionId') AS "Transaction ID",
json_extract_scalar(json_extract_scalar(metadata,'$.oneTimeMetadata'),'$.siprecMetadata') AS "siprec Metadata XML",
json_extract_scalar(json_extract_scalar(metadata,'$.oneTimeMetadata'),'$.siprecMetadataJson') AS "Siprec Metadata JSON",
json_extract_scalar(json_extract_scalar(metadata,'$.oneTimeMetadata'),'$.inviteHeaders') AS "Invite Headers"
FROM
"GlueDatabaseName"."call_analytics_metadata"
WHERE
callevent-type = "update";
```
## Extracting values from metadata (STRING datatype) in the call\$1analytics\$1recording\$1metadata table
`call_analytics_recording_metadata` has the metadata field in a JSON string format. Use the [json\$1extract\$1scalar function](https://docs.aws.amazon.com/athena/latest/ug/extracting-data-from-JSON.html) in Athena to query the elements in this string.
```
SELECT
json_extract_scalar(metadata,'$.voiceConnectorId') AS "VoiceConnector ID",
json_extract_scalar(metadata,'$.fromNumber') AS "From Number",
json_extract_scalar(metadata,'$.toNumber') AS "To Number",
json_extract_scalar(metadata,'$.callId') AS "Call ID",
json_extract_scalar(metadata,'$.direction') AS Direction,
json_extract_scalar(metadata,'$.transactionId') AS "Transaction ID"
FROM
"GlueDatabaseName"."call_analytics_recording_metadata"
WHERE
detail-subtype = "Recording"
```
## Extracting values from detail (STRUCT datatype) in the voice\$1analytics\$1status table
`voice_analytics_status` has a details field in the `struct` data type. The following example shows how to query a `struct` data type field:
```
SELECT
detail.transactionId AS "Transaction ID",
detail.voiceConnectorId AS "VoiceConnector ID",
detail.siprecmetadata AS "Siprec Metadata",
detail.inviteheaders AS "Invite Headers",
detail.streamStartTime AS "Stream Start Time"
FROM
"GlueDatabaseName"."voice_analytics_status"
```
## Joining the voice\$1analytics\$1status and call\$1analytics\$1metadata tables
The following example query joins `call_analytics_metadata` and `voice_analytics_status`:
```
SELECT
a.detail.transactionId AS "Transaction ID",
a.detail.voiceConnectorId AS "VoiceConnector ID",
a.detail.siprecmetadata AS "Siprec Metadata",
a.detail.inviteheaders AS "Invite Headers",
a.detail.streamStartTime AS "Stream Start Time"
json_extract_scalar(b.metadata,'$.fromNumber') AS "From Number",
json_extract_scalar(b.metadata,'$.toNumber') AS "To Number",
json_extract_scalar(b.metadata,'$.callId') AS "Call ID",
json_extract_scalar(b.metadata,'$.direction') AS Direction
FROM
"GlueDatabaseName"."voice_analytics_status" a
INNER JOIN
"GlueDatabaseName"."call_analytics_metadata" b
ON a.detail.transactionId = json_extract_scalar(b.metadata,'$.transactionId')
```
## Extracting transcripts from the transcribe\$1call\$1analytics\$1post\$1call table
transcribe\$1call\$1analytics\$1post\$1call has transcript field in struct format with nested arrays. Use the following query to un-nest the arrays:
```
SELECT
jobstatus,
languagecode,
IF(CARDINALITY(m.transcript)=0 OR CARDINALITY(m.transcript) IS NULL, NULL, e.transcript.id) AS utteranceId,
IF(CARDINALITY(m.transcript)=0 OR CARDINALITY(m.transcript) IS NULL, NULL, e.transcript.content) AS transcript,
accountid,
channel,
sessionid,
contentmetadata.output AS "Redaction"
FROM
"GlueDatabaseName"."transcribe_call_analytics_post_call" m
CROSS JOIN UNNEST
(IF(CARDINALITY(m.transcript)=0, ARRAY[NULL], transcript)) AS e(transcript)
```
## Joining the transcribe\$1call\$1analytics\$1post\$1call and call\$1analytics\$1metadata tables
The following query joins transcribe\$1call\$1analytics\$1post\$1call and call\$1analytics\$1metadata:
```
WITH metadata AS(
SELECT
from_iso8601_timestamp(time) AS "Timestamp",
date_parse(date_format(from_iso8601_timestamp(time), '%m/%d/%Y %H:%i:%s') , '%m/%d/%Y %H:%i:%s') AS "DateTime",
date_parse(date_format(from_iso8601_timestamp(time) , '%m/%d/%Y') , '%m/%d/%Y') AS "Date",
date_format(from_iso8601_timestamp(time) , '%H:%i:%s') AS "Time",
mediainsightspipelineid,
json_extract_scalar(metadata,'$.toNumber') AS "To Number",
json_extract_scalar(metadata,'$.voiceConnectorId') AS "VoiceConnector ID",
json_extract_scalar(metadata,'$.fromNumber') AS "From Number",
json_extract_scalar(metadata,'$.callId') AS "Call ID",
json_extract_scalar(metadata,'$.direction') AS Direction,
json_extract_scalar(metadata,'$.transactionId') AS "Transaction ID",
REGEXP_REPLACE(REGEXP_EXTRACT(json_extract_scalar(metadata,'$.oneTimeMetadata.s3RecordingUrl'), '[^/]+(?=\.[^.]+$)'), '\.wav$', '') AS "SessionID"
FROM
"GlueDatabaseName"."call_analytics_metadata"
),
transcript_events AS(
SELECT
jobstatus,
languagecode,
IF(CARDINALITY(m.transcript)=0 OR CARDINALITY(m.transcript) IS NULL, NULL, e.transcript.id) AS utteranceId,
IF(CARDINALITY(m.transcript)=0 OR CARDINALITY(m.transcript) IS NULL, NULL, e.transcript.content) AS transcript,
accountid,
channel,
sessionid,
contentmetadata.output AS "Redaction"
FROM
"GlueDatabaseName"."transcribe_call_analytics_post_call" m
CROSS JOIN UNNEST
(IF(CARDINALITY(m.transcript)=0, ARRAY[NULL], transcript)) AS e(transcript)
)
SELECT
jobstatus,
languagecode,
a.utteranceId,
transcript,
accountid,
channel,
a.sessionid,
"Redaction"
"Timestamp",
"DateTime",
"Date",
"Time",
mediainsightspipelineid,
"To Number",
"VoiceConnector ID",
"From Number",
"Call ID",
Direction,
"Transaction ID"
FROM
"GlueDatabaseName"."transcribe_call_analytics_post_call" a
LEFT JOIN
metadata b
ON
a.sessionid = b.SessionID
```
## Querying media object URLs for Voice enhancement call recording
The following example query joins `Voice enhancement call recording` URL:
```
SELECT
json_extract_scalar(metadata,'$.voiceConnectorId') AS "VoiceConnector ID",
json_extract_scalar(metadata,'$.fromNumber') AS "From Number",
json_extract_scalar(metadata,'$.toNumber') AS "To Number",
json_extract_scalar(metadata,'$.callId') AS "Call ID",
json_extract_scalar(metadata,'$.direction') AS Direction,
json_extract_scalar(metadata,'$.transactionId') AS "Transaction ID",
s3MediaObjectConsoleUrl
FROM
{GlueDatabaseName}."call_analytics_recording_metadata"
WHERE
detail-subtype = "VoiceEnhancement"
```
# Using Amazon Chime SDK voice analytics
The Amazon Chime SDK voice analytics feature enables you to implement speaker search and voice tone analysis. You use speaker search to identify and enroll new callers, and to identify repeat callers and assign a confidence score to those identifications. You use voice tone analysis to predict a caller's sentiment as `negative`, `neutral`, or `positive`.
You run voice analytics as an optional component of an Amazon Chime SDK call analytics session.
Voice analytics works with media insights pipelines or Amazon Chime SDK Voice Connectors calls. We recommend using the [ Media Pipelines SDK](media-pipelines.md) and invoking tasks on a media insights pipeline for finer grained control over, and information about, the tasks.
You can use Voice Connectors to ensure backward compatibility, but we only update the media insights pipeline APIs with new features.
For more information about creating and using Voice Connectors, see [ Managing Amazon Chime SDK Voice Connectors ](https://docs.aws.amazon.com/chime-sdk/latest/ag/voice-connectors.html) in the *Amazon Chime SDK Administrator Guide*.
Voice analytics also provides:
+ Asynchronous task processing. Tasks run independently from each other.
+ Control over when you process insights.
You can initiate voice analytics by calling the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html) and [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartVoiceToneAnalysisTask.html) APIs.
The following topics explain how to use voice analytics.
**Topics**
+ [Understanding voice analytics architecture for the Amazon Chime SDK](va-architecture.md)
+ [Understanding speaker search workflow for the Amazon Chime SDK](va-data-flow.md)
+ [Sample voice tone analysis workflow for the Amazon Chime SDK](va-tone-flow.md)
+ [Polling for task results for the Amazon Chime SDK](va-task-result-poll.md)
+ [Understanding notifications for the Amazon Chime SDK](va-notification-targets.md)
+ [Understanding data storage, opt-out, and data-retention policies for the Amazon Chime SDK](va-opt-out.md)
+ [Using voice APIs to run voice analytics for the Amazon Chime SDK](va-in-voice-namespace.md)
# Understanding voice analytics architecture for the Amazon Chime SDK
The topics in this section provide an overview of the Amazon Chime SDK voice analytics architecture, including the data flows for each feature.
This diagram provides a high-level view of how data flows through voice analytics.
![\[A diagram showing the high-level data flow through voice analytics.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/va-architecture-kvs.png)
In the diagram:
1. Audio is streamed to a Kinesis Video Stream for a caller and agent. You can use a Kinesis Video Streams producer or Amazon Chime SDK Voice Connector streaming to do this. For more information, see [Understanding workflows for machine-learning based analytics for the Amazon Chime SDK](ml-based-analytics.md) in this guide, and [ Streaming Amazon Chime SDK Voice Connector media to Kinesis](https://docs.aws.amazon.com/chime-sdk/latest/ag/start-kinesis-vc.html) in the *Amazon Chime SDK Administrator Guide*.
1. An application or builder triggers speaker search, voice tone analysis, or both, for the audio stream after the caller consents.
1. During the call, voice analytics sends notifications to a target, either Amazon Simple Queue Service (SQS), Amazon Simple Notification Service (SNS), AWS Lambda, or Amazon Kinesis Data Streams.
In addition, voice analytics provides these tools for managing the data that it generates.
**Voice profiles**
The combination of a voice embedding, the embedding's unique ID, and its expiration date. Voice profiles expire after three years for security reasons, and because voices change over time. To avoid re-creating voice profiles, call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_UpdateVoiceProfile.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_UpdateVoiceProfile.html) API. For more information about expiration dates, see [Understanding data retention for Amazon Chime SDK voice analytics](va-data-retention.md).
To enroll a voice embedding, or to update an enrolled voice embedding, you must call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceProfile.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceProfile.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_UpdateVoiceProfile.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_UpdateVoiceProfile.html) APIs within 24 hours after the call ends.
**Voice profile domains**
A collection of voice profiles.
# Understanding speaker search workflow for the Amazon Chime SDK
In this section, we show you an example data and program flow for an Amazon Chime SDK speaker search analysis.
The speaker search function involves the creation of a voice embedding, which can be used compare the voice of a caller against previously stored voice data. The collection, use, storage, and retention of biometric identifiers and biometric information in the form of a digital voiceprint may require the caller's informed consent via a written release. Such consent is required under various state laws, including biometrics laws in Illinois, Texas, Washington and other state privacy laws. Before using the speaker search feature, you must provide all notices, and obtain all consents as required by applicable law, and under the [AWS service terms](https://aws.amazon.com/service-terms/) governing your use of the feature.
The following diagram shows an example data flow through a speaker search analysis task. The numbered descriptions below the diagram describe each step of the process. The diagram assumes you have already configured an Amazon Chime SDK Voice Connector with a call analytics configuration that has a `VoiceAnalyticsProcessor`. For more information, see [Recording Voice Connector calls](record-vc-calls.md).
![\[A diagram showing the data flow through a speaker search analysis.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/speaker-search-workflow-2.png)
1. You or a system administrator create a voice profile domain for storing voice embeddings and voice profiles. For more information about creating voice profile domains, see [Creating voice profile domains](https://docs.aws.amazon.com/chime-sdk/latest/ag/create-vp-domain.html), in the *Amazon Chime SDK Administrator Guide*. You can also use the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceProfileDomain.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceProfileDomain.html) API.
1. A caller dials in using a phone number assigned to an Amazon Chime SDK Voice Connector. Or, an agent uses a Voice Connector number to make an outbound call.
1. The Amazon Chime SDK Voice Connector service creates a transaction ID and associates it with the call.
1. Assuming your application subscribes to EventBridge events, your application calls the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API with the with the media insights pipeline configuration and Kinesis Video Stream ARNs for the Voice Connector call.
For more information about using EventBridge, refer to [Understanding workflows for machine-learning based analytics for the Amazon Chime SDK](ml-based-analytics.md).
1. Your application—such as an Interactive Voice Response system—or agent provides notice to the caller regarding call recording and the use of voice embeddings for voice analytics and seeks their consent to participate.
1. Once the caller provides consent, your application or agent can call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask.html) API through the [ Voice SDK](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_Operations_Amazon_Chime_SDK_Voice.html) if you have a Voice Connector and a transaction ID. Or, if you have a media insights pipeline ID instead of a transaction ID, you call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html) API in the [ Media pipelines SDK](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_Operations_Amazon_Chime_SDK_Media_Pipelines.html).
Once the caller provides consent, your application or agent calls the `StartSpeakerSearchTask` API. You must pass the Voice Connector ID, transaction ID, and voice profile domain ID to the API. A speaker search task ID is returned to identify the asynchronous task.
**Note**
Before invoking the `StartSpeakerSearchTask` API in either of the SDKs, you must provide any necessary notices, and obtain any necessary consents, as required by law and under the [AWS service terms](https://aws.amazon.com/service-terms/).
1. The system accumulates 10 seconds of the caller's voice. The caller must speak for at least that amount of time. The system doesn't capture or analyze silence.
1. The media insights pipeline compares the speech to the voice profiles in the domain and lists top 10 high confidence matches. If it doesn't find a match, the Voice Connector creates a voice profile.
1. The media insights pipeline service sends a notification event to the configured notification targets.
1. The caller continues speaking and provides an additional 10 seconds of non-silence speech.
1. The media insights pipeline generates an enrollment voice embedding that you can use to create a voice profile or update an existing voice profile.
1. The media insights pipeline sends a `VoiceprintGenerationSuccessful` notification to the configured notification targets.
1. Your application calls the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceProfile.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceProfile.html) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_UpdateVoiceProfile.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_UpdateVoiceProfile.html) APIs to create or update the profile.
1. Your application calls the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_GetSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_GetSpeakerSearchTask.html) API as needed to get the latest status of the speaker search task.
# Sample voice tone analysis workflow for the Amazon Chime SDK
**Important**
Voice tone analysis involves making predictions on a speaker’s sentiment based on linguistic and tonal information. You must not use sentiment analysis in any manner prohibited by law, including in relation to making decisions about an individual that would produce legal or similarly significant impacts on such individuals (e.g., related to employment, housing, credit worthiness, or financial offers, etc.).
Voice tone analysis analyzes the voices of the people on a call and predicts their sentiment, either `positive`, `negative`, or `neutral`.
The following diagram shows an example workflow for a voice tone analysis. Numbered items below the image describe each step of the process.
**Note**
The diagram assumes you have already configured an Amazon Chime SDK Voice Connector with a call analytics configuration that has a `VoiceAnalyticsProcessor`. For more information, see [Recording Voice Connector calls](record-vc-calls.md).
![\[A diagram showing the data flow through a voice tone analysis.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/voice-tone-workflow-v2.png)
In the diagram:
1. A caller dials in using a phone number assigned to an Amazon Chime SDK Voice Connector. Or, an agent uses a Voice Connector number to make an outbound call.
1. The Voice Connector service creates a transaction ID and associates it with the call.
1. Your application—such as an Interactive Voice Response system—or agent provides notice to the caller regarding call recording and the use of voice embeddings for voice analytics and seeks their consent to participate.
1. Assuming your application subscribes to EventBridge events, your application calls the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_CreateMediaInsightsPipeline.html) API with the with the media insights pipeline configuration and Kinesis Video Stream ARNs for the Voice Connector call.
For more information about using EventBridge, refer to [Understanding workflows for machine-learning based analytics for the Amazon Chime SDK](ml-based-analytics.md).
1. Once the caller provides consent, your application or agent can call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask.html) API through the [ Voice SDK](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_Operations_Amazon_Chime_SDK_Voice.html) if you have a Voice Connector and a transaction ID. Or, if you have a media insights pipeline ID instead of a transaction ID, you call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_StartSpeakerSearchTask.html) API in the [ Media pipelines SDK](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_Operations_Amazon_Chime_SDK_Media_Pipelines.html).
Once the caller provides consent, your application or agent calls the `StartSpeakerSearchTask` API. You must pass the Voice Connector ID, transaction ID, and voice profile domain ID to the API. A speaker search task ID is returned to identify the asynchronous task.
1. The user speaks throughout the call.
1. The agent speaks throughout the call.
1. Every 5 seconds, the media insights pipeline uses a machine learning model to analyze the last 30 seconds of speech and predict the caller's tone for that interval, and for the entire call from the time when `StartVoiceToneAnalysisTask` was first called.
1. The media insights pipeline sends a notification with that information to the configured notification targets. You can identify the notification based on its stream ARN and channel ID. For more information, refer to [Understanding notifications for the Amazon Chime SDK](va-notification-targets.md), later in this section.
1. Repeat steps 9 and 10 until the call ends.
1. At the end of the call, the media insights pipeline sends one final notification with the current average tone prediction for the last 30 seconds, plus the average tone of the entire call.
1. Your application calls the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_GetVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_GetVoiceToneAnalysisTask.html) API as needed to get the latest status of the voice tone analysis task.
**Note**
The `GetVoiceToneAnalysisTask` API doesn't stream the tone data.
**Note**
The [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_GetVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_GetVoiceToneAnalysisTask.html) API doesn't return voice tone data.
# Polling for task results for the Amazon Chime SDK
**Important**
By default, voice analytics makes results available for 7 days, then it deletes the data automatically. You must store your task data if you want to use it for a longer time, or to comply with data-retention laws. For more information, see [Understanding data retention for Amazon Chime SDK voice analytics](va-data-retention.md), later in this guide.
Voice analytics tries to ensure at least one delivery of each task result. However, network issues can increase latency. To work around potential problems, or if you prefer synchronous processes, you can use the following APIs in either the [Media pipelines SDK](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_Operations_Amazon_Chime_SDK_Media_Pipelines.html) or the [Voice SDK](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_Operations_Amazon_Chime_SDK_Voice.html):
+ [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetSpeakerSearchTask.html)
+ [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_GetVoiceToneAnalysisTask.html)
**Important**
The `GetVoiceToneAnalysisTask` API only returns a task's status. It doesn't return task results. To see results, use an Amazon SQS, Amazon SNS, or AWS Lambda notification target.
The `GetSpeakerSearchTask` API gets the latest results synchronously for a task ID, delayed messages, or messages that arrive out of order. However, we recommend using notification targets and asynchronous processing. Doing so consumes fewer computing resources.
# Understanding notifications for the Amazon Chime SDK
Voice analytics automatically sends events to a target when speaker search or voice tone analysis tasks start, while they run, and when they finish. You use notification targets to receive those events. We recommend using multiple notification targets if your workflow or application needs high availability.
Also, you must use an IAM role with the policies needed to access your notification targets. For more information, see [Using the call analytics resource access role for the Amazon Chime SDK](call-analytics-resource-access-role.md).
**Note**
For Amazon SQS and Amazon SNS, we do not support first-in-first-out queues. As a result, messages may arrive out of order. We recommend checking the timestamps to order messages as needed, and persisting messages in a data store such as Amazon DynamoDB. You can also use the Get APIs described in [Polling for task results for the Amazon Chime SDK](va-task-result-poll.md) to receive the latest results.
The following table lists the events and their corresponding detail types.
| Notification event | Detail type |
| --- | --- |
| Voice analytics metadata | `VoiceAnalyticsStatus` |
| Speaker search | `SpeakerSearchStatus` |
| Voice tone analysis | `VoiceToneAnalysisStatus` |
# Understanding IAM policies for notification targets for the Amazon Chime SDK
You must use policies in the IAM role in a Call Analytics configuration that allow access to your Amazon SQS, Amazon SNS, AWS Lambda, or Amazon KDS notification targets. For more information, see [Using the call analytics resource access role for the Amazon Chime SDK](call-analytics-resource-access-role.md) in this guide.
## Speaker search events
Speaker search events have the `SpeakerSearchStatus` detail type.
Amazon Chime SDK Voice Connectors send the following speaker search events:
+ Identification matches
+ Voice embedding generation
The events can have the following statuses:
+ `IdentificationSuccessful` – Successfully identified at least one matching voice profile ID with a high confidence score in the given voice profile domain.
+ `IdentificationFailure` – Failed to perform identification. Causes: the caller doesn't talk for at least 10 seconds, poor audio quality.
+ `IdentificationNoMatchesFound` – Unable to find a high confidence match in the given voice profile domain. The caller may be new, or their voice may have changed.
+ `VoiceprintGenerationSuccessful` – The system generated a voice embedding using 20 seconds of non-silent audio.
+ `VoiceprintGenerationFailure` – The system failed to generate a voice embedding. Causes: caller doesn't talk for at least 20 seconds, poor audio quality.
### Identification matches
After the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask) API is called for a given `transactionId`, the Voice Connector service returns an identification match notification after 10 seconds of non-silent speech. The service returns the top 10 matches, along with a voice profile ID and confidence score ranging from [0, 1]. The higher the confidence score, the more likely the speaker from the call matches the voice profile ID. If the machine learning model finds no matches, the notification's `detailStatus` field contains `IdentificationNoMatchesFound`.
The following example shows notification for a successful match.
```
{
"version": "0",
"id": "12345678-1234-1234-1234-111122223333",
"detail-type": "SpeakerSearchStatus",
"service-type": "VoiceAnalytics",
"source": "aws.chime",
"account": "111122223333",
"time": "yyyy-mm-ddThh:mm:ssZ",
"region": "us-east-1",
"resources": [],
"detail": {
"taskId": "uuid",
"detailStatus": "IdentificationSuccessful",
"speakerSearchDetails" : {
"results": [
{
"voiceProfileId": "vp-505e0992-82da-49eb-9d4a-4b34772b96b6",
"confidenceScore": "0.94567856",
},
{
"voiceProfileId": "vp-fba9cbfa-4b8d-4f10-9e41-9dfdd66545ab",
"confidenceScore": "0.82783350",
},
{
"voiceProfileId": "vp-746995fd-16dc-45b9-8965-89569d1cf787",
"confidenceScore": "0.77136436",
}
]
},
"mediaInsightsPipelineId": "87654321-33ca-4dc6-9cdf-abcde6612345",
"sourceArn": "arn:aws:chime:us-east-1:111122223333:media-pipeline/87654321-33ca-4dc6-9cdf-abcde6612345",
"streamArn": "arn:aws:kinesisvideo:us-east-1:111122223333:stream/my-stream/0123456789012",
"channelId": 0
}
}
```
### Voice embedding generation
After an additional 10 seconds of non-silent speech, the Voice Connector sends a voice embedding generation notification to the notification targets. You can enroll new voice embeddings in a voice profile, or update a print already in a voice profile.
The following example shows the notification for a successful match, meaning you can update the associated voice profile.
```
{
"version": "0",
"id": "12345678-1234-1234-1234-111122223333",
"detail-type": "SpeakerSearchStatus",
"service-type": "VoiceAnalytics",
"source": "aws.chime",
"account": "111122223333",
"time": "yyyy-mm-ddThh:mm:ssZ",
"region": "us-east-1",
"resources": [],
"detail": {
"taskId": "guid",
"detailStatus": "VoiceprintGenerationSuccess",
"mediaInsightsPipelineId": "87654321-33ca-4dc6-9cdf-abcde6612345",
"sourceArn": "arn:aws:chime:us-east-1:111122223333:media-pipeline/87654321-33ca-4dc6-9cdf-abcde6612345",
"streamArn": "arn:aws:kinesisvideo:us-east-1:111122223333:stream/my-stream/0123456789012",
"channelId": 0
}
}
```
## Voice tone analysis events
Voice tone analysis events have the `VoiceToneAnalysisStatus` detail type. The analyses can return these statuses:
+ `VoiceToneAnalysisSuccessful` – Successfully analyzed the caller and agent voices into probabilities of sentiment—positive, negative, or neutral.
+ `VoiceToneAnalysisFailure` – Failed to perform tone analysis. This can happen if the caller hangs without talking for 10 seconds, or if the audio quality becomes too poor.
+ `VoiceToneAnalysisCompleted` – Successfully analyzed the user and agent voices into probabilities of sentiment for the entire call. This is the final event, sent when the voice tone analysis finishes.
The following example shows a typical voice tone analysis event.
```
{
"detail-type": "VoiceToneAnalysisStatus",
"service-type": "VoiceAnalytics",
"source": "aws.chime",
"account": "216539279014",
"time": "2022-08-26T17:55:15.563441Z",
"region": "us-east-1",
"detail": {
"taskId": "uuid",
"detailStatus": "VoiceToneAnalysisSuccessful",
"voiceToneAnalysisDetails": {
"currentAverageVoiceTone": {
"startTime": "2022-08-26T17:55:15.563Z",
"endTime": "2022-08-26T17:55:45.720Z",
"voiceToneLabel": "neutral",
"voiceToneScore": {
"neutral": "0.83",
"positive": "0.13",
"negative": "0.04"
}
},
"overallAverageVoiceTone": {
"startTime": "2022-08-26T16:23:13.344Z",
"endTime": "2022-08-26T17:55:45.720Z",
"voiceToneLabel": "positive",
"voiceToneScore": {
"neutral": "0.25",
"positive": "0.65",
"negative": "0.1"
}
}
},
"startFragmentNumber": "01234567890123456789",
"mediaInsightsPipelineId": "87654321-33ca-4dc6-9cdf-abcde6612345",
"sourceArn": "arn:aws:chime:us-east-1:111122223333:media-pipeline/87654321-33ca-4dc6-9cdf-abcde6612345",
"streamArn": "arn:aws:kinesisvideo:us-east-1:111122223333:stream/my-stream/0123456789012",
"channelId": 0
},
"version": "0",
"id": "Id-f928dfe3-f44b-4965-8a17-612f9fb92d59"
}
```
## Post-call summary events
Post call summary events are sent 5 minutes after the call has ended. These summaries provide an overview of the speaker search tasks that occurred throughout the call.
The following example shows a post call summary with the best voice profile match, confirmed speaker identity, and a list of the voice profiles created or updated through the `CreateVoiceProfile` and `UpdateVoiceProfile` API calls made during the call.
```
{
"version": "0",
"id": "12345678-1234-1234-1234-111122223333",
"detail-type": "VoiceAnalyticsStatus",
"service-type": "VoiceAnalytics",
"source": "aws.chime",
"account": "111122223333",
"time": "yyyy-mm-ddThh:mm:ssZ",
"region": "us-east-1",
"resources": [],
"detail": {
"detailStatus": "PostCallVoiceAnalytics",
"callId": "22e8dee8-bbd7-4f94-927b-2d0ebaeddc1c",
"transactionId": "daaeb6bf-2fe2-4e51-984e-d0fbf2f09436",
"voiceConnectorId": "abcdef1ghij2klmno3pqr4",
"isCaller": true | false,
"speakerSearchResults": {
"bestMatchedVoiceProfileId": "vp-04c25ba1-a059-4fd3-8495-4ac91b55e2bf",
"customerValidatedCallerIdentity": "vp-04c25ba1-a059-4fd3-8495-4ac91b55e2bf",
"createVoiceProfileTransactions": [
{
"voiceProfileId": "vp-04c25ba1-a059-4fd3-8495-4ac91b55e2bf",
"requestTimestamp": "2022-12-14T18:38:38.796Z"
},
{
"voiceProfileId": "vp-04c25ba1-a059-4fd3-8495-4ac91b55e2bf",
"requestTimestamp": "2022-12-14T18:38:38.796Z",
}
],
"updateVoiceProfileTransactions": [
{
"voiceProfileId": "vp-04c25ba1-a059-4fd3-8495-4ac91b55e2bf",
"requestTimestamp": "2022-12-14T18:38:38.796Z",
},
{
"voiceProfileId": "vp-04c25ba1-a059-4fd3-8495-4ac91b55e2bf",
"requestTimestamp": "2022-12-14T18:38:38.796Z",
}
]
}
}
}
```
# Voice analytics example Lambda function for the Amazon Chime SDK
The Python code in the following example processes notifications received from a Voice Connector. You can add the code to an AWS Lambda function. You can also use it to trigger your Amazon SQS queue, Amazon SNS topic, or Amazon Kinesis Data Stream. You can then store the notifications in an `EventTable` for future processing. For the exact notification formats, see [Understanding notifications for the Amazon Chime SDK](va-notification-targets.md).
```
import base64
import boto3
import json
import logging
import time
from datetime import datetime
from enum import Enum
log = logging.getLogger()
log.setLevel(logging.INFO)
dynamo = boto3.client("dynamodb")
EVENT_TABLE_NAME = "EventTable"
class EventType(Enum):
"""
This example code uses a single Lambda processor to handle either
triggers from SQS, SNS, Lambda, or Kinesis. You can adapt it to fit your
desired infrastructure depending on what you prefer. To distinguish
where we get events from, we use an EventType enum as an
example to show the different ways of parsing the notifications.
"""
SQS = "SQS"
SNS = "SNS"
LAMBDA = "LAMBDA"
KINESIS = "KINESIS"
class AnalyticsType(Enum):
"""
Define the various analytics event types that this Lambda will
handle.
"""
SPEAKER_SEARCH = "SpeakerSearch"
VOICE_TONE_ANALYSIS = "VoiceToneAnalysis"
ANALYTICS_READY = "AnalyticsReady"
UNKNOWN = "UNKNOWN"
class DetailType(Enum):
"""
Define the various detail types that Voice Connector's voice
analytics feature can return.
"""
SPEAKER_SEARCH_TYPE = "SpeakerSearchStatus"
VOICE_TONE_ANALYSIS_TYPE = "VoiceToneAnalysisStatus"
ANALYTICS_READY = "VoiceAnalyticsStatus"
def handle(event, context):
"""
Example of how to handle incoming Voice Analytics notification messages
from Voice Connector.
"""
logging.info(f"Received event of type {type(event)} with payload {event}")
is_lambda = True
# Handle triggers from SQS, SNS, and KDS. Use the below code if you would like
# to use this Lambda as a trigger for an existing SQS queue, SNS topic or Kinesis
# stream.
if "Records" in event:
logging.info("Handling event from SQS or SNS since Records exists")
is_lambda = False
for record in event.get("Records", []):
_process_record(record)
# If you would prefer to have your Lambda invoked directly, use the
# below code to have the Voice Connector directly invoke your Lambda.
# In this scenario, there are no "Records" passed.
if is_lambda:
logging.info(f"Handling event from Lambda")
event_type = EventType.LAMBDA
_process_notification_event(event_type, event)
def _process_record(record):
# SQS and Kinesis use eventSource.
event_source = record.get("eventSource")
# SNS uses EventSource.
if not event_source:
event_source = record.get("EventSource")
# Assign the event type explicitly based on the event source value.
event_type = None
if event_source == "aws:sqs":
event = record["body"]
event_type = EventType.SQS
elif event_source == "aws:sns":
event = record["Sns"]["Message"]
event_type = EventType.SNS
elif event_source == "aws:kinesis":
raw_data = record["kinesis"]["data"]
raw_message = base64.b64decode(raw_data).decode('utf-8')
event = json.loads(raw_message)
event_type = EventType.KINESIS
else:
raise Exception(f"Event source {event_source} is not supported")
_process_notification_event(event_type, event)
def _process_notification_event(
event_type: EventType,
event: dict
):
"""
Extract the attributes from the Voice Analytics notification message
and store it as a DynamoDB item to process later.
"""
message_id = event.get("id")
analytics_type = _get_analytics_type(event.get("detail-type"))
pk = None
if analytics_type == AnalyticsType.ANALYTICS_READY.value or analytics_type == AnalyticsType.UNKNOWN.value:
transaction_id = event.get("detail").get("transactionId")
pk = f"transactionId#{transaction_id}#notificationType#{event_type.value}#analyticsType#{analytics_type}"
else:
task_id = event.get("detail").get("taskId")
pk = f"taskId#{task_id}#notificationType#{event_type.value}#analyticsType#{analytics_type}"
logging.info(f"Generated PK {pk}")
_create_request_record(pk, message_id, json.dumps(event))
def _create_request_record(pk: str, sk: str, body: str):
"""
Record this notification message into the Dynamo db table
"""
try:
# Use consistent ISO8601 date format.
# 2019-08-01T23:09:35.369156 -> 2019-08-01T23:09:35.369Z
time_now = (
datetime.utcnow().isoformat()[:-3] + "Z"
)
response = dynamo.put_item(
Item={
"PK": {"S": pk},
"SK": {"S": sk},
"body": {"S": body},
"createdOn": {"S": time_now},
},
TableName=EVENT_TABLE_NAME,
)
logging.info(f"Added record in table {EVENT_TABLE_NAME}, response : {response}")
except Exception as e:
logging.error(f"Error in adding record: {e}")
def _get_analytics_type(detail_type: str):
"""
Get analytics type based on message detail type value.
"""
if detail_type == DetailType.SPEAKER_SEARCH_TYPE.value:
return AnalyticsType.SPEAKER_SEARCH.value
elif detail_type == DetailType.VOICE_TONE_ANALYSIS_TYPE.value:
return AnalyticsType.VOICE_TONE_ANALYSIS.value
elif detail_type == DetailType.ANALYTICS_READY.value:
return AnalyticsType.ANALYTICS_READY.value
else:
return AnalyticsType.UNKNOWN.value
```
**Important**
You must receive consent before you call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask) or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartVoiceToneAnalysis.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartVoiceToneAnalysis.html) APIs. We recommend that you persist the events in a holding area, such as Amazon DynamoDB, until you receive consent.
# Understanding data storage, opt-out, and data-retention policies for the Amazon Chime SDK
The Amazon Chime SDK uses voice data to provide and improve the speaker search service. As part of that, we use enrollment audio, the recorded snippets used to create voice embeddings, to train our machine learning and artificial intelligence models. You can opt out of having your data used to train the models, and the topics in this section explain how.
**Topics**
+ [Understanding data storage for speaker search for the Amazon Chime SDK](speaker-search-data-storage.md)
+ [Handling opt outs for speaker search for the Amazon Chime SDK](va-handle-opt-outs.md)
+ [Understanding data retention for Amazon Chime SDK voice analytics](va-data-retention.md)
# Understanding data storage for speaker search for the Amazon Chime SDK
The Amazon Chime SDK stores the following data for speaker search:
+ The voice embeddings attached to the voice profiles that we use to provide the speaker search functionality.
+ Enrollment audio, the recorded snippets of speech used to create the voice embeddings for each voice profile. We use the enrollment audio recordings to:
+ Keep the speaker search models up to date, a critical part of providing the speaker search feature.
+ Train the machine learning model to develop and improve the service. The use of enrollment audio for training is optional, and you can opt out of this use by selecting an opt-out policy as described in the following section.
# Handling opt outs for speaker search for the Amazon Chime SDK
You can handle opt outs for end users and entire organizations. Opting out has the following effects:
+ After you opt out, voice analytics will not use any new enrollment audio for model training, and it will not use any enrollment audio collected and stored prior to your opting out.
+ After you opt out, voice analytics will store and use enrollment audio in order to provide the speaker search service.
**Warning**
The following opt-out actions are irreversible. You can't recover deleted data.
**Handling end user opt-outs**
When end users want to opt out of speaker search, call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_DeleteVoiceProfile.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_DeleteVoiceProfile.html) API. This action removes the voice profile, plus the voice embeddings and enrollment audio.
To delete a group of voice embeddings, call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_DeleteVoiceProfileDomain.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_DeleteVoiceProfileDomain.html) API to remove the domain. This action deletes *all* the voice profiles in a domain.
**Handling opt-out at the organizational level**
To handle opt outs for an entire organization, use an AWS Organizations opt-out policy. Use the `chimesdkvoiceanalytics` service name. For information about the policies, see [ AI services opt-out policies](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_ai-opt-out.html) in the *AWS Organizations User Guide*.
**Note**
To use an opt-out policy, your AWS accounts must be centrally managed by AWS Organizations. If you haven't already created an organization for your AWS accounts, see [Creating and managing an organization](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_org.html) in the *AWS Organizations User Guide*.
# Understanding data retention for Amazon Chime SDK voice analytics
By default, Amazon Chime SDK voice analytics deletes voice embeddings after 3 years. We do this because people’s voices change over time, and also for security. You can use the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_UpdateVoiceProfile.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_UpdateVoiceProfile.html) API to update expired voice embeddings.
Results from [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask.html) and [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartVoiceToneAnalysisTask.html) will also be available from their respective [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_GetSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_GetSpeakerSearchTask.html) and [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_GetVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_GetVoiceToneAnalysisTask.html) APIs for a maximum of 7 days.
Voice embeddings generated from a [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask.html) are available for persistence via the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceProfile.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_CreateVoiceProfile.html) and [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_UpdateVoiceProfile.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_UpdateVoiceProfile.html) APIs for 24 hours, after which they're deleted and not available.
To remove results, and to handle consent withdrawals from your customers, see the previous section.
# Using voice APIs to run voice analytics for the Amazon Chime SDK
For backwards compatibility, you can use Amazon Chime SDK Voice APIs to start and manage voice analytics. However, only the media insights pipeline APIs for voice analytics provide new features, so we strongly recommend using them instead.
The following sections explain the differences between the voice and media insights pipelines APIs.
## Stopping tasks
If you use a Voice Connector to start voice analytics tasks, and you then use the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineStatus.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_media-pipelines-chime_UpdateMediaInsightsPipelineStatus.html) API to pause the pipeline, the tasks continue running. To stop the tasks, you must call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StopSpeakerSearchTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StopSpeakerSearchTask.html) and [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StopVoiceToneAnalysisTask.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StopVoiceToneAnalysisTask.html) APIs.
## Understanding the notification differences
When you use voice APIs to run voice analytics, the notifications differ from those generated by media insights pipelines.
+ Voice analytics ready events are only available for tasks started using voice APIs.
+ You need to use the `voiceConnectorId`, `transactionId`, or `callId` fields in your notifications to associate a voice analytics task with a call. If you use media insights pipelines to run voice analytics, you use the `mediaInsightsPipelineId` and `streamArn` or `channelId` fields to associate a task with a call.
The following topics explain how to use notifications with voice APIs.
**Topics**
+ [Voice analytics ready events](#va-ready-events)
+ [Speaker search events](#va-speaker-search-events)
+ [Voice tone analysis events](#va-tone-status)
### Voice analytics ready events
Voice analytics ready events have the `VoiceAnalyticsStatus` detail type.
You use Amazon Chime SDK Voice Connectors to start analytics tasks. When your receive a voice analytics ready event, you can trigger a speaker search or voice tone analysis task for the call, identified by the following properties:
+ `voiceConnectorId`
+ `transactionId`
**Note**
This notification is provided only when you have a media insights pipeline configuration with voice analytics enabled and associated with a Voice Connector. This notification is NOT provided when customers call the `CreateMediaInsightsPipeline` API and launch a speaker search task or voice tone analysis task via the Media Pipelines SDK.
The SIP headers returned by a Voice Connector contain the `transactionId`. If you don't have access to the SIP headers, the `AnalyticsReady` notification event also contains the `voiceConnectorId` and `transactionId`. That allows you to programmatically receive the information and call the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask), or [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartVoiceToneAnalysis.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartVoiceToneAnalysis.html) APIs.
When voice analytics is ready for processing, the Voice Connector sends an event with `"detailStatus": "AnalyticsReady"` to the notification target as a JSON body. If you use Amazon SNS or Amazon SQS, that body appears in the “Records” field in the Amazon SNS or Amazon SQS payload.
The following example shows a typical JSON body.
```
{
"detail-type": "VoiceAnalyticsStatus",
"version": "0",
"id": "Id-f928dfe3-f44b-4965-8a17-612f9fb92d59",
"source": "aws.chime",
"account": "123456789012",
"time": "2022-08-26T17:55:15.563441Z",
"region": "us-east-1",
"resources": [],
"detail": {
"detailStatus": "AnalyticsReady",
"callDetails": {
"isCaller": false,
"transactionId": "daaeb6bf-2fe2-4e51-984e-d0fbf2f09436",
"voiceConnectorId": "fuiopl1fsv9caobmqf2vy7"
}
}
}
```
This notification allows you to trigger additional callbacks to your application, and to handle any legal requirements, such as notice and consent, prior to calling the voice analytics task APIs.
### Speaker search events
Speaker search events have the `SpeakerSearchStatus` detail type.
Amazon Chime SDK Voice Connectors send the following speaker search events:
+ Identification matches
+ Voice embedding generation
The events can have the following statuses:
+ `IdentificationSuccessful` – Successfully identified at least one matching voice profile ID with a high confidence score in the given voice profile domain.
+ `IdentificationFailure` – Failed to perform identification. Causes: the caller doesn't talk for at least 10 seconds, poor audio quality.
+ `IdentificationNoMatchesFound` – Unable to find a high confidence match in the given voice profile domain. The caller may be new, or their voice may have changed.
+ `VoiceprintGenerationSuccessful` – The system generated a voice embedding using 20 seconds of non-silent audio.
+ `VoiceprintGenerationFailure` – The system failed to generate a voice embedding. Causes: caller doesn't talk for at least 20 seconds, poor audio quality.
#### Identification matches
After the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_voice-chime_StartSpeakerSearchTask) API is called for a given `transactionId`, the Voice Connector service returns an identification match notification after 10 seconds of non-silent speech. The service returns the top 10 matches, along with a voice profile ID and confidence score ranging from [0, 1]. The higher the confidence score, the more likely the speaker from the call matches the voice profile ID. If the machine learning model finds no matches, the notification's `detailStatus` field contains `IdentificationNoMatchesFound`.
The following example shows notification for a successful match.
```
{
"version": "0",
"id": "12345678-1234-1234-1234-111122223333",
"detail-type": "SpeakerSearchStatus",
"service-type": "VoiceAnalytics",
"source": "aws.chime",
"account": "111122223333",
"time": "yyyy-mm-ddThh:mm:ssZ",
"region": "us-east-1",
"resources": [],
"detail": {
"taskId": "uuid",
"detailStatus": "IdentificationSuccessful",
"speakerSearchDetails" : {
"results": [
{
"voiceProfileId": "vp-505e0992-82da-49eb-9d4a-4b34772b96b6",
"confidenceScore": "0.94567856",
},
{
"voiceProfileId": "vp-fba9cbfa-4b8d-4f10-9e41-9dfdd66545ab",
"confidenceScore": "0.82783350",
},
{
"voiceProfileId": "vp-746995fd-16dc-45b9-8965-89569d1cf787",
"confidenceScore": "0.77136436",
}
]
},
"isCaller": false,
"voiceConnectorId": "abcdef1ghij2klmno3pqr4",
"transactionId": "daaeb6bf-2fe2-4e51-984e-d0fbf2f09436"
}
}
```
#### Voice embedding generation
After an additional 10 seconds of non-silent speech, the Voice Connector sends a voice embedding generation notification to the notification targets. You can enroll new voice embeddings in a voice profile, or update a print already in a voice profile.
The following example shows the notification for a successful match, meaning you can update the associated voice profile.
```
{
"version": "0",
"id": "12345678-1234-1234-1234-111122223333",
"detail-type": "SpeakerSearchStatus",
"service-type": "VoiceAnalytics",
"source": "aws.chime",
"account": "111122223333",
"time": "yyyy-mm-ddThh:mm:ssZ",
"region": "us-east-1",
"resources": [],
"detail": {
"taskId": "guid",
"detailStatus": "VoiceprintGenerationSuccess",
"isCaller": false,
"transactionId": "12345678-1234-1234",
"voiceConnectorId": "abcdef1ghij2klmno3pqr"
}
}
```
### Voice tone analysis events
Voice tone analysis events have the `VoiceToneAnalysisStatus` detail type. The analyses can return these statuses:
+ `VoiceToneAnalysisSuccessful` – Successfully analyzed the caller and agent voices into probabilities of sentiment—positive, negative, or neutral.
+ `VoiceToneAnalysisFailure` – Failed to perform tone analysis. This can happen if the caller hangs without talking for 10 seconds, or if the audio quality becomes too poor.
+ `VoiceToneAnalysisCompleted` – Successfully analyzed the user and agent voices into probabilities of sentiment for the entire call. This is the final event, sent when the voice tone analysis finishes.
The following example shows a typical voice tone analysis event.
```
{
"detail-type": "VoiceToneAnalysisStatus",
"service-type": "VoiceAnalytics",
"source": "aws.chime",
"account": "216539279014",
"time": "2022-08-26T17:55:15.563441Z",
"region": "us-east-1",
"detail": {
"taskId": "uuid",
"detailStatus": "VoiceToneAnalysisSuccessful",
"voiceToneAnalysisDetails": {
"currentAverageVoiceTone": {
"startTime": "2022-08-26T17:55:15.563Z",
"endTime": "2022-08-26T17:55:45.720Z",
"voiceToneLabel": "neutral",
"voiceToneScore": {
"neutral": "0.83",
"positive": "0.13",
"negative": "0.04"
}
},
"overallAverageVoiceTone": {
"startTime": "2022-08-26T16:23:13.344Z",
"endTime": "2022-08-26T17:55:45.720Z",
"voiceToneLabel": "positive",
"voiceToneScore": {
"neutral": "0.25",
"positive": "0.65",
"negative": "0.1"
}
}
},
"isCaller": true,
"transactionId": "daaeb6bf-2fe2-4e51-984e-d0fbf2f09436",
"voiceConnectorId": "fuiopl1fsv9caobmqf2vy7"
},
"version": "0",
"id": "Id-f928dfe3-f44b-4965-8a17-612f9fb92d59"
}
```
# Call analytics service quotas for the Amazon Chime SDK
The tables in this section list the service quotas for Amazon Chime SDK call analytics.
For more information about the call analytics Regions, refer to [Available AWS Regions for the Amazon Chime SDK](sdk-available-regions.md), earlier in this guide.
Amazon Chime SDK call analytics and voice analytics have the following service quotas.
| Resource | Default limit | Adjustable |
| --- | --- | --- |
| Media Insights Pipeline Configurations per region | 100 | Yes |
| Active Media Insights Pipelines per region | 20 | Yes |
| Voice profile domains per region | 3 | Yes |
| Voice profiles per voice profile domain | 20 | Yes |
| Active speaker search tasks per region | 25 | Yes |
| Active voice tone analysis tasks per region | 25 | Yes |
| Active Voice Connector calls with voice analytics per region | 25 | Yes |
| Active speaker search tasks per Voice Connector call per transaction ID | 1 | No |
| Active voice tone analysis task per Voice Connector call per transaction ID | 1 | No |
| Maximum concurrent API calls per voice profile domain | 1 | Yes |
| Maximum concurrent API calls per voice profile | 1 | Yes |
| Maximum concurrent API calls per speaker search task | 1 | Yes |
| Maximum concurrent API calls per voice tone analysis task | 1 | Yes |
For more information about API rates and quotas, refer to [Amazon Chime SDK endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/chime-sdk.html) in the *AWS General Reference*.
**Note**
If you exceed the quota for any Region, you receive a **Resource Limit Exceeded** exception. You can use the **Service Quotas** page in the AWS console to request an increase, or you can contact your [customer support representative](https://docs.aws.amazon.com/awssupport/latest/user/getting-started.html).
Several of the call analytics APIs create resources and API requests for other AWS services. Those additional count against your account's quotas. If you request a quota or transactions-per-second increase from call analytics, you must also request increases for those other AWS services. Otherwise, your requests may be throttled and fail.