# Using Amazon Chime SDK live transcription
You use Amazon Chime SDK live transcription to generate live, user-attributed transcripts of your meetings. Amazon Chime SDK live transcription integrates with the Amazon Transcribe and Amazon Transcribe Medical services to generate transcripts of Amazon Chime SDK meetings while they're in progress.
Amazon Chime SDK live transcription processes each user’s audio separately for improved accuracy in multi-speaker scenarios. The Amazon Chime SDK uses its active talker algorithm to select the top two active talkers, and then sends their audio to Amazon Transcribe, in separate channels, via a single stream. Meeting participants receive user-attributed transcriptions via Amazon Chime SDK data messages. You can use transcriptions in a variety of ways, such as displaying subtitles, creating meeting transcripts, or using the transcriptions for content analysis.
Live transcription uses one stream to Amazon Transcribe for the duration of the meeting transcription. Standard Amazon Transcribe and Amazon Transcribe Medical costs apply. For more information, refer to [Amazon Transcribe Pricing](https://aws.amazon.com/transcribe/pricing/). For questions about usage or billing, contact your AWS account manager.
**Important**
By default, Amazon Transcribe may use and store audio content processed by the service to develop and improve AWS AI/ML services as further described in section 50 of the [AWS Service Terms](https://aws.amazon.com/service-terms/). Using Amazon Transcribe may be subject to federal and state laws or regulations regarding the recording or interception of electronic communications. It is your and your end users’ responsibility to comply with all applicable laws regarding the recording, including properly notifying all participants in a recorded session or communication that the session or communication is being recorded, and obtaining all necessary consents. You can opt out from AWS using audio content to develop and improve AWS AI/ML services by configuring an AI services opt out policy using AWS Organizations.
**Topics**
+ [System architecture](#sys-architecture)
+ [Billing and usage](#billing-and-usage)
+ [Configuring your account for Amazon Chime SDK live transcription](configure-transcribe.md)
+ [Choosing Amazon Chime SDK live transcription options](transcription-options.md)
+ [Starting and stopping Amazon Chime SDK live transcription](initiate-transcription.md)
+ [Amazon Chime SDK live transcription parameters](#transcription-parameters)
+ [Understanding Amazon Chime SDK live transcription events](transcription-events.md)
+ [Understanding Amazon Chime SDK live transcription messages](process-msgs.md)
+ [Processing a received Amazon Chime SDK live transcript event](delivery-examples.md)
+ [Parsing Amazon Chime SDK transcripts](parse-transcripts.md)
## System architecture
The Amazon Chime SDK creates real-time meeting transcriptions, without audio leaving the AWS network, via a service-side integration with your Amazon Transcribe or Amazon Transcribe Medical account. For improved accuracy, users’ audio is processed separately, then mixed into the meeting. The Amazon Chime SDK uses its active talker algorithm to select the top two active talkers, and then sends their audio to Amazon Transcribe or Amazon Transcribe Medical in separate channels via a single stream. For reduced latency, user-attributed transcriptions are sent directly to every meeting participant via data messages. When using a media pipeline to capture meeting audio, the meeting’s transcription information is also captured.
![\[A diagram showing the data flow of meeting transcription.\]](http://docs.aws.amazon.com/chime-sdk/latest/dg/images/transcription-architecture.png)
## Billing and usage
Live transcription uses one stream to Amazon Transcribe or Amazon Transcribe Medical for the duration of the meeting transcription. Standard Amazon Transcribe and Amazon Transcribe Medical costs apply. For more information, see [Amazon Transcribe Pricing](https://aws.amazon.com/transcribe/pricing/).. For questions about usage or billing, contact your AWS account manager.
# Configuring your account for Amazon Chime SDK live transcription
Before you can use Amazon Chime SDK live transcription, you must grant Amazon Chime SDK permission to call Amazon Transcribe and Amazon Transcribe Medical in your AWS account. You do that by adding the Chime Transcription service-linked role to your account. For information about creating the service-linked role for live transcription, refer to [Using roles with live transcription](https://docs.aws.amazon.com/chime-sdk/latest/ag/using-service-linked-roles-transcription.html) in the *Amazon Chime SDK Administration Guide*. For more information about IAM service-linked roles, refer to [Service Linked Roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html) in the *IAM User Guide*.
# Choosing Amazon Chime SDK live transcription options
When you use Amazon Chime SDK live transcription, you use [Amazon Transcribe](https://aws.amazon.com/transcribe/) or [Amazon Transcribe Medical](https://aws.amazon.com/transcribe/medical/) in your AWS account. You have access to all the [streaming languages supported by Amazon Transcribe](https://docs.aws.amazon.com/transcribe/latest/dg/what-is-transcribe.html), plus features such as [custom vocabularies](https://docs.aws.amazon.com/transcribe/latest/dg/how-vocabulary.html) and [vocabulary filters](https://docs.aws.amazon.com/transcribe/latest/dg/filter-unwanted-words.html). When using Amazon Transcribe Medical, you can choose a medical specialty, conversation type, and optionally provide any custom vocabulary. Standard Amazon Transcribe and Amazon Transcribe Medical costs apply.
The process of choosing transcription options follows these steps.
## Step 1: Choosing a transcription service
You need to decide which transcription service to use, [Amazon Transcribe](https://aws.amazon.com/transcribe/) or [Amazon Transcribe Medical](https://aws.amazon.com/transcribe/medical/).
If your use case requires medical speech to text capabilities, you probably want to use Amazon Transcribe Medical. For all other use cases, you probably want to use Amazon Transcribe.
You specify which transcription service to use when you call the `StartMeetingTranscription` API:
+ To use Amazon Transcribe, specify a `TranscriptionConfiguration` with `EngineTranscribeSettings`.
+ To use Amazon Transcribe Medical, specify a `TranscriptionConfiguration` with `EngineTranscribeMedicalSettings`.
## Step 2: Choosing a transcription Region
You need to choose an AWS Region for the transcription service. For information about the available AWS Regions for Amazon Transcribe and Amazon Transcribe Medical, refer to the [AWS Regional Services](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/) table.
In general, the lowest latency between a meeting’s media Region and transcription Region provides the best user experience. For the lowest latency, use the same Region for media and transcription whenever possible. However, you may have other factors to consider in selecting a Region, such as regulatory requirements or the Regions where you have configured Amazon Transcribe or Amazon Transcribe Medical.
Amazon Transcribe and Amazon Transcribe Medical features, such as custom vocabularies or vocabulary filters, are Region specific. If you configure any of those features, you should do so identically in all of the AWS Regions in which you intend to use live transcription. Alternately, you can use the same Amazon Transcribe Region for all meetings.
You can specify the Region that the transcription service uses. You do that by adding the region name to the `Region` field of the transcription engine settings when you call the `StartMeetingTranscription` API. If you do not specify a Region, the Amazon Chime SDK attempts to use the transcription service in the meeting’s media region. To have the Amazon Chime SDK select the Region for the transcription service for you, specify `auto` in the `Region` field. When you do that, Amazon Chime selects the transcription service Region based the meeting’s media Region as described in the tables below. For more information about the `StartMeetingTranscription` API, refer to [Starting and stopping Amazon Chime SDK live transcription](initiate-transcription.md) in this guide.
**Note**
The transcription region selected by the Amazon Chime SDK is subject to change as AWS, Amazon Chime SDK, Amazon Transcribe and Amazon Transcribe Medical make more regions available.
**Automatic region selection for Amazon Transcribe**
| Amazon Chime SDK Media Region | Region code | Transcription Region |
| --- | --- | --- |
| US East (Ohio) | us-east-2 | us-east-2 |
| US East (N. Virginia) | us-east-1 | us-east-1 |
| US West (N. California) | us-west-1 | us-west-2 |
| US West (Oregon) | us-west-2 | us-west-2 |
| Africa (Cape Town)**\$1** | af-south-1 | eu-west-2 |
| Asia Pacific (Mumbai) | ap-south-1 | eu-west-2 |
| Asia Pacific (Seoul) | ap-northeast-2 | ap-northeast-2 |
| Asia Pacific (Singapore) | ap-southeast-1 | ap-northeast-1 |
| Asia Pacific (Sydney) | ap-southeast-2 | ap-southeast-2 |
| Asia Pacific (Tokyo) | ap-northeast-1 | ap-northeast-1 |
| Canada (Central) | ca-central-1 | ca-central-1 |
| Europe (Frankfurt) | eu-central-1 | eu-central-1 |
| Europe (Ireland) | eu-west-1 | eu-west-1 |
| Europe (London) | eu-west-2 | eu-west-2 |
| Europe (Milan)**\$1** | eu-south-1 | eu-central-1 |
| Europe (Paris) | eu-west-3 | eu-central-1 |
| Europe (Stockholm) | eu-north-1 | eu-central-1 |
| South America (São Paulo) | sa-east-1 | sa-east-1 |
| GovCloud (US-East) | us-gov-east-1 | us-gov-west-1 |
| GovCloud (US-West) | us-gov-west-1 | us-gov-west-1 |
**Automatic region selection for Amazon Transcribe Medical**
| Amazon Chime SDK Media Region | Region code | Transcription Region |
| --- | --- | --- |
| US East (Ohio) | us-east-2 | us-east-2 |
| US East (N. Virginia) | us-east-1 | us-east-1 |
| US West (N. California) | us-west-1 | us-west-2 |
| US West (Oregon) | us-west-2 | us-west-2 |
| Africa (Cape Town)**\$1** | af-south-1 | eu-west-1 |
| Asia Pacific (Mumbai) | ap-south-1 | eu-west-1 |
| Asia Pacific (Seoul) | ap-northeast-2 | us-west-2 |
| Asia Pacific (Singapore) | ap-southeast-1 | ap-southeast-2 |
| Asia Pacific (Sydney) | ap-southeast-2 | ap-southeast-2 |
| Asia Pacific (Tokyo) | ap-northeast-1 | us-west-2 |
| Canada (Central) | ca-central-1 | ca-central-1 |
| Europe (Frankfurt) | eu-central-1 | eu-west-1 |
| Europe (Ireland) | eu-west-1 | eu-west-1 |
| Europe (London) | eu-west-2 | us-east-1 |
| Europe (Milan)**\$1** | eu-south-1 | eu-west-1 |
| Europe (Paris) | eu-west-3 | eu-west-1 |
| Europe (Stockholm) | eu-north-1 | eu-west-1 |
| South America (São Paulo) | sa-east-1 | us-east-1 |
**Note**
To use live transcription in Regions marked with an asterisk (**\$1**), you must first enable the Region in your AWS account. For more information, refer to [Enabling a Region](https://docs.aws.amazon.com/general/latest/gr/rande-manage.html) in the AWS General Reference.
For more information about the regions and endpoints for each service, refer to:
+ [Amazon Chime SDK media Regions](https://docs.aws.amazon.com/chime-sdk/latest/dg/chime-sdk-meetings-regions.html)
+ [Amazon Transcribe endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/transcribe.html#transcribe_region)
+ [Amazon Transcribe Medical endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/transcribe-medical.html)
## Step 3: Review service quotas
Each Amazon Chime SDK meeting with live transcription requires exactly one HTTP/2 stream to Amazon Transcribe or Amazon Transcribe Medical. Both services have regional service quotas for the number of concurrent HTTP/2 streams, and for start-stream transactions per second. For more information about the quotas, refer to [Guidelines and quotas](https://docs.aws.amazon.com/transcribe/latest/dg/limits-guidelines.html) in the *Amazon Transcribe Developer Guide*. For information about quota increases, refer to Service Quotas in the AWS console.
# Starting and stopping Amazon Chime SDK live transcription
You use the Amazon Chime SDK [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_StartMeetingTranscription.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_StartMeetingTranscription.html) API to initiate meeting transcription by applying a `TranscriptionConfiguration` to the meeting. The Amazon Chime SDK controller forwards the configuration to the meeting asynchronously. The success or failure of initiating meeting transcription is signaled through a message via Amazon Simple Notification Service (Amazon SNS) and Amazon EventBridge.
**Starting transcription**
This example shows how to start live transcription with Amazon Transcribe.
```
POST /meetings/meetingId/transcription?operation=start HTTP/1.1
Content-type: application/json
{
"TranscriptionConfiguration": {
"EngineTranscribeSettings": {
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_ResponseSyntax](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_ResponseSyntax)": "en-US",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_ResponseSyntax](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_ResponseSyntax)": "tag",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax)": "profanity",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax)": "lingo",
"Region": "us-east-1"
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax)": true,
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax)": "high",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax)": "PII",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax)": "PII",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax)": "ALL",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestSyntax)": "language-model"
}
}
}
```
This example shows how to start live transcription with Amazon Transcribe Medical.
```
POST /meetings/meetingId/transcription?operation=start HTTP/1.1
Content-type: application/json
{
"TranscriptionConfiguration": {
"EngineTranscribeMedicalSettings": {
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html)": "en-US",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html)": "PRIMARYCARE",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html)": "CONVERSATION",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html)": "lingo",
"Region": "us-east-1",
"[https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html)": "PHI",
}
}
}
```
`StartMeetingTranscription` – Starts transcription for the meeting.
`meetingId` – The ID of the meeting, returned by the [CreateMeeting API](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_CreateMeeting.html#API_CreateMeeting_ResponseSyntax).
`TranscriptionConfiguration` – Encapsulates the parameters for live transcription. You must specify exactly one configuration, `EngineTranscribeSettings` or `EngineTranscribeMedicalSettings`.
`EngineTranscribeSettings` – Specifies the use of Amazon Transcribe and passes its settings through to [https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestParameters](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html#API_streaming_StartStreamTranscription_RequestParameters).
`LanguageCode` – Required.
`VocabularyFilterMethod` – Optional.
`VocabularyFilterName` – Optional.
`VocabularyName` – Optional.
`Region` – Optional.
`EnablePartialResultsStabilization` – Optional.
`PartialResultsStability` – Optional.
` ContentIdentificationType` – Optional.
`ContentRedactionType` – Optional.
`PiiEntityTypes ` – Optional.
`LanguageModelName` – Optional.
`EngineTranscribeMedicalSettings` – Specifies the use of Amazon Transcribe Medical and passes its settings through to [https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html#API_streaming_StartMedicalStreamTranscription_RequestParameters](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html#API_streaming_StartMedicalStreamTranscription_RequestParameters).
`LanguageCode` – Required.
`Speciality` – Required.
`Type` – Required.
`VocabularyName` – Optional.
`Region` – Optional.
` ContentIdentificationType` – Optional.
**Responses**
Amazon Transcribe and Amazon Transcribe Medical take the following responses:
+ `OK` (200) with empty body, if you successfully apply the `TranscriptionConfiguration` to the meeting.
**Error messages**
Amazon Transcribe and Amazon Transcribe Medical display the following error messages:
+ **BadRequestException (400):** The input parameters don't match the service's restrictions.
+ **ForbiddenException (403):** The client is permanently forbidden from making the request.
+ **NotFoundException (404):** The `meetingId` does not exist.
+ **ResourceLimitExceededException (400):** The request exceeds the resource limit. For example, too many meetings have live transcription enabled.
+ **ServiceFailureException (500):** The service encountered an unexpected error.
+ **ServiceUnavailableException (503):** The service is currently unavailable.
+ **ThrottledClientException (429):** The client exceeded its request rate limit.
+ **UnauthorizedClientException (401):** The client is not currently authorized to make the request.
Calling `StartMeetingTranscription` a second time updates the `TranscriptionConfiguration` applied to the meeting.
**Stopping transcription**
You use the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_meeting-chime_StopMeetingTranscription.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_meeting-chime_StopMeetingTranscription.html) API to remove the `TranscriptionConfiguration` for a given `meetingID` and end meeting transcription. Ending a meeting stops transcription automatically.
This example shows the request syntax that invokes `StopMeetingTranscription`.
```
POST/meetings/meetingId/transcription?operation=stop HTTP/1.1
```
**Responses**
Amazon Transcribe and Amazon Transcribe Medical take the following responses:
+ `OK` (200) with empty body, if you successfully remove the `TranscriptionConfiguration` from the meeting.
**Error messages**
Amazon Transcribe and Amazon Transcribe Medical display the following error messages:
+ **BadRequestException (400):** The input parameters don't match the service's restrictions.
+ **ForbiddenException (403):** The client is permanently forbidden from making the request.
+ **NotFoundException (404):** The `meetingId` does not exist.
+ **ServiceFailureException (500):** The service encountered an unexpected error.
+ **ServiceUnavailableException (503):** The service is currently unavailable.
+ **ThrottledClientException (429):** The client exceeded its request rate limit.
+ **UnauthorizedClientException (401):** The client is not currently authorized to make the request.
## Amazon Chime SDK live transcription parameters
The Amazon Transcribe and Amazon Transcribe Medical APIs offer a number of parameters when initiating streaming transcription, such as [https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartStreamTranscription.html) and [https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/dg/API_streaming_StartMedicalStreamTranscription.html). You can use t hose parameters in the `StartMeetingTranscription` API unless the Amazon Chime SDK predetermines the parameter’s value. For example, the `MediaEncoding` and `MediaSampleRateHertz` parameters are not available because the Amazon Chime SDK sets them automatically.
Amazon Transcribe and Amazon Transcribe Medical validate the parameters, and that allows you to use new parameter values as soon as they become available. For example, if Amazon Transcribe Medical launches support for a new language, you only need to specify the new language value in the `LanguageCode` parameter.
# Understanding Amazon Chime SDK live transcription events
The Amazon Chime SDK sends transcription lifecycle events, which you can use to trigger notifications and initiate downstream work flows. Some examples of using transcription events include:
+ Measuring the adoption of live transcription in Amazon Chime SDK meetings
+ Tracking language preferences
You can send events to Amazon EventBridge, Amazon Simple Notification Service, and Amazon Simple Queue Service. For more information, refer to [Events from AWS services](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-service-event.html) in the *Amazon EventBridge User Guide*.
## Amazon Chime SDK meeting transcription started
The Amazon Chime SDK sends this event when meeting transcription is started or the [https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_TranscriptionConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_TranscriptionConfiguration.html) is updated.
**Example: Event data**
The following is example data for this event.
```
{
"version": "0",
"source": "aws.chime",
"account": "111122223333",
"id": "12345678-1234-1234-1234-111122223333",
"region": "us-east-1",
"detail-type": "Chime Meeting State Change",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail": {
"version": "0",
"eventType": "chime:TranscriptionStarted",
"timestamp": 12344566754,
"meetingId": "87654321-4321-4321-1234-111122223333",
"externalMeetingId": "mymeeting",
"mediaRegion": "us-west-1",
"transcriptionRegion": "us-west-2",
"[https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_meeting-chime_StartMeetingTranscription.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_meeting-chime_StartMeetingTranscription.html)": "{...}"
}
}
```
## Amazon Chime SDK meeting transcription stopped
The Amazon Chime SDK sends this event when meeting transcription is stopped.
**Example: Event data**
The following is example data for this event.
```
{
"version": "0",
"source": "aws.chime",
"account": "111122223333",
"id": "12345678-1234-1234-1234-111122223333",
"region": "us-east-1",
"detail-type": "Chime Meeting State Change",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail": {
"version": "0",
"eventType": "chime:TranscriptionStopped",
"timestamp": 12344566754,
"meetingId": "87654321-4321-4321-1234-111122223333",
"externalMeetingId": "mymeeting",
"mediaRegion": "us-west-1",
"transcriptionRegion": "us-west-2",
"[https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_meeting-chime_StopMeetingTranscription.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_meeting-chime_StopMeetingTranscription.html)": "{...}"
}
}
```
## Amazon Chime SDK meeting transcription interrupted
The Amazon Chime SDK sends this event if meeting transcription is interrupted.
**Example: Event data**
The following is example data for this event.
```
{
"version": "0",
"source": "aws.chime",
"account": "111122223333",
"id": "12345678-1234-1234-1234-111122223333",
"region": "us-east-1",
"detail-type": "Chime Meeting State Change",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail": {
"version": "0",
"eventType": "chime:TranscriptionInterrupted",
"timestamp": 12344566754,
"meetingId": "87654321-4321-4321-1234-111122223333",
"externalMeetingId": "mymeeting",
"message": "Internal server error",
"mediaRegion": "us-west-1",
"transcriptionRegion": "us-west-2",
"[https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_TranscriptionConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_TranscriptionConfiguration.html)": "{...}"
}
}
```
## Amazon Chime SDK meeting transcription resumed
The Amazon Chime SDK sends this event if meeting transcription is resumed after an interruption.
**Example: Event data**
The following is example data for this event.
```
{
"version": "0",
"source": "aws.chime",
"account": "111122223333",
"id": "12345678-1234-1234-1234-111122223333",
"region": "us-east-1",
"detail-type": "Chime Meeting State Change",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail": {
"version": "0",
"eventType": "chime:TranscriptionResumed",
"timestamp": 12344566754,
"meetingId": "87654321-4321-4321-1234-111122223333",
"externalMeetingId": "mymeeting",
"mediaRegion": "us-west-1",
"transcriptionRegion": "us-west-2",
"[https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_TranscriptionConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_TranscriptionConfiguration.html)": "{...}"
}
}
```
## Amazon Chime SDK meeting transcription failed
The Amazon Chime SDK sends this event if meeting transcription failed to start, or failed to resume after an interruption.
**Example: Event data**
The following is example data for this event.
```
{
"version": "0",
"source": "aws.chime",
"account": "111122223333",
"id": "12345678-1234-1234-1234-111122223333",
"region": "us-east-1",
"detail-type": "Chime Meeting State Change",
"time": "yyyy-mm-ddThh:mm:ssZ",
"resources": []
"detail": {
"version": "0",
"eventType": "chime:TranscriptionFailed",
"timestamp": 12344566754,
"meetingId": "87654321-4321-4321-1234-111122223333",
"externalMeetingId": "mymeeting",
"message": "Internal server error",
"mediaRegion": "us-west-1",
"transcriptionRegion": "us-west-2",
"[https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_TranscriptionConfiguration.html](https://docs.aws.amazon.com/chime-sdk/latest/APIReference/API_TranscriptionConfiguration.html)": "{...}"
}
}
```
# Understanding Amazon Chime SDK live transcription messages
The Amazon Chime SDK service shares transcription information with attendees by sending `TranscriptEvent` objects in data messages. A `TranscriptEvent` delivers a `Transcript` or a `TranscriptionStatus`.
A `Transcript` includes results with timestamped, user-attributed words and punctuation. A result may be “partial”, in which case the system usually updates it in a subsequent `TranscriptEvent`. This allows you to see transcriptions quickly and apply inline updates later as necessary.
A `TranscriptStatus` may deliver one of the `TranscriptionStatusType` events, listed in the example in the next section.
Newer versions of the Amazon Chime SDKs include additional data types and helper functions for common processing a `TranscriptEvent`.
## TranscriptEvent
This example shows a typical transcription event.
```
type TranscriptEvent = Transcript | TranscriptionStatus;
export class TranscriptEventConverter {
static from(dataMessage: DataMessage): TranscriptEvent[] {
// convert DataMessage to TranscriptEvents
return ...
}
}
export default class TranscriptionStatus {
type: TranscriptionStatusType;
eventTimeMs: number;
transcriptionRegion: string;
transcriptionConfiguration: string;
message?: string;
}
enum TranscriptionStatusType {
STARTED = 'started',
INTERRUPTED = 'interrupted',
RESUMED = 'resumed',
STOPPED = 'stopped',
FAILED = 'failed',
}
export default class Transcript {
results: TranscriptResult[]; // at least one
}
export class TranscriptResult {
resultId: string;
isPartial: boolean;
startTimeMs: number;
endTimeMs: number;
alternatives: TranscriptAlternative[]; // most confident first
}
export default class TranscriptAlternative {
items: TranscriptItem[]; // in start time order
transcript: string; //concatenated transcript items
entities?: TranscriptEntity[];
}
export default class TranscriptItem {
type: TranscriptItemType;
startTimeMs: number;
endTimeMs: number;
attendee: Attendee;
content: string;
vocabularyFilterMatch?: boolean;
confidence?: number;
stable?: boolean;
}
enum TranscriptItemType {
PRONUNCIATION = 'pronunciation',// content is a word
PUNCTUATION = 'punctuation',// content is punctuation
}
export default class TranscriptEntity {
category: string;
confidence: number;
content: string;
endTimeMs: number;
startTimeMs: number;
type?: string;
}
// This is an existing SDK model
export default class Attendee {
attendeeId: string;
externalUserId: string;
}
```
## Data guidelines
Keep these guidelines in mind as you go.
1. `transcription.results` may have more than one result.
1. If `transcription.results[i].isPartial = true`, then there may be an update for the entire result. The update is likely, but not guaranteed. The update has the same `transcript.result[i].resultId`. If you want to avoid low-confidence transcriptions, you can skip partial results completely. If you want low-latency results, you can display partial results, then overwrite completely when the update arrives.
1. `transcription.results[i].alternatives` always contains at least one entry. If it contains more than one entry, the most confident entry is first in the list. In most cases, you can take the first entry in `transcription.results[i].alternatives` and ignore the others.
1. `transcription.results[i].alternatives[j].items` includes an entry for each word or punctuation mark.
1. `transcription.results[i].alternatives[j].items[k].` content is what was spoken.
1. `transcription.results[i].alternatives[j].items[k].attendee` is the user attribution (who) of the content.
1. `transcription.results[i].alternatives[j].items[k].startTimeMs` is the "when" of the content. This enables word-by-word rendering of user-attributed transcription across different users in the order that the words were spoken.
1. The `transcription.results[i].alternatives[j].items[k].endTimeMs` field can generally be ignored, but is supplied for completeness of who said what when.
1. `transcription.results[i].alternatives[j].items[k].vocabularyFilterMatch` is true if the content matched a word in the filter, otherwise it is false.
1. `transcription.results[i].alternatives[j].items[k].confidence` is a value between 0 and 1. It indicates the engine's confidence that the item content correctly matches the spoken word, with 0 being the lowest confidence and 1 being the highest confidence.
1. `transcription.results[i].alternatives[j].items[k].stable` indicates whether the current word will change in future partial result updates. This value can only be true if you enable the partial results stabilization feature by setting `EnablePartialResultsStabilization` to `true` in your request.
1. `transcription.results[i].alternatives[j].entities` includes an entry for each entity that the Content Identification or Redaction features detect. The list is only populated if you enable Content Identification or Redaction. An entity can be data such as personally identifiable information or personal health information. You can use entities to highlight, or take action on, words of interest during transcription.
1. `transcription.results[i].alternatives[j].entities[k].category` is the entity's category. It equals the Content Identification or Redaction type, such as "PII" or "PHI", which is provided in the request.
1. `transcription.results[i].alternatives[j].entities[k].confidence` measures how strong the engine is that the particular content is truly an entity. Note that this is different than the item-level confidence, which measures how confident the engine is in the correctness of the words themselves.
1. `transcription.results[i].alternatives[j].entities[k].content` is the actual text that makes up the entity. This can be multiple items, such as an address.
1. `transcription.results[i].alternatives[j].entities[k].startTimeMs` captures the time at which the entity started being spoken.
1. `transcription.results[i].alternatives[j].entities[k].endTimeMs` captures the time at which the entity finished being spoken.
1. `transcription.results[i].alternatives[j].entities[k].type` is only supported for the Transcribe engine and provides the sub-type of the entity. These are values such as `ADDRESS`, `CREDIT\$1DEBIT\$1NUMBER`, and so on.
## Registering event handlers for TranscriptEvents
The following examples use the Amazon Chime SDK client library for JavaScript. However, the pattern is consistent across all Amazon Chime SDKs.
The `TranscriptionController` in the `RealtimeController` and `RealtimeControllerFacade` includes specific functions for adding a handler that processes `TranscriptionEvents`:
```
/**
* Returns the [[TranscriptionController]] for this real-time controller.
*/
readonly transcriptionController?: TranscriptionController;
```
The `TranscriptionController` has two functions to manage subscribing and unsubscribing to `TranscriptionEvent` callbacks:
```
import TranscriptEvent from './TranscriptEvent';
export default interface TranscriptionController {
/**
* Subscribe a callback to handle received transcript event
*/
subscribeToTranscriptEvent(callback: (transcriptEvent: TranscriptEvent) => void): void;
/**
* Unsubscribe a callback from receiving transcript event
*/
unsubscribeFromTranscriptEvent(callback: (transcriptEvent: TranscriptEvent) => void): void;
}
```
**Using the optional `TranscriptionController`**
We provide a default implementation of `TranscriptionController` interface named `DefaultTranscriptionController`. The default implementation in `DefaultRealtimeController` and `DefaultAudioVideoFacade` returns a `DefaultTranscriptionController` object:
```
/**
get transcriptionController(): TranscriptionController {
return this.realtimeController.transcriptionController;
}
```
`DefaultRealtimeController` also takes an optional `TranscriptionController` object in its constructor. That allows you to override the `DefaultTranscriptionController` behavior. Developer applications subscribe and unsubscribe to one or more callbacks through the `TranscriptionController` object of the `AudioVideoFacade` object:
```
// Subscribe
this.audioVideo.transcriptionController?.subscribeToTranscriptEvent(this.transcriptEventHandler);
// Unsubscribe
this.audioVideo.transcriptionController?.unsubscribeFromTranscriptEvent(this.transcriptEventHandler););
```
# Processing a received Amazon Chime SDK live transcript event
The following examples show how to process a received `TranscriptEvent`.
**Note**
The exact output depends on several factors, including how quickly individuals talk and when they pause.
## Example 1: StartMeetingTranscription
This example shows a typical `StartMeetingTranscription` operation.
```
meeting.StartMeetingTranscription(
{ EngineTranscribeSettings: { Languagecode: ‘en-US’ } } );
```
The operation generates a `TranscriptEvent`.
```
{
status: {
type: 'started',
eventTimeMs: 1620118800000,
transcriptionConfig: {
LanguageCode: 'en-US'
}
}
}
```
## Example 2: A partial transcript result
In this example, an attendee says, "The quick brown fox jumps over the lazy dog." Note that in this example, the `isPartial` value is `true`. If you look deeper into the message, you can see that the system processed the word "fox" as "facts." The system uses the same `resultId` to update the transcript.
```
{
transcript: {
results: [{
resultId:"1", isPartial: true,
startTimeMs: 1620118800000, endTimeMs: 1620118801000,
alternatives: [{
items:[{
type: 'pronunciation',
startTimeMs: 1620118800000, endTimeMs: 1620118800200,
attendee: { attendeeId: "1", externalUserId: "A"},
content: "the", vocabularyFilterMatch: false
},
{
type: 'pronunciation',
startTimeMs: 1620118800200, endTimeMs: 1620118800400,
attendee: { attendeeId: "1", externalUserId: "A" },
content:"quick", vocabularyFilterMatch: false
},
{
type:'pronunciation',
startTimeMs: 1620118800400, endTimeMs: 1620118800750,
attendee: { attendeeId: "1", externalUserId: "A" },
content:"brown", vocabularyFilterMatch: false
},
{
type:'pronunciation',
startTimeMs: 1620118800750, endTimeMs: 1620118801000,
attendee:{ attendeeId: "1", externalUserId: "A" },
content:"facts", vocabularyFilterMatch: false
},
{
type:'punctuation',
startTimeMs: 1620118801000, endTimeMs: 1620118801500,
attendee:{ attendeeId: "1", externalUserId: "A" },
content: ",", vocabularyFilterMatch: false
}]
}]
}]
}
}
```
## Example 3: A final transcript result
In the event of a partial transcript, the system processes the phrase again. This example has an `isPartial` value of `false`, and the message contains "fox" instead of "facts." The system re-issues the message using the same ID.
```
{
transcript: {
results: [{
resultId:"1", isPartial: false,
startTimeMs: 1620118800000, endTimeMs: 1620118801000,
alternatives: [{
items:[{
type: 'pronunciation',
startTimeMs: 1620118800000, endTimeMs: 1620118800200,
attendee: { attendeeId: "1", externalUserId: "A"},
content: "the", vocabularyFilterMatch: false
},
{
type: 'pronunciation',
startTimeMs: 1620118800200, endTimeMs: 1620118800400,
attendee: { attendeeId: "1", externalUserId: "A" },
content:"quick", vocabularyFilterMatch: false
},
{
type:'pronunciation',
startTimeMs: 1620118800400, endTimeMs: 1620118800750,
attendee: { attendeeId: "1", externalUserId: "A" },
content:"brown", vocabularyFilterMatch: false
},
{
type:'pronunciation',
startTimeMs: 1620118800750, endTimeMs: 1620118801000,
attendee: { attendeeId: "1", externalUserId: "A" },
content:"fox", vocabularyFilterMatch: false
},
{
type:'punctuation',
startTimeMs: 1620118801000, endTimeMs: 1620118801500,
attendee: { attendeeId: "1", externalUserId: "A" },
content: ",", vocabularyFilterMatch: false
}]
}]
}]
}
}
```
# Parsing Amazon Chime SDK transcripts
Use the following command to parse transcription content from a transcription message. The command parses complete sentences from the transcript-message.txt files.
```
with open('transcript-message.txt') as f:
for line in f:
result_json = json.loads(line)["transcript"]["results"][0]
if result_json['isPartial'] == False:
print(result_json["alternatives"][0]["transcript"])
```