# Amazon Transcribe Medical Amazon Transcribe Medical is an automatic speech recognition (ASR) service designed for medical professionals who want to transcribe medical-related speech, such as physician-dictated notes, drug safety monitoring, telemedicine appointments, or physician-patient conversations. Amazon Transcribe Medical is available through either real-time streaming (via microphone) or transcription of an uploaded file (batch). **Important** Amazon Transcribe Medical is not a substitute for professional medical advice, diagnosis, or treatment. Identify the right confidence threshold for your use case, and use high confidence thresholds in situations that require high accuracy. For certain use cases, results should be reviewed and verified by appropriately trained human reviewers. Amazon Transcribe Medical transcriptions should only be used in patient care scenarios after review for accuracy and sound medical judgment by trained medical professionals. Amazon Transcribe Medical operates under a shared responsibility model, whereby AWS is responsible for protecting the infrastructure that runs Amazon Transcribe Medical and you are responsible for managing your data. For more information, see [Shared Responsibility Model](https://aws.amazon.com/compliance/shared-responsibility-model/). Amazon Transcribe Medical is available in US English (en-US). For best results, use a lossless audio format, such as FLAC or WAV, with PCM 16-bit encoding. Amazon Transcribe Medical supports sample rates of 16,000 Hz or higher. For analysis of your transcripts, you can use other AWS services, such as [Amazon Comprehend Medical](https://docs.aws.amazon.com/comprehend/latest/dg/comprehend-medical.html). **Supported specialties** | Specialty | Sub-specialty | Audio input | | --- | --- | --- | | Cardiology | none | streaming only | | Neurology | none | streaming only | | Oncology | none | streaming only | | Primary Care | Family Medicine | batch, streaming | | Primary Care | Internal Medicine | batch, streaming | | Primary Care | Obstetrics and Gynecology (OB-GYN) | batch, streaming | | Primary Care | Pediatrics | batch, streaming | | Radiology | none | streaming only | | Urology | none | streaming only | ## Region availability and quotas Call Analytics is supported in the following AWS Regions: | **Region** | **Transcription type** | | --- | --- | | af-south-1 (Cape Town) | batch | | ap-east-1 (Hong Kong) | batch | | ap-northeast-1 (Tokyo) | batch, streaming | | ap-northeast-2 (Seoul) | batch, streaming | | ap-south-1 (Mumbai) | batch | | ap-southeast-1 (Singapore) | batch | | ap-southeast-2 (Sydney) | batch, streaming | | ca-central-1 (Canada, Central) | batch, streaming | | eu-central-1 (Frankfurt) | batch, streaming | | eu-north-1 (Stockholm) | batch | | eu-west-1 (Ireland) | batch, streaming | | eu-west-2 (London) | batch, streaming | | eu-west-3 (Paris) | batch | | me-south-1 (Bahrain) | batch | | sa-east-1 (São Paulo) | batch, streaming | | us-east-1 (N. Virginia) | batch, streaming | | us-east-2 (Ohio) | batch, streaming | | us-gov-east-1 (GovCloud, US-East) | batch, streaming | | us-gov-west-1 (GovCloud, US-West) | batch, streaming | | us-west-1 (San Francisco) | batch | | us-west-2 (Oregon) | batch, streaming | Note that Region support differs for [Amazon Transcribe](what-is.md#tsc-regions), Amazon Transcribe Medical, and [Call Analytics.](call-analytics.md#tca-regions). To get the endpoints for each supported Region, see [Service endpoints](https://docs.aws.amazon.com/general/latest/gr/transcribe.html#transcribe_region) in the *AWS General Reference*. For a list of quotas that pertain to your transcriptions, refer to the [Service quotas](https://docs.aws.amazon.com/general/latest/gr/transcribe.html#limits-amazon-transcribe) in the *AWS General Reference*. Some quotas can be changed upon request. If the **Adjustable** column contains '**Yes**', you can request an increase. To do so, select the provided link. # Medical specialties and terms When creating a medical transcription job, you specify the language, the medical specialty, and the audio type of the source file. You input US English (en-US) as the language and `PRIMARYCARE` as the medical specialty. Entering primary care as the value enables you to generate transcriptions from source audio in the following medical specialties: + Family Medicine + Internal Medicine + Obstetrics and Gynecology (OB-GYN) + Pediatrics You have the choice between dictation and conversation for your audio type. Choose dictation for audio files where the physician is giving a report about a patient visit or procedure. Choose conversation for audio files that involve a conversation between a physician and a patient or a conversation between physicians. To store the output of your transcription job, select an Amazon S3 bucket that you've already created. For more information on Amazon S3 buckets see [Getting Started with Amazon Simple Storage Service](https://docs.aws.amazon.com/AmazonS3/latest/gsg/GetStartedWithS3.html). The following are the minimum number of request parameters to enter in the sample JSON. ``` { "MedicalTranscriptionJobName": "my-first-transcription-job", "LanguageCode": "en-US", "Media": { "MediaFileUri": "s3://path to your audio file" }, "OutputBucketName": “your output bucket name", "Specialty": "PRIMARYCARE", "Type": "CONVERSATION" } ``` Amazon Transcribe Medical enables you to generate alternative transcriptions. For more information, see [Generating alternative transcriptions](alternative-med-transcriptions.md). You can also enable speaker partitioning or identify channels in your audio. For more information, see [Enabling speaker partitioning](conversation-diarization-med.md) and [Transcribing multi-channel audio](conversation-channel-id-med.md). # Transcribing medical terms and measurements Amazon Transcribe Medical can transcribe medical terms and measurements. Amazon Transcribe Medical outputs abbreviations for spoken terms. For example, "blood pressure" is transcribed as BP. You can find a list of conventions that Amazon Transcribe Medical uses for medical terms and measurements in the table on this page. The *Spoken Term* column refers to the term spoken in the source audio. The *Output* column refers to the abbreviation you see in your transcription results. You can see how the terms spoken in source audio correspond to the transcription output here. | Term spoken in source audio | Abbreviation used in output | Example output | | --- | --- | --- | | Centigrade | C | The patient's temperature is 37.4 C. | | Celsius | C | The patient's temperature is 37.4 C. | | Fahrenheit | F | The patient's temperature is 101 F. | | grams | g | A mass of 100 g was extracted from the patient. | | meters | m | The patient is 1.8 m tall. | | feet | ft | The patient is 6 ft tall. | | kilos | kg | The patient weighs 80 kg. | | kilograms | kg | The patient weighs 80 kg. | | c c | cc | Patient received 100 cc of saline solution. | | cubic centimeter | cc | Patient received 100 cc of saline solution. | | milliliter | mL | Patient excreted 100 mL urine. | | blood pressure | BP | Patient BP was elevated. | | b p | BP | Patient BP was elevated. | | X over Y | X/Y | Patient BP was 120/80. | | beats per min | BPM | Patient had atrial fibrillation with heart rate of 160 BPM. | | beats per minute | BPM | Patient had atrial fibrillation with heart rate of 160 BPM. | | O 2 | O2 | Patient O2 saturation was 98%. | | CO2 | CO2 | Patient required respiratory support for elevated CO2. | | post operation | POSTOP | Patient came for POSTOP evaluation. | | post op | POSTOP | Patient came for POSTOP evaluation. | | cat scan | CT Scan | Patient indication of cerebral hemorrhage required use of CT Scan. | | Pulse 80 | P 80 | Patient vitals were P 80, R 17,... | | Respiration 17 | R 17 | Patient vitals were P 80, R 17,... | | in and out | I/O | Patient was I/O sinus rhythm | | L five | L5 | Lumbar puncture was performed between L4 and L5 | # Transcribing numbers Amazon Transcribe Medical transcribes digits as numbers instead of words. For example, the spoken number "one thousand two hundred forty-two" is transcribed as `1242`. Numbers are transcribed according to the following rules. | Rule | Description | | --- | --- | | Convert cardinal numbers greater than 10 to numbers. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Convert cardinal numbers followed by "million" or "billion" to numbers followed by a word when "million" or "billion" is not followed by a number. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Convert ordinal numbers greater than 10 to numbers. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Convert fractions to their numeric format. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Convert numbers less than 10 to digits if there are more than one in a row. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Decimals are indicated by "dot" or "point." | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Convert the word "percent" after a number to a percent sign. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Convert the words "dollar," "US dollar," "Australian dollar, "AUD," or "USD" after a number to a dollar symbol before the number. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Convert the words "pounds," or "milligrams" to "lbs" or "mg". | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Convert the words "rupees," "Indian rupees," or "INR" after a number to rupee sign (₹) before the number. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Convert times to numbers. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Combine years expressed as two digits into four. Only valid for the 20th, 21st, and 22nd centuries. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Convert dates to numbers. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | | Separate spans of numbers by the word "to." | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transcribe/latest/dg/how-numbers-med.html) | # Transcribing a medical conversation You can use Amazon Transcribe Medical to transcribe a medical conversation between a clinician and a patient using either a batch transcription job or a real-time stream. Batch transcription jobs enable you to transcribe audio files. To ensure that Amazon Transcribe Medical produces transcription results with the highest possible accuracy, you must specify the medical specialty of the clinician in your transcription job or stream. You can transcribe a clinician-patient visit in the following medical specialities: + Cardiology – available in streaming transcription only + Neurology – available in streaming transcription only + Oncology – available in streaming transcription only + Primary Care – includes the following types of medical practice: + Family medicine + Internal medicine + Obstetrics and Gynecology (OB-GYN) + Pediatrics + Urology – available in streaming transcription only You can improve transcription accuracy by using medical custom vocabularies. For information on how medical custom vocabularies work, see [Improving transcription accuracy with medical custom vocabularies](vocabulary-med.md). By default, Amazon Transcribe Medical returns the transcription with the highest confidence level. If you'd like to configure it to return alternative transcriptions, see [Generating alternative transcriptions](alternative-med-transcriptions.md). For information about how numbers and medical measurements appear in the transcription output, see [Transcribing numbers](how-numbers-med.md) and [Transcribing medical terms and measurements](how-measurements-med.md). **Topics** + [Transcribing an audio file of a medical conversation](batch-medical-conversation.md) + [Transcribing a medical conversation in a real-time stream](streaming-medical-conversation.md) + [Enabling speaker partitioning](conversation-diarization-med.md) + [Transcribing multi-channel audio](conversation-channel-id-med.md) # Transcribing an audio file of a medical conversation Use a batch transcription job to transcribe audio files of medical conversations. You can use this to transcribe a clinician-patient dialogue. You can start a batch transcription job in either the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API or the AWS Management Console. When you start a medical transcription job with the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API, you specify `PRIMARYCARE` as the value of the `Specialty` parameter. ## AWS Management Console **To transcribe a clinician-patient dialogue (AWS Management Console)** To use the AWS Management Console to transcribe a clinician-patient dialogue, create a transcription job and choose **Conversation** for **Audio input type**. 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Transcription jobs**. 1. Choose **Create job**. 1. On the **Specify job details** page, under **Job settings **, specify the following. 1. **Name** – the name of the transcription job. 1. **Audio input type** – **Conversation** 1. For the remaining fields, specify the Amazon S3 location of your audio file and where you want to store the output of your transcription job. 1. Choose **Next**. 1. Choose **Create**. ## API **To transcribe a medical conversation using a batch transcription job (API)** + For the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API, specify the following. 1. For `MedicalTranscriptionJobName`, specify a name unique in your AWS account. 1. For `LanguageCode`, specify the language code that corresponds to the language spoken in your audio file and the language of your vocabulary filter. 1. For the `MediaFileUri` parameter of the `Media` object, specify the name of the audio file that you want to transcribe. 1. For `Specialty`, specify the medical specialty of the clinician speaking in the audio file as `PRIMARYCARE`. 1. For `Type`, specify `CONVERSATION`. 1. For `OutputBucketName`, specify the Amazon S3 bucket to store the transcription results. The following is an example request that uses the AWS SDK for Python (Boto3) to transcribe a medical conversation of a clinician in the `PRIMARYCARE` specialty and a patient. ``` from __future__ import print_function import time import boto3 transcribe = boto3.client('transcribe', 'us-west-2') job_name = "my-first-med-transcription-job" job_uri = "s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac" transcribe.start_medical_transcription_job( MedicalTranscriptionJobName = job_name, Media = { 'MediaFileUri': job_uri }, OutputBucketName = 'amzn-s3-demo-bucket', OutputKey = 'output-files/', LanguageCode = 'en-US', Specialty = 'PRIMARYCARE', Type = 'CONVERSATION' ) while True: status = transcribe.get_medical_transcription_job(MedicalTranscriptionJobName = job_name) if status['MedicalTranscriptionJob']['TranscriptionJobStatus'] in ['COMPLETED', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) ``` The following example code shows the transcription results of a clinician-patient conversation. ``` { "jobName": "conversation-medical-transcription-job", "accountId": "111122223333", "results": { "transcripts": [ { "transcript": "... come for a follow up visit today..." } ], "items": [ { ... "start_time": "4.85", "end_time": "5.12", "alternatives": [ { "confidence": "1.0", "content": "come" } ], "type": "pronunciation" }, { "start_time": "5.12", "end_time": "5.29", "alternatives": [ { "confidence": "1.0", "content": "for" } ], "type": "pronunciation" }, { "start_time": "5.29", "end_time": "5.33", "alternatives": [ { "confidence": "0.9955", "content": "a" } ], "type": "pronunciation" }, { "start_time": "5.33", "end_time": "5.66", "alternatives": [ { "confidence": "0.9754", "content": "follow" } ], "type": "pronunciation" }, { "start_time": "5.66", "end_time": "5.75", "alternatives": [ { "confidence": "0.9754", "content": "up" } ], "type": "pronunciation" }, { "start_time": "5.75", "end_time": "6.02", "alternatives": [ { "confidence": "1.0", "content": "visit" } ] ... }, "status": "COMPLETED" } ``` ## AWS CLI **To transcribe a medical conversation using a batch transcription job (AWS CLI)** + Run the following code. ``` aws transcribe start-medical-transcription-job \ --region us-west-2 \ --cli-input-json file://example-start-command.json ``` The following code shows the contents of `example-start-command.json`. ``` { "MedicalTranscriptionJobName": "my-first-med-transcription-job", "Media": { "MediaFileUri": "s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac" }, "OutputBucketName": "amzn-s3-demo-bucket", "OutputKey": "my-output-files/", "LanguageCode": "en-US", "Specialty": "PRIMARYCARE", "Type": "CONVERSATION" } ``` # Transcribing a medical conversation in a real-time stream You can transcribe an audio stream of a medical conversation using either the HTTP/2 or [WebSocket ](https://tools.ietf.org/html/rfc6455) protocols. For information on how to start a stream using the WebSocket protocol, see [Setting up a WebSocket stream](streaming-setting-up.md#streaming-websocket). To start an HTTP/2 stream, use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API. You can transcribe streaming audio in the following medical specialties: + Cardiology + Neurology + Oncology + Primary Care + Urology Each medical specialty includes many types of procedures and appointments. Clinicians therefore dictate many different types of notes. Use the following examples as guidance to help you specify the value of the `specialty` URI parameter of the WebSocket request, or the `Specialty` parameter of the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API: + For electrophysiology or echocardiography consultations, choose `CARDIOLOGY`. + For medical oncology, surgical oncology, or radiation oncology consultations, choose `ONCOLOGY`. + For a physician providing a consultation to a patient who had a stroke, either a transient ischemic attack or a cerebrovascular attack, choose `NEUROLOGY`. + For a consultation around urinary incontinence, choose `UROLOGY`. + For yearly checkup or urgent care visits, choose `PRIMARYCARE`. + For inpatient hospitalist visits, choose `PRIMARYCARE`. + For consultations regarding fertility, tubal ligation, IUD insertion, or abortion, choose `PRIMARYCARE`. ## AWS Management Console **To transcribe a streaming medical conversation (AWS Management Console)** To use the AWS Management Console to transcribe a clinician-patient dialogue in real-time stream, choose the option to transcribe a medical conversation, start the stream, and begin speaking into the microphone. 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Real-time transcription**. 1. Choose **Conversation**. 1. For **Medical specialty**, choose the clinician's specialty. 1. Choose **Start streaming**. 1. Speak into the microphone. ## Transcribing a medical conversation in an HTTP/2 stream The following is the syntax for the parameters of an HTTP/2 request. To transcribe an HTTP/2 stream of a medical conversation, use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API and specify the following: + `LanguageCode` – The language code. The valid value is `en-US` + `MediaEncoding` – The encoding used for the input audio. Valid values are `pcm`, `ogg-opus`, and `flac`. + `Specialty` – The specialty of the medical professional. + `Type` – `CONVERSATION` To improve transcription accuracy of specific terms in a real-time stream, use a custom vocabulary. To enable a custom vocabulary, set the value of `VocabularyName` parameter to the name of the custom vocabulary that you want to use. For more information, see [Improving transcription accuracy with medical custom vocabularies](vocabulary-med.md). To label the speech from different speakers, set the `ShowSpeakerLabel` parameter to `true`. For more information, see [Enabling speaker partitioning](conversation-diarization-med.md). For more information on setting up an HTTP/2 stream to transcribe a medical conversation, see [Setting up an HTTP/2 stream](streaming-setting-up.md#streaming-http2). ## Transcribing a medical conversation in a WebSocket stream You can use a WebSocket request to transcribe a medical conversation. When you make a WebSocket request, you create a presigned URI. This URI contains the information needed to set up the audio stream between your application and Amazon Transcribe Medical. For more information on creating WebSocket requests, see [Setting up a WebSocket stream](streaming-setting-up.md#streaming-websocket). Use the following template to create your presigned URI. ``` GET wss://transcribestreaming.us-west-2.amazonaws.com:8443/medical-stream-transcription-websocket ?language-code=languageCode &X-Amz-Algorithm=AWS4-HMAC-SHA256 &X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20220208%2Fus-west-2%2Ftranscribe%2Faws4_request &X-Amz-Date=20220208T235959Z &X-Amz-Expires=300 &X-Amz-Security-Token=security-token &X-Amz-Signature=Signature Version 4 signature &X-Amz-SignedHeaders=host &media-encoding=flac &sample-rate=16000 &session-id=sessionId &specialty=medicalSpecialty &type=CONVERSATION &vocabulary-name=vocabularyName &show-speaker-label=boolean ``` To improve transcription accuracy of specific terms in a real-time stream, use a custom vocabulary. To enable a custom vocabulary, set the value of `vocabulary-name` to the name of the custom vocabulary that you want to use. For more information, see [Improving transcription accuracy with medical custom vocabularies](vocabulary-med.md). To label the speech from different speakers, set the `show-speaker-label` parameter in to `true`. For more information, see [Enabling speaker partitioning](conversation-diarization-med.md). For more information on creating pre-signed URIs, see [Setting up a WebSocket stream](streaming-setting-up.md#streaming-websocket). # Enabling speaker partitioning To enable speaker partitioning in Amazon Transcribe Medical, use *speaker diarization*. This enables you to see what the patient said and what the clinician said in the transcription output. When you enable speaker diarization, Amazon Transcribe Medical labels each speaker *utterance* with a unique identifier for each speaker. An *utterance* is a unit of speech that is typically separated from other utterances by silence. In batch transcription, an utterance from the clinician could receive a label of `spk_0` and an utterance the patient could receive a label of `spk_1`. If an utterance from one speaker overlaps with an utterance from another speaker, Amazon Transcribe Medical orders them in the transcription by their start times. Utterances that overlap in the input audio don't overlap in the transcription output. You can enable speaker diarization when you transcribe an audio file using batch transcription job, or in a real-time stream. **Topics** + [Enabling speaker partitioning in batch transcriptions](conversation-diarization-batch-med.md) + [Enabling speaker partitioning in real-time streams](conversation-diarization-streaming-med.md) # Enabling speaker partitioning in batch transcriptions You can enable speaker partitioning in a batch transcription job using either the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API or the AWS Management Console. This enables you to partition the text per speaker in a clinician-patient conversation and determine who said what in the transcription output. ## AWS Management Console To use the AWS Management Console to enable speaker diarization in your transcription job, you enable audio identification and then speaker partitioning. 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Transcription jobs**. 1. Choose **Create job**. 1. On the **Specify job details** page, provide information about your transcription job. 1. Choose **Next**. 1. Enable **Audio identification**. 1. For **Audio identification type**, choose **Speaker partitioning**. 1. For **Maximum number of speakers**, enter the maximum number of speakers that you think are speaking in your audio file. 1. Choose **Create**. ## API **To enable speaker partitioning using a batch transcription job (API)** + For the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API, specify the following. 1. For `MedicalTranscriptionJobName`, specify a name that is unique in your AWS account. 1. For `LanguageCode`, specify the language code that corresponds to the language spoken in the audio file. 1. For the `MediaFileUri` parameter of the `Media` object, specify the name of the audio file that you want to transcribe. 1. For `Specialty`, specify the medical specialty of the clinician speaking in the audio file. 1. For `Type`, specify `CONVERSATION`. 1. For `OutputBucketName`, specify the Amazon S3 bucket to store the transcription results. 1. For the `Settings` object, specify the following. 1. `ShowSpeakerLabels` – `true`. 1. `MaxSpeakerLabels` – An integer between 2 and 10 to indicate the number of speakers that you think are speaking in your audio. The following request uses the AWS SDK for Python (Boto3) to start a batch transcription job of a primary care clinician patient dialogue with speaker partitioning enabled. ``` from __future__ import print_function import time import boto3 transcribe = boto3.client('transcribe', 'us-west-2') job_name = "my-first-transcription-job" job_uri = "s3://amzn-s3-demo-bucket/my-input-files/my-media-file.flac" transcribe.start_medical_transcription_job( MedicalTranscriptionJobName = job_name, Media={ 'MediaFileUri': job_uri }, OutputBucketName = 'amzn-s3-demo-bucket', OutputKey = 'my-output-files/', LanguageCode = 'en-US', Specialty = 'PRIMARYCARE', Type = 'CONVERSATION', OutputBucketName = 'amzn-s3-demo-bucket', Settings = {'ShowSpeakerLabels': True, 'MaxSpeakerLabels': 2 } ) while True: status = transcribe.get_medical_transcription_job(MedicalTranscriptionJobName = job_name) if status['MedicalTranscriptionJob']['TranscriptionJobStatus'] in ['COMPLETED', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) ``` The following example code shows the transcription results of a transcription job with speaker partitioning enabled. ``` { "jobName": "job ID", "accountId": "111122223333", "results": { "transcripts": [ { "transcript": "Professional answer." } ], "speaker_labels": { "speakers": 1, "segments": [ { "start_time": "0.000000", "speaker_label": "spk_0", "end_time": "1.430", "items": [ { "start_time": "0.100", "speaker_label": "spk_0", "end_time": "0.690" }, { "start_time": "0.690", "speaker_label": "spk_0", "end_time": "1.210" } ] } ] }, "items": [ { "start_time": "0.100", "end_time": "0.690", "alternatives": [ { "confidence": "0.8162", "content": "Professional" } ], "type": "pronunciation" }, { "start_time": "0.690", "end_time": "1.210", "alternatives": [ { "confidence": "0.9939", "content": "answer" } ], "type": "pronunciation" }, { "alternatives": [ { "content": "." } ], "type": "punctuation" } ] }, "status": "COMPLETED" } ``` ## AWS CLI **To transcribe an audio file of a conversation between a clinician practicing primary care and a patient (AWS CLI)** + Run the following code. ``` aws transcribe start-transcription-job \ --region us-west-2 \ --cli-input-json file://example-start-command.json ``` The following code shows the contents of `example-start-command.json`. ``` { "MedicalTranscriptionJobName": "my-first-med-transcription-job", "Media": { "MediaFileUri": "s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac" }, "OutputBucketName": "amzn-s3-demo-bucket", "OutputKey": "my-output-files/", "LanguageCode": "en-US", "Specialty": "PRIMARYCARE", "Type": "CONVERSATION", "Settings":{ "ShowSpeakerLabels": true, "MaxSpeakerLabels": 2 } } ``` # Enabling speaker partitioning in real-time streams To partition speakers and label their speech in a real-time stream, use the AWS Management Console or a streaming request. Speaker partitioning works best for between two and five speakers in a stream. Although Amazon Transcribe Medical can partition more than five speakers in a stream, the accuracy of the partitions decrease if you exceed that number. To start an HTTP/2 request, use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API. To start a WebSocket request, use a pre-signed URI. The URI contains the information required to set up bi-directional communication between your application and Amazon Transcribe Medical. ## Enabling speaker partitioning in audio that is spoken into your microphone (AWS Management Console) You can use the AWS Management Console to start a real-time stream of a clinician-patient conversation, or a dictation that is spoken into your microphone in real-time. 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, for Amazon Transcribe Medical choose **Real-time transcription**. 1. For **Audio input type**, choose the type of medical speech that you want to transcribe. 1. For **Additional settings**, choose **Speaker partitioning**. 1. Choose **Start streaming** to start transcribing your real-time audio. 1. Speak into the microphone. ## Enabling speaker partitioning in an HTTP/2 stream To enable speaker partitioning in an HTTP/2 stream of a medical conversation, use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API and specify the following: + For `LanguageCode`, specify the language code that corresponds to the language in the stream. The valid value is `en-US`. + For `MediaSampleHertz`, specify the sample rate of the audio. + For `Specialty`, specify the medical specialty of the provider. + `ShowSpeakerLabel` – `true` For more information on setting up an HTTP/2 stream to transcribe a medical conversation, see [Setting up an HTTP/2 stream](streaming-setting-up.md#streaming-http2). ## Enabling speaker partitioning in a WebSocket request To partition speakers in WebSocket streams with the API, use the following format to create a pre-signed URI to start a WebSocket request and set `show-speaker-label` to `true`. ``` GET wss://transcribestreaming.us-west-2.amazonaws.com:8443/medical-stream-transcription-websocket ?language-code=languageCode &X-Amz-Algorithm=AWS4-HMAC-SHA256 &X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20220208%2Fus-west-2%2Ftranscribe%2Faws4_request &X-Amz-Date=20220208T235959Z &X-Amz-Expires=300 &X-Amz-Security-Token=security-token &X-Amz-Signature=Signature Version 4 signature &X-Amz-SignedHeaders=host &media-encoding=flac &sample-rate=16000 &session-id=sessionId &specialty=medicalSpecialty &type=CONVERSATION &vocabulary-name=vocabularyName &show-speaker-label=boolean ``` The following code shows the truncated example response of a streaming request. ``` { "Transcript": { "Results": [ { "Alternatives": [ { "Items": [ { "Confidence": 0.97, "Content": "From", "EndTime": 18.98, "Speaker": "0", "StartTime": 18.74, "Type": "pronunciation", "VocabularyFilterMatch": false }, { "Confidence": 1, "Content": "the", "EndTime": 19.31, "Speaker": "0", "StartTime": 19, "Type": "pronunciation", "VocabularyFilterMatch": false }, { "Confidence": 1, "Content": "last", "EndTime": 19.86, "Speaker": "0", "StartTime": 19.32, "Type": "pronunciation", "VocabularyFilterMatch": false }, ... { "Confidence": 1, "Content": "chronic", "EndTime": 22.55, "Speaker": "0", "StartTime": 21.97, "Type": "pronunciation", "VocabularyFilterMatch": false }, ... "Confidence": 1, "Content": "fatigue", "EndTime": 24.42, "Speaker": "0", "StartTime": 23.95, "Type": "pronunciation", "VocabularyFilterMatch": false }, { "EndTime": 25.22, "StartTime": 25.22, "Type": "speaker-change", "VocabularyFilterMatch": false }, { "Confidence": 0.99, "Content": "True", "EndTime": 25.63, "Speaker": "1", "StartTime": 25.22, "Type": "pronunciation", "VocabularyFilterMatch": false }, { "Content": ".", "EndTime": 25.63, "StartTime": 25.63, "Type": "punctuation", "VocabularyFilterMatch": false } ], "Transcript": "From the last note she still has mild sleep deprivation and chronic fatigue True." } ], "EndTime": 25.63, "IsPartial": false, "ResultId": "XXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXX", "StartTime": 18.74 } ] } } ``` Amazon Transcribe Medical breaks your incoming audio stream based on natural speech segments, such as a change in speaker or a pause in the audio. The transcription is returned progressively to your application, with each response containing more transcribed speech until the entire segment is transcribed. The preceding code is a truncated example of a fully-transcribed speech segment. Speaker labels only appear for entirely transcribed segments. The following list shows the organization of the objects and parameters in a streaming transcription output. **`Transcript`** Each speech segment has its own `Transcript` object. **`Results`** Each `Transcript` object has its own `Results` object. This object contains the `isPartial` field. When its value is `false`, the results returned are for an entire speech segment. **`Alternatives`** Each `Results` object has an `Alternatives` object. **`Items`** Each `Alternatives` object has its own `Items` object that contains information about each word and punctuation mark in the transcription output. When you enable speaker partitioning, each word has a `Speaker` label for fully-transcribed speech segments. Amazon Transcribe Medical uses this label to assign a unique integer to each speaker in the stream. The `Type` parameter having a value of `speaker-change` indicates that one person has stopped speaking and that another person is about to begin. **`Transcript`** Each Items object contains a transcribed speech segment as the value of the `Transcript` field. For more information about WebSocket requests, see [Setting up a WebSocket stream](streaming-setting-up.md#streaming-websocket). # Transcribing multi-channel audio If you have an audio file or stream that has multiple channels, you can use *channel identification* to transcribe the speech from each of those channels. Amazon Transcribe Medical transcribes the speech from each channel separately. It combines the separate transcriptions of each channel into a single transcription output. Use channel identification to identify the separate channels in your audio and transcribe the speech from each of those channels. Enable this in situations such as a caller and agent scenario. Use this to distinguish a caller from an agent in recordings or streams from contact centers that perform drug safety monitoring. You can enable channel identification for both batch processing and real-time streaming. The following list describes how to enable it for each method. + Batch transcription – AWS Management Console and [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API + Streaming transcription – WebSocket streaming and [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API ## Transcribing multi-channel audio files When you transcribe an audio file, Amazon Transcribe Medical returns a list of *items* for each channel. An item is a transcribed word or punctuation mark. Each word has a start time and an end time. If a person on one channel speaks over a person on a separate channel, the start times and end times of the items for each channel overlap while the individuals are speaking over each other. By default, you can transcribe audio files with two channels. You can request a quota increase if you need to transcribe files that have more than two channels. For information about requesting a quota increase, see [AWS service quotas](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html). To transcribe multi-channel audio in a batch transcription job, use the AWS Management Console or the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API. ### AWS Management Console To use the AWS Management Console to enable channel identification in your batch transcription job, you enable audio identification and then channel identification. Channel identification is a subset of audio identification in the AWS Management Console. 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Transcription jobs**. 1. Choose **Create job**. 1. On the **Specify job details** page, provide information about your transcription job. 1. Choose **Next**. 1. Enable **Audio identification**. 1. For **Audio identification type**, choose **Channel identification**. 1. Choose **Create**. ### API **To transcribe a multi-channel audio file (API)** + For the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API, specify the following. 1. For `TranscriptionJobName`, specify a name unique to your AWS account. 1. For `LanguageCode`, specify the language code that corresponds to the language spoken in the audio file. The valid value is `en-US`. 1. For the `MediaFileUri` parameter of the `Media` object, specify the name of the media file that you want to transcribe. 1. For the `Settings` object, set `ChannelIdentification` to `true`. The following is an example request using the AWS SDK for Python (Boto3). ``` from __future__ import print_function import time import boto3 transcribe = boto3.client('transcribe', 'us-west-2') job_name = "my-first-transcription-job" job_name = "my-first-med-transcription-job" job_uri = "s3://amzn-s3-demo-bucket/my-input-files/my-media-file.flac" transcribe.start_medical_transcription_job( MedicalTranscriptionJobName = job_name, Media = { 'MediaFileUri': job_uri }, OutputBucketName = 'amzn-s3-demo-bucket', OutputKey = 'output-files/', LanguageCode = 'en-US', Specialty = 'PRIMARYCARE', Type = 'CONVERSATION', Settings = { 'ChannelIdentification': True } ) while True: status = transcribe.get_transcription_job(MedicalTranscriptionJobName = job_name) if status['MedicalTranscriptionJob']['TranscriptionJobStatus'] in ['COMPLETED', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) ``` ### AWS CLI **To transcribe a multi-channel audio file using a batch transcription job (AWS CLI)** + Run the following code. ``` aws transcribe start-medical-transcription-job \ --region us-west-2 \ --cli-input-json file://example-start-command.json ``` The following is the code of `example-start-command.json`. ``` { "MedicalTranscriptionJobName": "my-first-med-transcription-job", "Media": { "MediaFileUri": "s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac" }, "OutputBucketName": "amzn-s3-demo-bucket", "OutputKey": "my-output-files/", "LanguageCode": "en-US", "Specialty": "PRIMARYCARE", "Type": "CONVERSATION", "Settings":{ "ChannelIdentification": true } } ``` The following code shows the transcription output for an audio file that has a conversation on two channels. ``` { "jobName": "job id", "accountId": "111122223333", "results": { "transcripts": [ { "transcript": "When you try ... It seems to ..." } ], "channel_labels": { "channels": [ { "channel_label": "ch_0", "items": [ { "start_time": "12.282", "end_time": "12.592", "alternatives": [ { "confidence": "1.0000", "content": "When" } ], "type": "pronunciation" }, { "start_time": "12.592", "end_time": "12.692", "alternatives": [ { "confidence": "0.8787", "content": "you" } ], "type": "pronunciation" }, { "start_time": "12.702", "end_time": "13.252", "alternatives": [ { "confidence": "0.8318", "content": "try" } ], "type": "pronunciation" }, ... ] }, { "channel_label": "ch_1", "items": [ { "start_time": "12.379", "end_time": "12.589", "alternatives": [ { "confidence": "0.5645", "content": "It" } ], "type": "pronunciation" }, { "start_time": "12.599", "end_time": "12.659", "alternatives": [ { "confidence": "0.2907", "content": "seems" } ], "type": "pronunciation" }, { "start_time": "12.669", "end_time": "13.029", "alternatives": [ { "confidence": "0.2497", "content": "to" } ], "type": "pronunciation" }, ... ] } } ``` ## Transcribing multi-channel audio streams You can transcribe audio from separate channels in either HTTP/2 or WebSocket streams using the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API. By default, you can transcribe streams with two channels. You can request a quota increase if you need to transcribe streams that have more than two channels. For information about requesting a quota increase, see [AWS service quotas](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html). ### Transcribing multi-channel audio in an HTTP/2 stream To transcribe multi-channel audio in an HTTP/2 stream, use the [StartMedicalStreamTranscription](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API and specify the following: + `LanguageCode` – The language code of the audio. The valid value is `en-US`. + `MediaEncoding` – The encoding of the audio. Valid values are `ogg-opus`, `flac`, and `pcm`. + `EnableChannelIdentification` – `true` + `NumberOfChannels` – the number of channels in your streaming audio. For more information on setting up an HTTP/2 stream to transcribe a medical conversation, see [Setting up an HTTP/2 stream](streaming-setting-up.md#streaming-http2). ### Transcribing multi-channel audio in a WebSocket stream To partition speakers in WebSocket streams, use the following format to create a pre-signed URI and start a WebSocket request. Specify `enable-channel-identification` as `true` and the number of channels in your stream in `number-of-channels`. A pre-signed URI contains the information needed to set up bi-directional communication between your application and Amazon Transcribe Medical. ``` GET wss://transcribestreaming.us-west-2.amazonaws.com:8443/medical-stream-transcription-websocket ?language-code=languageCode &X-Amz-Algorithm=AWS4-HMAC-SHA256 &X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20220208%2Fus-west-2%2Ftranscribe%2Faws4_request &X-Amz-Date=20220208T235959Z &X-Amz-Expires=300 &X-Amz-Security-Token=security-token &X-Amz-Signature=Signature Version 4 signature &X-Amz-SignedHeaders=host &media-encoding=flac &sample-rate=16000 &session-id=sessionId &enable-channel-identification=true &number-of-channels=2 ``` Parameter definitions can be found in the [API Reference](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_Reference.html); parameters common to all AWS API operations are listed in the [Common Parameters](https://docs.aws.amazon.com/transcribe/latest/APIReference/CommonParameters.html) section. For more information about WebSocket requests, see [Setting up a WebSocket stream](streaming-setting-up.md#streaming-websocket). ### Multi-channel streaming output The output of a streaming transcription is the same for HTTP/2 and WebSocket requests. The following is an example output. ``` { "resultId": "XXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXX", "startTime": 0.11, "endTime": 0.66, "isPartial": false, "alternatives": [ { "transcript": "Left.", "items": [ { "startTime": 0.11, "endTime": 0.45, "type": "pronunciation", "content": "Left", "vocabularyFilterMatch": false }, { "startTime": 0.45, "endTime": 0.45, "type": "punctuation", "content": ".", "vocabularyFilterMatch": false } ] } ], "channelId": "ch_0" } ``` For each speech segment, there is a `channelId` flag that indicates which channel the speech belongs to. # Transcribing a medical dictation You can use Amazon Transcribe Medical to transcribe clinician-dictated medical notes using either a batch transcription job or a real-time stream. Batch transcription jobs enable you to transcribe audio files. You specify the medical specialty of the clinician in your transcription job or stream to ensure that Amazon Transcribe Medical produces transcription results with the highest possible accuracy. You can transcribe a medical dictation in the following specialties: + Cardiology – available in streaming transcription only + Neurology – available in streaming transcription only + Oncology – available in streaming transcription only + Primary Care – includes the following types of medical practice: + Family medicine + Internal medicine + Obstetrics and Gynecology (OB-GYN) + Pediatrics + Radiology – available in streaming transcription only + Urology – available in streaming transcription only You can improve transcription accuracy by using custom vocabularies. For information on how medical custom vocabularies work, see [Improving transcription accuracy with medical custom vocabularies](vocabulary-med.md). By default, Amazon Transcribe Medical returns the transcription with the highest confidence level. If you'd like to configure it to return alternative transcriptions, see [Generating alternative transcriptions](alternative-med-transcriptions.md). For information about how numbers and medical measurements appear in the transcription output, see [Transcribing numbers](how-numbers-med.md) and [Transcribing medical terms and measurements](how-measurements-med.md). **Topics** + [Transcribing an audio file of a medical dictation](batch-medical-dictation.md) + [Transcribing a medical dictation in a real-time stream](streaming-medical-dictation.md) # Transcribing an audio file of a medical dictation Use a batch transcription job to transcribe audio files of medical conversations. You can use this to transcribe a clinician-patient dialogue. You can start a batch transcription job in either the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API or the AWS Management Console. When you start a medical transcription job with the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API, you specify `PRIMARYCARE` as the value of the `Specialty` parameter. ## AWS Management Console **To transcribe a clinician-patient dialogue (AWS Management Console)** To use the AWS Management Console to transcribe a clinician-patient dialogue, create a transcription job and choose **Conversation** for **Audio input type**. 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Transcription jobs**. 1. Choose **Create job**. 1. On the **Specify job details** page, under **Job settings **, specify the following. 1. **Name** – the name of the transcription job. 1. **Audio input type** – **Dictation** 1. For the remaining fields, specify the Amazon S3 location of your audio file and where you want to store the output of your transcription job. 1. Choose **Next**. 1. Choose **Create**. ## API **To transcribe a medical conversation using a batch transcription job (API)** + For the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API, specify the following. 1. For `MedicalTranscriptionJobName`, specify a name unique in your AWS account. 1. For `LanguageCode`, specify the language code that corresponds to the language spoken in your audio file and the language of your vocabulary filter. 1. In the `MediaFileUri` parameter of the `Media` object, specify the name of the audio file that you want to transcribe. 1. For `Specialty`, specify the medical specialty of the clinician speaking in the audio file. 1. For `Type`, specify `DICTATION`. 1. For `OutputBucketName`, specify the Amazon S3 bucket to store the transcription results. The following is an example request that uses the AWS SDK for Python (Boto3) to transcribe a medical dictation of a clinician in the `PRIMARYCARE` specialty. ``` from __future__ import print_function import time import boto3 transcribe = boto3.client('transcribe') job_name = "my-first-med-transcription-job" job_uri = "s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac" transcribe.start_medical_transcription_job( MedicalTranscriptionJobName = job_name, Media = { 'MediaFileUri': job_uri }, OutputBucketName = 'amzn-s3-demo-bucket', OutputKey = 'my-output-files/', LanguageCode = 'en-US', Specialty = 'PRIMARYCARE', Type = 'DICTATION' ) while True: status = transcribe.get_medical_transcription_job(MedicalTranscriptionJobName = job_name) if status['MedicalTranscriptionJob']['TranscriptionJobStatus'] in ['COMPLETED', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) ``` The following example code shows the transcription results of a medical dictation. ``` { "jobName": "dictation-medical-transcription-job", "accountId": "111122223333", "results": { "transcripts": [ { "transcript": "... came for a follow up visit today..." } ], "items": [ { ... "start_time": "4.85", "end_time": "5.12", "alternatives": [ { "confidence": "1.0", "content": "came" } ], "type": "pronunciation" }, { "start_time": "5.12", "end_time": "5.29", "alternatives": [ { "confidence": "1.0", "content": "for" } ], "type": "pronunciation" }, { "start_time": "5.29", "end_time": "5.33", "alternatives": [ { "confidence": "0.9955", "content": "a" } ], "type": "pronunciation" }, { "start_time": "5.33", "end_time": "5.66", "alternatives": [ { "confidence": "0.9754", "content": "follow" } ], "type": "pronunciation" }, { "start_time": "5.66", "end_time": "5.75", "alternatives": [ { "confidence": "0.9754", "content": "up" } ], "type": "pronunciation" }, { "start_time": "5.75", "end_time": "6.02", "alternatives": [ { "confidence": "1.0", "content": "visit" } ] ... }, "status": "COMPLETED" } ``` ## AWS CLI **To enable speaker partitioning in a batch transcription job (AWS CLI)** + Run the following code. ``` aws transcribe start-medical-transcription-job \ --region us-west-2 \ --cli-input-json file://example-start-command.json ``` The following code shows the contents of `example-start-command.json`. ``` { "MedicalTranscriptionJobName": "my-first-med-transcription-job", "Media": { "MediaFileUri": "s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac" }, "OutputBucketName": "amzn-s3-demo-bucket", "OutputKey": "my-output-files/", "LanguageCode": "en-US", "Specialty": "PRIMARYCARE", "Type": "DICTATION" } ``` # Transcribing a medical dictation in a real-time stream Use a WebSocket stream to transcribe a medical dictation as an audio stream. You can also use the AWS Management Console to transcribe speech that you or others speak directly into a microphone. For an HTTP/2 or a WebSocket stream, you can transcribe audio in the following medical specialties: + Cardiology + Oncology + Neurology + Primary Care + Radiology + Urology Each medical specialty includes many types of procedures and appointments. Clinicians therefore dictate many different types of notes. Use the following examples as guidance to help you specify the value of the `specialty` URI parameter of the WebSocket request, or the `Specialty` parameter of the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API: + For a dictation after electrophysiology or echocardiogram procedure, choose `CARDIOLOGY`. + For a dictation after a surgical oncology or radiation oncology procedure, choose `ONCOLOGY`. + For a physician dictating notes indicating a diagnosis of encephalitis, choose `NEUROLOGY`. + For a dictation of procedure notes to break up a bladder stone, choose `UROLOGY`. + For a dictation of clinician notes after an internal medicine consultation, choose `PRIMARYCARE`. + For a dictation of a physician communicating the findings of a CT scan, PET scan, MRI, or radiograph, choose `RADIOLOGY`. + For a dictation of physician notes after a gynecology consultation, choose `PRIMARYCARE`. To improve transcription accuracy of specific terms in a real-time stream, use a custom vocabulary. To enable a custom vocabulary, set the value of `vocabulary-name` to the name of the custom vocabulary you want to use. ## Transcribing a dictation spoken into your microphone with the AWS Management Console To use the AWS Management Console to transcribe streaming audio of a medical dictation, choose the option to transcribe a medical dictation, start the stream, and begin speaking into the microphone. **To transcribe streaming audio of a medical dictation (AWS Management Console)** 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Real-time transcription**. 1. Choose **Dictation**. 1. For **Medical specialty**, choose the medical specialty of the clinician speaking in the stream. 1. Choose **Start streaming**. 1. Speak into the microphone. ## Transcribing a dictation in an HTTP/2 stream To transcribe an HTTP/2 stream of a medical dictation, use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API and specify the following: + `LanguageCode` – The language code. The valid value is `en-US` + `MediaEncoding` – The encoding used for the input audio. Valid values are `pcm`, `ogg-opus`, and `flac`. + `Specialty` – The specialty of the medical professional. + `Type` – `DICTATION` For more information on setting up an HTTP/2 stream to transcribe a medical dictation, see [Setting up an HTTP/2 stream](streaming-setting-up.md#streaming-http2). ## Using a WebSocket streaming request to transcribe a medical dictation To transcribe a medical dictation in a real-time stream using a WebSocket request, you create a presigned URI. This URI contains the information needed to set up the audio stream between your application and Amazon Transcribe Medical. For more information on creating WebSocket requests, see [Setting up a WebSocket stream](streaming-setting-up.md#streaming-websocket). Use the following template to create your presigned URI. ``` GET wss://transcribestreaming.us-west-2.amazonaws.com:8443/medical-stream-transcription-websocket ?language-code=languageCode &X-Amz-Algorithm=AWS4-HMAC-SHA256 &X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20220208%2Fus-west-2%2Ftranscribe%2Faws4_request &X-Amz-Date=20220208T235959Z &X-Amz-Expires=300 &X-Amz-Security-Token=security-token &X-Amz-Signature=Signature Version 4 signature &X-Amz-SignedHeaders=host &media-encoding=flac &sample-rate=16000 &session-id=sessionId &specialty=medicalSpecialty &type=DICTATION &vocabulary-name=vocabularyName &show-speaker-label=boolean ``` For more information on creating pre-signed URIs, see [Setting up a WebSocket stream](streaming-setting-up.md#streaming-websocket). # Improving transcription accuracy with medical custom vocabularies To improve transcription accuracy in Amazon Transcribe Medical, create and use one or more medical custom vocabularies. A *custom vocabulary* is a collection of words or phrases that are domain-specific. This collection helps improve the performance of Amazon Transcribe Medical in transcribing those words or phrases. You are responsible for the integrity of your own data when you use Amazon Transcribe Medical. Do not enter confidential information, personal information (PII), or protected health information (PHI), into a custom vocabulary. For best results, create separate small custom vocabularies that each help transcribe a specific audio recording. You receive greater improvements in transcription accuracy than if you created one large custom vocabulary to use with all of your recordings. By default, you can have up to 100 custom vocabularies in your AWS account. A custom vocabulary can't exceed 50 KB in size. For information on requesting an increase to the number of custom vocabularies that you can have in your AWS account, see [AWS service quotas](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html). Custom vocabularies are available in US English (en-US). **Topics** + [Creating a text file for your medical custom vocabulary](create-med-vocab-text.md) + [Using a text file to create a medical custom vocabulary](create-med-custom-vocabulary.md) + [Transcribing an audio file using a medical custom vocabulary](start-med-custom-vocab-job.md) + [Transcribing a real-time stream using a medical custom vocabulary](start-med-vocab-stream.md) + [Character set for Amazon Transcribe Medical](charsets-med.md) # Creating a text file for your medical custom vocabulary To create a custom vocabulary, you create a text file that is in UTF-8 format. In this file, you create a four column table, with each column specifying a field. Each field tells Amazon Transcribe Medical either how the domain-specific terms are pronounced or how to display these terms in your transcriptions. You store the text file containing these fields in an Amazon S3 bucket. ## Understanding how to format your text file To create a medical custom vocabulary, you enter the column names as a header row. You enter the values for each column beneath the header row. The following are the names of the four columns of the table: + `Phrase` – column required, values required + `IPA` – column required, values can be optional + `SoundsLike` – column required, values can be optional + `DisplayAs` – column required, values can be optional When you create a custom vocabulary, make sure that you: + Separate each column with a single Tab character. Amazon Transcribe throws an error message if you try to separate the columns with spaces or multiple Tab characters. + Make sure that there's no trailing spaces or white space after each value within a column. Make sure that the values that you enter for each column: + Have fewer than 256 characters, including hyphens + Use only characters from the allowed character set, see [Character set for Amazon Transcribe Medical](charsets-med.md). ## Entering values for the columns of the table The following information shows you how to specify values for the four columns of the table: + `Phrase` – The word or phrase that should be recognized. You must enter values in this column. If the entry is a phrase, separate the words with a hyphen (-). For example, enter **cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy** as **cerebral-autosomal-dominant-arteriopathy-with-subcortical-infarcts-and-leukoencephalopathy**. Enter acronyms or other words whose letters should be pronounced individually as single letters followed by dots, such as **D.N.A.** or **S.T.E.M.I.**. To enter the plural form of an acronym, such as "STEMIs," separate the "s" from the acronym with a hyphen: "**S.T.E.M.I-s**" You can use either uppercase or lowercase letters for acronyms. The `Phrase` column is required. You can use any of the allowed characters for the input language. For allowed characters, see [Character set for Amazon Transcribe Medical](charsets-med.md). If you don't specify the `DisplayAs` column, Amazon Transcribe Medical uses the contents of the `Phrase` column in the output file. + `IPA` (column required, values can be optional) – To specify the pronunciation of a word or phrase, you can include characters in the [International Phonetic Alphabet (IPA)](https://en.wikipedia.org/wiki/International_Phonetic_Alphabet) in this column. The `IPA` column can't contain leading or trailing spaces, and you must use a single space to separate each phoneme in the input. For example, in English you would enter the phrase **acute-respiratory-distress-syndrome** as **ə k j u t ɹ ɛ s p ɪ ɹ ə t ɔ ɹ i d ɪ s t ɹ ɛ s s ɪ n d ɹ oʊ m**. You would enter the phrase **A.L.L.** as **eɪ ɛ l ɛ l**. Even if you don't specify the contents of the `IPA` column, you must include a blank `IPA` column. If you include values in the `IPA` column, you can't provide values for the `SoundsLike` column. For a list of allowed IPA characters for a specific language, see [Character set for Amazon Transcribe Medical](charsets-med.md). US English is the only language available in Amazon Transcribe Medical. + `SoundsLike` (column required, values can be optional) – You can break a word or phrase down into smaller segments and provide a pronunciation for each segment using the standard orthography of the language to mimic the way that the word sounds. For example, you can provide pronunciation hints for the phrase **cerebral-autosomal-dominant-arteriopathy-with-subcortical-infarcts-and-leukoencephalopathy** like this: **sir-e-brul-aut-o-som-ul-dah-mi-nant-ar-ter-ri-o-pa-thy-with-sub-cor-ti-cul-in-farcts-and-lewk-o-en-ce-phul-ah-pu-thy**. The hint for the phrase **atrioventricular-nodal-reentrant-tachycardia** would look like this: **ay-tree-o-ven-trick-u-lar-node-al-re-entr-ant-tack-ih-card-ia**. You separate each part of the hint with a hyphen (-). Even if you don't provide values for the `SoundsLike` column, you must include a blank `SoundsLike` column. If you include values in the `SoundsLike` column, you can't provide values for the `IPA` column. You can use any of the allowed characters for the input language. For the list of allowed characters, see [Character set for Amazon Transcribe Medical](charsets-med.md). + `DisplayAs` (column required, values can be optional) – Defines how the word or phrase looks when it's output. For example, if the word or phrase is **cerebral-autosomal-dominant-arteriopathy-with-subcortical-infarcts-and-leukoencephalopathy**, you can specify the display form as `cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy`, so that the hyphen is not present. You can also specify `DisplayAs` as `CADASIL` if you'd like to show the acronym instead of the full term in the output. If you don't specify the `DisplayAs` column, Amazon Transcribe Medical uses the `Phrase` column from the input file in the output. You can use any UTF-8 character in the `DisplayAs` column. You can include spaces only for the values in the `IPA` and `DisplayAs` columns. To create the text file of your custom vocabulary, place each word or phrase in your text file on a separate line. Separate the columns with Tab characters. Include spaces only for values in the `IPA` and `DisplayAs` columns. Save the file with the extension `.txt` in an Amazon S3 bucket in the same AWS Region where you use Amazon Transcribe Medical to create your custom vocabulary. If you edit your text file in Windows, make sure that your file is in `LF` format and not in `CRLF` format. Otherwise, you will be unable to create your custom vocabulary. Some text editors enable you to change the formatting with Find and Replace commands. The following examples show text that you can use to create custom vocabularies. To create a custom vocabulary from these examples, copy an example into a text editor, replace `[TAB]` with a Tab character, and upload the saved text file to Amazon S3. ``` Phrase[TAB]IPA[TAB]SoundsLike[TAB]DisplayAs acute-respiratory-distress-syndrome[TAB][TAB][TAB]acute respiratory distress syndrome A.L.L.[TAB]eɪ ɛ l ɛ l[TAB][TAB]ALL atrioventricular-nodal-reentrant-tachycardia[TAB][TAB]ay-tree-o-ven-trick-u-lar-node-al-re-entr-ant-tack-ih-card-ia[TAB] ``` You can enter columns in any order. The following examples show other valid structures for the custom vocabulary input file. ``` Phrase[TAB]SoundsLike[TAB]IPA[TAB]DisplayAs acute-respiratory-distress-syndrome[TAB][TAB][TAB]acute respiratory distress syndrome A.L.L.[TAB][TAB]eɪ ɛ l ɛ l[TAB]ALL atrioventricular-nodal-reentrant-tachycardia[TAB]ay-tree-o-ven-trick-u-lar-node-al-re-entr-ant-tack-ih-card-ia[TAB][TAB] ``` ``` DisplayAs[TAB]SoundsLike[TAB]IPA[TAB]Phrase acute respiratory distress syndrome[TAB][TAB][TAB]acute-respiratory-distress-syndrome ALL[TAB][TAB]eɪ ɛ l ɛ l[TAB]A.L.L. [TAB]ay-tree-o-ven-trick-u-lar-node-al-re-entr-ant-tack-ih-card-ia[TAB][TAB]atrioventricular-nodal-reentrant-tachycardia ``` For reading ease, the following tables show the preceding examples more clearly in html format. They are meant only to illustrate the examples. | Phrase | IPA | SoundsLike | DisplayAs | | --- | --- | --- | --- | | acute-respiratory-distress-syndrome | | | acute respiratory distress syndrome | | A.L.L. | eɪ ɛ l ɛ l | | ALL | | atrioventricular-nodal-reentrant-tachycardia | | ay-tree-o-ven-trick-u-lar-node-al-re-entr-ant-tack-ih-card-ia | | | Phrase | SoundsLike | IPA | DisplayAs | | --- | --- | --- | --- | | acute-respiratory-distress-syndrome | | | acute respiratory distress syndrome | | atrioventricular-nodal-reentrant-tachycardia | ay-tree-o-ven-trick-u-lar-node-al-re-entr-ant-tack-ih-card-ia | | | | A.L.L. | | eɪ ɛ l ɛ l | ALL | | DisplayAs | SoundsLike | IPA | Phrase | | --- | --- | --- | --- | | acute respiratory distress syndrome | | | acute-respiratory-distress-syndrome | | ALL | | eɪ ɛ l ɛ l | A.L.L. | | | ay-tree-o-ven-trick-u-lar-node-al-re-entr-ant-tack-ih-card-ia | | atrioventricular-nodal-reentrant-tachycardia | # Using a text file to create a medical custom vocabulary To create a custom vocabulary, you must have prepared a text file that contains a collection a words or phrases. Amazon Transcribe Medical uses this text file to create a custom vocabulary that you can use to improve the transcription accuracy of those words or phrases. You can create a custom vocabulary using the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_CreateMedicalVocabulary.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_CreateMedicalVocabulary.html) API or the Amazon Transcribe Medical console. ## AWS Management Console To use the AWS Management Console to create a custom vocabulary, you provide the Amazon S3 URI of the text file containing your words or phrases. 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Custom vocabulary**. 1. For **Name**, under **Vocabulary settings**, choose a name for your custom vocabulary. 1. Specify the location of your audio file or video file in Amazon S3: + For **Vocabulary input file location on S3** under **Vocabulary settings**, specify the Amazon S3 URI that identifies the text file you will use to create your custom vocabulary. + For **Vocabulary input file location in S3**, choose **Browse S3** to browse for the text file and choose it. 1. Choose **Create vocabulary**. You can see the processing status of your custom vocabulary in the AWS Management Console. ## API **To create a medical custom vocabulary (API)** + For the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartTranscriptionJob.html) API, specify the following. 1. For `LanguageCode`, specify `en-US`. 1. For `VocabularyFileUri`, specify the Amazon S3 location of the text file that you use to define your custom vocabulary. 1. For `VocabularyName`, specify a name for your custom vocabulary. The name you specify must be unique within your AWS account. To see the processing status of your custom vocabulary, use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_GetMedicalVocabulary.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_GetMedicalVocabulary.html) API. The following is an example request using the AWS SDK for Python (Boto3) to create a custom vocabulary. ``` from __future__ import print_function import time import boto3 transcribe = boto3.client('transcribe', 'us-west-2') vocab_name = "my-first-vocabulary" response = transcribe.create_medical_vocabulary( VocabularyName = job_name, VocabularyFileUri = 's3://amzn-s3-demo-bucket/my-vocabularies/my-vocabulary-table.txt' LanguageCode = 'en-US', ) while True: status = transcribe.get_medical_vocabulary(VocabularyName = vocab_name) if status['VocabularyState'] in ['READY', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) ``` ## AWS CLI **To enable speaker partitioning in a batch transcription job (AWS CLI)** + Run the following code. ``` aws transcribe create-medical-vocabulary \ --vocabulary-name my-first-vocabulary \ --vocabulary-file-uri s3://amzn-s3-demo-bucket/my-vocabularies/my-vocabulary-file.txt \ --language-code en-US ``` # Transcribing an audio file using a medical custom vocabulary Use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) or the AWS Management Console to start a transcription job that uses a custom vocabulary to improve transcription accuracy. ## AWS Management Console 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Transcription jobs**. 1. Choose **Create job**. 1. On the **Specify job details** page, provide information about your transcription job. 1. Choose **Next**. 1. Under **Customization**, enable **Custom vocabulary**. 1. Under **Vocabulary selection**, choose a custom vocabulary. 1. Choose **Create**. ## API **To enable speaker partitioning in an audio file using a batch transcription job (API)** + For the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API, specify the following. 1. For `MedicalTranscriptionJobName`, specify a name that is unique in your AWS account. 1. For `LanguageCode`, specify the language code that corresponds to the language spoken in your audio file and the language of your vocabulary filter. 1. For the `MediaFileUri` parameter of the `Media` object, specify the name of the audio file that you want to transcribe. 1. For `Specialty`, specify the medical specialty of the clinician speaking in the audio file. 1. For `Type`, specify whether the audio file is a conversation or a dictation. 1. For `OutputBucketName`, specify the Amazon S3 bucket to store the transcription results. 1. For the `Settings` object, specify the following. 1. `VocabularyName` – the name of your custom vocabulary. The following request uses the AWS SDK for Python (Boto3) to start a batch transcription job with a custom vocabulary. ``` from __future__ import print_function import time import boto3 transcribe = boto3.client('transcribe', 'us-west-2') job_name = "my-first-med-transcription-job" job_uri = "s3://amzn-s3-demo-bucket/my-input-files/my-media-file.flac" transcribe.start_medical_transcription_job( MedicalTranscriptionJobName = job_name, Media = { 'MediaFileUri': job_uri }, OutputBucketName = 'amzn-s3-demo-bucket', OutputKey = 'my-output-files/', LanguageCode = 'en-US', Specialty = 'PRIMARYCARE', Type = 'CONVERSATION', Settings = { 'VocabularyName': 'example-med-custom-vocab' } ) while True: status = transcribe.get_medical_transcription_job(MedicalTranscriptionJobName = job_name) if status['MedicalTranscriptionJob']['TranscriptionJobStatus'] in ['COMPLETED', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) ``` # Transcribing a real-time stream using a medical custom vocabulary To improve transcription accuracy in a real-time stream, you can use a custom vocabulary using either HTTP/2 or WebSocket streams. To start an HTTP/2 request, use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API. You can use a custom vocabulary in real-time using either the AWS Management Console, the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API, or by using the WebSocket protocol. ## Transcribing a dictation that is spoken into your Microphone (AWS Management Console) To use the AWS Management Console to transcribe streaming audio of a medical dictation, choose the option to transcribe a medical dictation, start the stream, and begin speaking into the microphone. **To transcribe streaming audio of a medical dictation (AWS Management Console)** 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Real-time transcription**. 1. For **Medical specialty**, choose the medical specialty of the clinician speaking in the stream. 1. For **Audio input type**, choose either **Conversation** or **Dictation**. 1. For **Additional settings**, choose **Custom vocabulary**. 1. For **Vocabulary selection**, choose the custom vocabulary. 1. Choose **Start streaming**. 1. Speak into the microphone. ## Enabling speaker partitioning in an HTTP/2 stream The following is the syntax for the parameters of an HTTP/2 request. ``` POST /medical-stream-transcription HTTP/2 host: transcribestreaming.us-west-2.amazonaws.com authorization: Generated value x-amz-target: com.amazonaws.transcribe.Transcribe.StartMedicalStreamTranscription x-amz-content-sha256: STREAMING-MED-AWS4-HMAC-SHA256-EVENTS x-amz-date: 20220208T235959Z x-amzn-transcribe-session-id: my-first-http2-med-stream x-amzn-transcribe-language-code: en-US x-amzn-transcribe-media-encoding: flac x-amzn-transcribe-sample-rate: 16000 x-amzn-transcribe-vocabulary-name: my-first-med-vocab x-amzn-transcribe-specialty: PRIMARYCARE x-amzn-transcribe-type: CONVERSATION x-amzn-transcribe-show-speaker-label: true Content-type: application/vnd.amazon.eventstream transfer-encoding: chunked ``` Parameter descriptions: + **host**: Update the AWS Region ('us-west-2' in the preceding example) with the AWS Region you are calling. For a list of valid AWS Regions, see [AWS Regions and Endpoints](https://docs.aws.amazon.com/general/latest/gr/rande.html#transcribe_region). + **authorization**: This is a generated field. To learn more about creating a signature, see [Signing AWS requests with Signature Version 4](https://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html). + **x-amz-target**: Don't alter this field; use the content shown in the preceding example. + **x-amz-content-sha256**: This is a generated field. To learn more about calculating a signature, see [Signing AWS requests with Signature Version 4](https://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html). + **x-amz-date**: The date and time the signature is created. The format is YYYYMMDDTHHMMSSZ, where YYYY=year, MM=month, DD=day, HH=hour, MM=minute, SS=seconds, and 'T' and 'Z' are fixed characters. For more information, refer to [Handling Dates in Signature Version 4](https://docs.aws.amazon.com/general/latest/gr/sigv4-date-handling.html). + **x-amzn-transcribe-session-id**: The name for your streaming session. + **x-amzn-transcribe-language-code**: The encoding used for your input audio. Refer to [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) or [Supported languages and language-specific features](supported-languages.md) for a list of valid values. + **x-amzn-transcribe-media-encoding**: The encoding used for your input audio. Valid values are `pcm`, `ogg-opus`, and `flac`. + **x-amzn-transcribe-sample-rate**: The sample rate of the input audio (in Hertz). Amazon Transcribe supports a range from 8,000 Hz to 48,000 Hz. Low-quality audio, such as telephone audio, is typically around 8,000 Hz. High-quality audio typically ranges from 16,000 Hz to 48,000 Hz. Note that the sample rate you specify **must** match that of your audio. + **x-amzn-transcribe-vocabulary-name**: The name of the vocabulary you want to use with your transcription. + **x-amzn-transcribe-specialty**: The medical specialty being transcribed. + **x-amzn-transcribe-type**: Choose whether this is a dictation or a conversation. + **x-amzn-transcribe-show-speaker-label**: To enable diarization, this value must be `true`. + **content-type**: Don't alter this field; use the content shown in the preceding example. ## Enabling speaker partitioning in a WebSocket request To partition speakers in WebSocket streams with the API, use the following format to create a pre-signed URI to start a WebSocket request and set `vocabulary-name` to the name of the custom vocabulary. ``` GET wss://transcribestreaming.us-west-2.amazonaws.com:8443/medical-stream-transcription-websocket ?language-code=en-US &X-Amz-Algorithm=AWS4-HMAC-SHA256 &X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20220208%2Fus-west-2%2Ftranscribe%2Faws4_request &X-Amz-Date=20220208T235959Z &X-Amz-Expires=300 &X-Amz-Security-Token=security-token &X-Amz-Signature=Signature Version 4 signature &X-Amz-SignedHeaders=host &media-encoding=flac &sample-rate=16000 &session-id=sessionId &specialty=medicalSpecialty &type=CONVERSATION &vocabulary-name=vocabularyName &show-speaker-label=boolean ``` # Character set for Amazon Transcribe Medical To use custom vocabularies in Amazon Transcribe Medical, use the following character set. ## English character set For English custom vocabularies, you can use the following characters in the `Phrase` and `SoundsLike` columns: + a - z + A - Z + ' (apostrophe) + - (hyphen) + . (period) You can use the following International Phonetic Alphabet (IPA) characters in the `IPA` column of the vocabulary input file. | Character | Code | Character | Code | | --- | --- | --- | --- | | aʊ | 0061 028A | w | 0077 | | aɪ | 0061 026A | z | 007A | | b | 0062 | æ | 00E6 | | d | 0064 | ð | 00F0 | | eɪ | 0065 026A | ŋ | 014B | | f | 0066 | ɑ | 0251 | | g | 0067 | ɔ | 0254 | | h | 0068 | ɔɪ | 0254 026A | | i | 0069 | ə | 0259 | | j | 006A | ɛ | 025B | | k | 006B | ɝ | 025D | | l | 006C | ɡ | 0261 | | l̩ | 006C 0329 | ɪ | 026A | | m | 006D | ɹ | 0279 | | n | 006E | ʃ | 0283 | | n̩ | 006E 0329 | ʊ | 028A | | oʊ | 006F 028A | ʌ | 028C | | p | 0070 | ʍ | 028D | | s | 0073 | ʒ | 0292 | | t | 0074 | ʤ | 02A4 | | u | 0075 | ʧ | 02A7 | | v | 0076 | θ | 03B8 | # Identifying personal health information (PHI) in a transcription Use *Personal Health Information Identification* to label personal health information (PHI) in your transcription results. By reviewing labels, you can find PHI that could be used to identify a patient. You can identify PHI using either a real-time stream or batch transcription job. You can use your own post-processing to redact the PHI identified in the transcription output. Use Personal Health Information Identification to identify the following types of PHI: + Personal PHI: + Names – Full name or last name and initial + Gender + Age + Phone numbers + Dates (not including the year) that directly relate to the patient + Email addresses + Geographic PHI: + Physical address + Zip code + Name of medical center or practice + Account PHI: + Fax numbers + Social security numbers (SSNs) + Health insurance beneficiary numbers + Account numbers + Certificate or license numbers + Vehicle PHI: + Vehicle identification number (VIN) + License plate number + Other PHI: + Web Uniform Resource Location (URL) + Internet Protocol (IP) address numbers Amazon Transcribe Medical is a Health Insurance Portability and Accountability Act of 1996 (HIPAA) eligible service. For more information, see [Amazon Transcribe Medical](transcribe-medical.md). For information about identifying PHI in an audio file, see [Identifying PHI in an audio file](phi-id-batch.md). For information about identifying PHI in a stream, see [Identifying PHI in a real-time stream](phi-id-stream.md). **Topics** + [Identifying PHI in an audio file](phi-id-batch.md) + [Identifying PHI in a real-time stream](phi-id-stream.md) # Identifying PHI in an audio file Use a batch transcription job to transcribe audio files and identify the personal health information (PHI) within them. When you activate Personal Health Information (PHI) Identification, Amazon Transcribe Medical labels the PHI that it identified in the transcription results. For information about the PHI that Amazon Transcribe Medical can identify, see [Identifying personal health information (PHI) in a transcription](phi-id.md). You can start a batch transcription job using either the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API or the AWS Management Console. ## AWS Management Console To use the AWS Management Console to transcribe a clinician-patient dialogue, create a transcription job and choose **Conversation** for **Audio input type**. **To transcribe an audio file and identify its PHI (AWS Management Console)** 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Transcription jobs**. 1. Choose **Create job**. 1. On the **Specify job details** page, under **Job settings **, specify the following. 1. **Name** – The name of the transcription job that is unique to your AWS account. 1. **Audio input type** – **Conversation** or **Dictation**. 1. For the remaining fields, specify the Amazon S3 location of your audio file and where you want to store the output of your transcription job. 1. Choose **Next**. 1. Under **Audio settings**, choose **PHI Identification**. 1. Choose **Create**. ## API **To transcribe an audio file and identify its PHI using a batch transcription job (API)** + For the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API, specify the following. 1. For `MedicalTranscriptionJobName`, specify a name that is unique to your AWS account. 1. For `LanguageCode`, specify the language code that corresponds to the language spoken in your audio file. 1. For the `MediaFileUri` parameter of the `Media` object, specify the name of the audio file that you want to transcribe. 1. For `Specialty`, specify the medical specialty of the clinician speaking in the audio file as `PRIMARYCARE`. 1. For `Type`, specify either `CONVERSATION` or `DICTATION`. 1. For `OutputBucketName`, specify the Amazon S3 bucket where you want to store the transcription results. The following is an example request that uses the AWS SDK for Python (Boto3) to transcribe an audio file and identify the PHI of a patient. ``` from __future__ import print_function import time import boto3 transcribe = boto3.client('transcribe') job_name = "my-first-transcription-job" job_uri = "s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac" transcribe.start_medical_transcription_job( MedicalTranscriptionJobName = job_name, Media = {'MediaFileUri': job_uri}, LanguageCode = 'en-US', ContentIdentificationType = 'PHI', Specialty = 'PRIMARYCARE', Type = 'type', # Specify 'CONVERSATION' for a medical conversation. Specify 'DICTATION' for a medical dictation. OutputBucketName = 'amzn-s3-demo-bucket' ) while True: status = transcribe.get_medical_transcription_job(MedicalTranscriptionJobName = job_name) if status['MedicalTranscriptionJob']['TranscriptionJobStatus'] in ['COMPLETED', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) ``` The following example code shows the transcription results with patient PHI identified. ``` { "jobName": "my-medical-transcription-job-name", "accountId": "111122223333", "results": { "transcripts": [{ "transcript": "The patient's name is Bertrand." }], "items": [{ "id": 0, "start_time": "0.0", "end_time": "0.37", "alternatives": [{ "confidence": "0.9993", "content": "The" }], "type": "pronunciation" }, { "id": 1, "start_time": "0.37", "end_time": "0.44", "alternatives": [{ "confidence": "0.9981", "content": "patient's" }], "type": "pronunciation" }, { "id": 2, "start_time": "0.44", "end_time": "0.52", "alternatives": [{ "confidence": "1.0", "content": "name" }], "type": "pronunciation" }, { "id": 3, "start_time": "0.52", "end_time": "0.92", "alternatives": [{ "confidence": "1.0", "content": "is" }], "type": "pronunciation" }, { "id": 4, "start_time": "0.92", "end_time": "0.9989", "alternatives": [{ "confidence": "1.0", "content": "Bertrand" }], "type": "pronunciation" }, { "id": 5, "alternatives": [{ "confidence": "0.0", "content": "." }], "type": "punctuation" }], "entities": [{ "content": "Bertrand", "category": "PHI*-Personal*", "startTime": 0.92, "endTime": 1.2, "confidence": 0.9989 }], "audio_segments": [ { "id": 0, "transcript": "The patient's name is Bertrand.", "start_time": "0.0", "end_time": "0.9989", "items": [ 0, 1, 2, 3, 4, 5 ] } ] }, "status": "COMPLETED" } ``` ## AWS CLI **To transcribe an audio file and identify PHI using a batch transcription job (AWS CLI)** + Run the following code. ``` aws transcribe start-medical-transcription-job \ --medical-transcription-job-name my-medical-transcription-job-name\ --language-code en-US \ --media MediaFileUri="s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac" \ --output-bucket-name amzn-s3-demo-bucket \ --specialty PRIMARYCARE \ --type type \ # Choose CONVERSATION to transcribe a medical conversation. Choose DICTATION to transcribe a medical dictation. --content-identification-type PHI ``` # Identifying PHI in a real-time stream You can identify Personal Health Information (PHI) in either HTTP/2 or WebSocket streams. When you activate PHI Identification, Amazon Transcribe Medical labels the PHI that it identifies in the transcription results. For information about the PHI that Amazon Transcribe Medical can identify, see [Identifying personal health information (PHI) in a transcription](phi-id.md). ## Identifying PHI in a dictation that is spoken into your microphone To use the AWS Management Console to transcribe the speech picked up by your microphone and identify any PHI, choose **Dictation** as the audio input type, start the stream, and begin speaking into the microphone on your computer. **To identify PHI in a dictation using the AWS Management Console** 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, choose **Real-time transcription**. 1. For **Audio input type**, choose **Dictation**. 1. For **Additional settings**, choose **PHI identification**. 1. Choose **Start streaming** and speak into the microphone. 1. Choose **Stop streaming** to end the dictation. ## Identifying PHI in an HTTP/2 stream To start an HTTP/2 stream with PHI Identification activated, use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API and specify the following: + For `LanguageCode`, specify the language code for the language spoken in the stream. For US English, specify `en-US`. + For `MediaSampleHertz`, specify the sample rate of the audio. + For `content-identification-type`, specify `PHI`. ## Identifying PHI in a WebSocket stream To a start a WebSocket stream with PHI Identification activated, use the following format to create a presigned URL. ``` GET wss://transcribestreaming.us-west-2.amazonaws.com:8443/medical-stream-transcription-websocket? &X-Amz-Algorithm=AWS4-HMAC-SHA256 &X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20220208%2Fus-west-2%2Ftranscribe%2Faws4_request &X-Amz-Date=20220208T235959Z &X-Amz-Expires=300 &X-Amz-Security-Token=security-token &X-Amz-Signature=Signature Version 4 signature &X-Amz-SignedHeaders=host &language-code=en-US &media-encoding=flac &sample-rate=16000 &specialty=medical-specialty &content-identification-type=PHI ``` Parameter definitions can be found in the [API Reference](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_Reference.html); parameters common to all AWS API operations are listed in the [Common Parameters](https://docs.aws.amazon.com/transcribe/latest/APIReference/CommonParameters.html) section. # Generating alternative transcriptions When you use Amazon Transcribe Medical, you get the transcription that has the highest confidence level. However, you can configure Amazon Transcribe Medical to return additional transcriptions with lower confidence levels. Use alternative transcriptions to see different interpretations of the transcribed audio. For example, in an application that enables a person to review the transcription, you can present the alternative transcriptions for the person to choose from. You can generate alternative transcriptions with the AWS Management Console or the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API. ## AWS Management Console To use the AWS Management Console to generate alternative transcriptions, you enable alternative results when you configure your job. 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Transcription jobs**. 1. Choose **Create job**. 1. On the **Specify job details** page, provide information about your transcription job. 1. Choose **Next**. 1. Enable **Alternative results**. 1. For **Maximum alternatives**, enter an integer value between 2 and 10, for the maximum number of alternative transcriptions you want in the output. 1. Choose **Create**. ## API **To separate text per speaker in an audio file using a batch transcription job (API)** + For the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API, specify the following. 1. For `MedicalTranscriptionJobName`, specify a name that is unique in your AWS account. 1. For `LanguageCode`, specify the language code that corresponds to the language spoken in your audio file and the language of your vocabulary filter. 1. In the `MediaFileUri` parameter of the `Media` object, specify the location of the audio file you want to transcribe. 1. For `Specialty`, specify the medical specialty of the clinician speaking in the audio file. 1. For `Type`, specify whether you're transcribing a medical conversation or a dictation. 1. For `OutputBucketName`, specify the Amazon S3 bucket to store the transcription results. 1. For the `Settings` object, specify the following. 1. `ShowAlternatives` – `true`. 1. `MaxAlternatives` - An integer between 2 and 10 to indicate the number of alternative transcriptions you want in the transcription output. The following request uses the AWS SDK for Python (Boto3) to start a transcription job that generates up to two alternative transcriptions. ``` from __future__ import print_function import time import boto3 transcribe = boto3.client('transcribe', 'us-west-2') job_name = "my-first-transcription-job" job_uri = s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac transcribe.start_medical_transcription_job( MedicalTranscriptionJobName = job_name, Media = { 'MediaFileUri': job_uri }, OutputBucketName = 'amzn-s3-demo-bucket', OutputKey = 'my-output-files/', LanguageCode = 'en-US', Specialty = 'PRIMARYCARE', Type = 'CONVERSATION', Settings = { 'ShowAlternatives': True, 'MaxAlternatives': 2 } ) while True: status = transcribe.get_medical_transcription_job(MedicalTranscriptionJobName = job_name) if status['MedicalTranscriptionJob']['TranscriptionJobStatus'] in ['COMPLETED', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) ``` ## AWS CLI **To transcribe an audio file of a conversation between a primary care clinician and a patient in an audio file (AWS CLI)** + Run the following code. ``` aws transcribe start-transcription-job \ --cli-input-json file://filepath/example-start-command.json ``` The following code shows the contents of `example-start-command.json`. ``` { "MedicalTranscriptionJobName": "my-first-transcription-job", "LanguageCode": "en-US", "Specialty": "PRIMARYCARE", "Type": "CONVERSATION", "OutputBucketName":"amzn-s3-demo-bucket", "Media": { "MediaFileUri": "s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac" }, "Settings":{ "ShowAlternatives": true, "MaxAlternatives": 2 } } ``` # Amazon Transcribe Medical and interface VPC endpoints (AWS PrivateLink) You can establish a private connection between your VPC and Amazon Transcribe Medical by creating an *interface VPC endpoint*. Interface endpoints are powered by [AWS PrivateLink](https://aws.amazon.com/privatelink), a technology that enables you to privately access Amazon Transcribe Medical APIs without an internet gateway, NAT device, VPN connection, or Direct Connect connection. Instances in your VPC don't need public IP addresses to communicate with Amazon Transcribe Medical APIs. Traffic between your VPC and Amazon Transcribe Medical does not leave the Amazon network. Each interface endpoint is represented by one or more [Elastic Network Interfaces](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html) in your subnets. For more information, see [Interface VPC endpoints (AWS PrivateLink)](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html) in the *Amazon VPC User Guide*. ## Considerations for Amazon Transcribe Medical VPC endpoints Before you set up an interface VPC endpoint for Amazon Transcribe Medical, ensure that you review [Interface endpoint properties and limitations](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#vpce-interface-limitations) in the *Amazon VPC User Guide*. Amazon Transcribe Medical supports making calls to all of its API actions from your VPC. ## Creating an interface VPC endpoint for Amazon Transcribe Medical You can create a VPC endpoint for the Amazon Transcribe Medical service using either the AWS Management Console or the AWS CLI. For more information, see [Creating an interface endpoint](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint) in the *Amazon VPC User Guide*. For batch transcription in Amazon Transcribe Medical, create a VPC endpoint using the following service name: + com.amazonaws.*us-west-2*.transcribe For streaming transcription in Amazon Transcribe Medical, create a VPC endpoint using the following service name: + com.amazonaws.*us-west-2*.transcribestreaming If you enable private DNS for the endpoint, you can make API requests to Amazon Transcribe Medical using its default DNS name for the AWS Region, for example, `transcribestreaming.us-east-2.amazonaws.com`. For more information, see [Accessing a service through an interface endpoint](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#access-service-though-endpoint) in the *Amazon VPC User Guide*. ## Creating a VPC endpoint policy for Amazon Transcribe Medical streaming You can attach an endpoint policy to your VPC endpoint that controls access to Amazon Transcribe Medical. The policy specifies the following information: + The principal that can perform actions. + The actions that can be performed. + The resources on which actions can be performed. For more information, see [Controlling access to services with VPC endpoints](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints-access.html) in the *Amazon VPC User Guide*. **Example: VPC endpoint policy for Amazon Transcribe Medical streaming transcription actions** The following is an example of an endpoint policy for streaming transcription in Amazon Transcribe Medical. When attached to an endpoint, this policy grants access to the listed Amazon Transcribe Medical actions for all principals on all resources. ``` { "Statement":[ { "Principal":"*", "Effect":"Allow", "Action":[ "transcribe:StartMedicalStreamTranscription", ], "Resource":"*" } ] } ``` **Example: VPC endpoint policy for Amazon Transcribe Medical batch transcription actions** The following is an example of an endpoint policy for batch transcription in Amazon Transcribe Medical. When attached to an endpoint, this policy grants access to the listed Amazon Transcribe Medical actions for all principals on all resources. ``` { "Statement":[ { "Principal":"*", "Effect":"Allow", "Action":[ "transcribe:StartMedicalTranscriptionJob" ], "Resource":"*" } ] } ``` ## Shared subnets You cannot create, describe, modify, or delete VPC endpoints in subnets that are shared with you. However, you can use the VPC endpoints in subnets that are shared with you. For information about VPC sharing, see [Share your VPC with other accounts](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-sharing.html#vpc-sharing-service-behavior) in the Amazon Virtual Private Cloud guide.