本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。 # AWS HealthScribe 串流 透過 AWS HealthScribe 串流，您可以即時轉錄醫療對話。 AWS HealthScribe 串流是一種以 HTTP2 為基礎的即時雙向服務，在一個頻道上接受音訊串流，並在另一個頻道上提供音訊轉錄。串流完成後， AWS HealthScribe 會分析串流內容，並產生文字記錄 JSON 檔案和臨床備註 JSON 檔案。 若要開始串流，請使用 [StartMedicalScribeStream](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalScribeStream.html) API 操作。此 API 會啟動 HTTP2 型雙向頻道，讓您用來串流音訊事件。 當您啟動串流時，請先在 中指定串流組態`MedicalScribeConfigurationEvent`。此事件包括頻道定義、加密設定和串流後分析設定，例如彙總文字記錄和產生臨床備註的輸出組態。 開始串流音訊後，您可以管理串流，如下所示： + 完成後，若要使用串流後分析開始處理結果，請傳送`MedicalScribeSessionControlEvent`具有 `Type` 的 `END_OF_SESSION`， AWS HealthScribe 會開始分析。 + 若要暫停串流，請完成輸入串流，而不傳送 `MedicalScribeSessionControlEvent`。 + 若要繼續暫停的串流，請使用 `StartMedicalScribeStream` API 操作並指定相同的 `SessionId`。這是您最初啟動串流`SessionId`時使用的 。 **Topics** + [準則和要求](#health-scribe-streaming-requirements) + [ResourceAccessRoleArn 角色許可](#health-scribe-data-access-role) + [Starting AWS HealthScribe 串流轉錄](health-scribe-streaming-setting-up.md) ## 準則和要求 以下是 AWS HealthScribe 串流的指導方針和要求： + 傳送音訊事件之前，您必須先在 中指定串流組態`[MedicalScribeConfigurationEvent](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_MedicalScribeConfigurationEvent.html)`。 + 若要執行串流後分析， `ResourceAccessRoleArn`中的 `MedicalScribeConfigurationEvent` 必須具有正確的許可。如需詳細資訊，請參閱[ResourceAccessRoleArn 角色許可](#health-scribe-data-access-role)。 + 您可以在初始串流建立後 5 小時內繼續工作階段任意次數。 + 您可以跨所有串流請求，透過工作階段串流最多 2 小時的音訊。 + 根據預設， AWS HealthScribe 提供靜態加密，以保護使用 Amazon S3受管金鑰的敏感客戶資料。當您啟動串流時，您可以為第二層加密指定 AWS KMS 金鑰。您的 `ResourceAccessRoleArn` 必須具有使用 AWS KMS 金鑰的許可。如需詳細資訊，請參閱[for AWS HealthScribe 靜態資料加密](health-scribe-encryption.md)。 + 您可以使用 AWS HealthScribe 串流搭配 AWS SDKs，不包括適用於 Python (Boto3) 的 SDK 和適用於 PHP 的 SDK。 + 如果在結束串流後發生`LimitExceededException`例外狀況，您可以重新啟動工作階段，但仍會產生串流後分析。若要重新啟動串流，請使用 [StartMedicalScribeStream](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalScribeStream.html) API，並使用相同的 `SessionID`。然後傳送`MedicalScribeSessionControlEvent`具有 `Type`的 `END_OF_SESSION`， AWS HealthScribe 會開始分析。 ## ResourceAccessRoleArn 角色許可 若要執行串流後分析， `ResourceAccessRoleArn`中的 `MedicalScribeConfigurationEvent` 必須能夠存取您的 Amazon S3 輸出儲存貯體，而且如果您提供它，則必須能夠存取您的 AWS KMS 金鑰。此外，角色的信任政策必須授予`transcribe.streaming.amazonaws.com`服務擔任角色的許可。 以下是授予 Amazon S3 儲存貯體許可和 AWS KMS 金鑰許可的 IAM 政策範例。如需詳細資訊，請參閱[for AWS HealthScribe 靜態資料加密](health-scribe-encryption.md)。 ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Action": [ "s3:PutObject" ], "Resource": [ "arn:aws:s3:::amzn-s3-demo-bucket", "arn:aws:s3:::amzn-s3-demo-bucket/*" ], "Effect": "Allow" }, { "Action": [ "kms:DescribeKey", "kms:Decrypt", "kms:Encrypt", "kms:GenerateDataKey*" ], "Resource": "arn:aws:kms:us-west-2:123456789012:key/1234abcd-12ab-34cd-56ef-123456SAMPLE", "Effect": "Allow" } ] } ``` ------ 信任政策範例如下。 ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "transcribe.streaming.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] } ``` ------ # Starting AWS HealthScribe 串流轉錄 下列程式碼範例示範如何使用 AWS SDKs 設定 AWS HealthScribe 串流轉錄。 **Topics** + [適用於 Java 2.x 的 SDK](#health-scribe-java-stream) + [串流轉錄輸出範例](#health-scribe-streaming-output-example) ## 適用於 Java 2.x 的 SDK 下列範例使用適用於 Java 的 SDK 2.x 來設定串流並提出 [StartMedicalScribeStream](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalScribeStream.html) 請求。 ``` package org.example; import io.reactivex.rxjava3.core.BackpressureStrategy; import io.reactivex.rxjava3.core.Flowable; import org.reactivestreams.Publisher; import org.reactivestreams.Subscriber; import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider; import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider; import software.amazon.awssdk.core.SdkBytes; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.transcribestreaming.TranscribeStreamingAsyncClient; import software.amazon.awssdk.services.transcribestreaming.model.ClinicalNoteGenerationSettings; import software.amazon.awssdk.services.transcribestreaming.model.LanguageCode; import software.amazon.awssdk.services.transcribestreaming.model.MediaEncoding; import software.amazon.awssdk.services.transcribestreaming.model.MedicalScribeInputStream; import software.amazon.awssdk.services.transcribestreaming.model.MedicalScribePostStreamAnalyticsSettings; import software.amazon.awssdk.services.transcribestreaming.model.MedicalScribeSessionControlEventType; import software.amazon.awssdk.services.transcribestreaming.model.MedicalScribeTranscriptEvent; import software.amazon.awssdk.services.transcribestreaming.model.MedicalScribeTranscriptSegment; import software.amazon.awssdk.services.transcribestreaming.model.StartMedicalScribeStreamRequest; import software.amazon.awssdk.services.transcribestreaming.model.StartMedicalScribeStreamResponseHandler; import software.amazon.awssdk.services.transcribestreaming.model.medicalscribeinputstream.DefaultConfigurationEvent; import software.amazon.awssdk.http.nio.netty.NettyNioAsyncHttpClient; import javax.sound.sampled.AudioFormat; import javax.sound.sampled.AudioInputStream; import javax.sound.sampled.AudioSystem; import javax.sound.sampled.DataLine; import javax.sound.sampled.LineUnavailableException; import javax.sound.sampled.TargetDataLine; import java.io.BufferedInputStream; import java.io.IOException; import java.io.InputStream; import java.io.UncheckedIOException; import java.util.Arrays; import java.util.concurrent.CompletableFuture; public class HealthScribeStreamingDemoApp { private static final int CHUNK_SIZE_IN_BYTES = 6400; private static final int SAMPLE_RATE = 16000; private static final Region REGION = Region.US_EAST_1; private static final String sessionId = "1234abcd-12ab-34cd-56ef-123456SAMPLE"; private static final String bucketName = "amzn-s3-demo-bucket"; private static final String resourceAccessRoleArn = "arn:aws:iam::123456789012:role/resource-access-role"; private static TranscribeStreamingAsyncClient client; public static void main(String args[]) { client = TranscribeStreamingAsyncClient.builder() .credentialsProvider(getCredentials()) .httpClientBuilder(NettyNioAsyncHttpClient.builder()) .region(REGION) .build(); try { StartMedicalScribeStreamRequest request = StartMedicalScribeStreamRequest.builder() .languageCode(LanguageCode.EN_US.toString()) .mediaSampleRateHertz(SAMPLE_RATE) .mediaEncoding(MediaEncoding.PCM.toString()) .sessionId(sessionId) .build(); MedicalScribeInputStream endSessionEvent = MedicalScribeInputStream.sessionControlEventBuilder() .type(MedicalScribeSessionControlEventType.END_OF_SESSION) .build(); CompletableFuture result = client.startMedicalScribeStream( request, new AudioStreamPublisher(getStreamFromMic(), getConfigurationEvent(),endSessionEvent), getMedicalScribeResponseHandler()); result.get(); client.close(); } catch (Exception e) { System.err.println("Error occurred: " + e.getMessage()); e.printStackTrace(); } } private static AudioInputStream getStreamFromMic() throws LineUnavailableException { // Signed PCM AudioFormat with 16kHz, 16 bit sample size, mono AudioFormat format = new AudioFormat(SAMPLE_RATE, 16, 1, true, false); DataLine.Info info = new DataLine.Info(TargetDataLine.class, format); if (!AudioSystem.isLineSupported(info)) { System.out.println("Line not supported"); throw new LineUnavailableException("The audio system microphone line is not supported."); } TargetDataLine line = (TargetDataLine) AudioSystem.getLine(info); int bufferSize = (CHUNK_SIZE_IN_BYTES / format.getFrameSize()) * format.getFrameSize(); line.open(format); line.start(); // Create a wrapper class that can be closed when Enter is pressed AudioInputStream audioStream = new AudioInputStream(line); // Start a thread to monitor for Enter key System.out.println("Recording... Press Enter to stop"); Thread monitorThread = new Thread(() -> { try { System.in.read(); line.stop(); line.close(); } catch (IOException e) { e.printStackTrace(); } }); monitorThread.setDaemon(true); // Set as daemon thread so it doesn't prevent JVM shutdown monitorThread.start(); return new AudioInputStream( new BufferedInputStream(new AudioInputStream(line)), format, AudioSystem.NOT_SPECIFIED ); } private static AwsCredentialsProvider getCredentials() { return DefaultCredentialsProvider.create(); } private static StartMedicalScribeStreamResponseHandler getMedicalScribeResponseHandler() { return StartMedicalScribeStreamResponseHandler.builder() .onResponse(r -> { System.out.println("Received Initial response"); }) .onError(Throwable::printStackTrace) .onComplete(() -> { System.out.println("=== All records streamed successfully ==="); }) .subscriber(event -> { if (event instanceof MedicalScribeTranscriptEvent) { MedicalScribeTranscriptSegment segment = ((MedicalScribeTranscriptEvent) event).transcriptSegment(); if (segment != null && segment.content() != null && !segment.content().isEmpty()) { System.out.println(segment.content()); } } }) .build(); } private static DefaultConfigurationEvent getConfigurationEvent() { MedicalScribePostStreamAnalyticsSettings postStreamSettings = MedicalScribePostStreamAnalyticsSettings .builder() .clinicalNoteGenerationSettings( ClinicalNoteGenerationSettings.builder() .outputBucketName(bucketName) .build() ) .build(); return (DefaultConfigurationEvent) MedicalScribeInputStream.configurationEventBuilder() .resourceAccessRoleArn(resourceAccessRoleArn) .postStreamAnalyticsSettings(postStreamSettings) .build(); } private static class AudioStreamPublisher implements Publisher { private final InputStream audioInputStream; private final MedicalScribeInputStream configEvent; private final MedicalScribeInputStream endSessionEvent; private AudioStreamPublisher(AudioInputStream audioInputStream, MedicalScribeInputStream configEvent, MedicalScribeInputStream endSessionEvent) { this.audioInputStream = audioInputStream; this.configEvent = configEvent; this.endSessionEvent = endSessionEvent; } @Override public void subscribe(Subscriber subscriber) { createAudioFlowable() .doOnComplete(() -> { try { audioInputStream.close(); } catch (IOException e) { throw new UncheckedIOException(e); } }) .subscribe(subscriber); } private Flowable createAudioFlowable() { // Start with config event Flowable configFlow = Flowable.just(configEvent); // Create audio chunk flowable Flowable audioFlow = Flowable.create(emitter -> { byte[] buffer = new byte[CHUNK_SIZE_IN_BYTES]; int bytesRead; try { while (!emitter.isCancelled() && (bytesRead = audioInputStream.read(buffer)) > 0) { byte[] audioData = bytesRead < buffer.length ? Arrays.copyOfRange(buffer, 0, bytesRead) : buffer; MedicalScribeInputStream audioEvent = MedicalScribeInputStream.audioEventBuilder() .audioChunk(SdkBytes.fromByteArray(audioData)) .build(); emitter.onNext(audioEvent); } emitter.onComplete(); } catch (IOException e) { emitter.onError(e); } }, BackpressureStrategy.BUFFER); // End with session end event Flowable endFlow = Flowable.just(endSessionEvent); // Concatenate all flows return Flowable.concat(configFlow, audioFlow, endFlow); } } } ``` ## 串流轉錄輸出範例 串流完成後， AWS HealthScribe 會分析串流內容，並產生文字記錄 JSON 檔案和臨床備註 JSON 檔案。下列是每個輸出類型的範例： ### 範例文字記錄輸出 以下是來自串流工作階段的 a AWS HealthScribe 文字記錄檔案範例。 ``` { "Conversation": { "ClinicalInsights": [{ "Attributes": [], "Category": "MEDICAL_CONDITION", "InsightId": "insightUUID1", "InsightType": "ClinicalEntity", "Spans": [{ "BeginCharacterOffset": 12, "Content": "pain", "EndCharacterOffset": 15, "SegmentId": "uuid1" }], "Type": "DX_NAME" }, { "Attributes": [], "Category": "TEST_TREATMENT_PROCEDURE", "InsightId": "insightUUID2", "InsightType": "ClinicalEntity", "Spans": [{ "BeginCharacterOffset": 4, "Content": "mammogram", "EndCharacterOffset": 12, "SegmentId": "uuid2" }], "Type": "TEST_NAME" }, { "Attributes": [], "Category": "TEST_TREATMENT_PROCEDURE", "InsightId": "insightUUID3", "InsightType": "ClinicalEntity", "Spans": [{ "BeginCharacterOffset": 15, "Content": "pap smear", "EndCharacterOffset": 23, "SegmentId": "uuid3" }], "Type": "TEST_NAME" }, { "Attributes": [], "Category": "MEDICATION", "InsightId": "insightUUID4", "InsightType": "ClinicalEntity", "Spans": [{ "BeginCharacterOffset": 28, "Content": "phentermine", "EndCharacterOffset": 38, "SegmentId": "uuid4" }], "Type": "GENERIC_NAME" }, { "Attributes": [{ "AttributeId": "attributeUUID1", "Spans": [{ "BeginCharacterOffset": 38, "Content": "high", "EndCharacterOffset": 41, "SegmentId": "uuid5" }], "Type": "TEST_VALUE" }], "Category": "TEST_TREATMENT_PROCEDURE", "InsightId": "insightUUID5", "InsightType": "ClinicalEntity", "Spans": [{ "BeginCharacterOffset": 14, "Content": "weight", "EndCharacterOffset": 19, "SegmentId": "uuid6" }], "Type": "TEST_NAME" }, { "Attributes": [], "Category": "ANATOMY", "InsightId": "insightUUID6", "InsightType": "ClinicalEntity", "Spans": [{ "BeginCharacterOffset": 60, "Content": "heart", "EndCharacterOffset": 64, "SegmentId": "uuid7" }], "Type": "SYSTEM_ORGAN_SITE" }], "ConversationId": "sampleConversationUUID", "LanguageCode": "en-US", "SessionId": "sampleSessionUUID", "TranscriptItems": [{ "Alternatives": [{ "Confidence": 0.7925, "Content": "Okay" }], "BeginAudioTime": 0.16, "EndAudioTime": 0.6, "Type": "PRONUNCIATION" }, { "Alternatives": [{ "Confidence": 0, "Content": "." }], "BeginAudioTime": 0, "EndAudioTime": 0, "Type": "PUNCTUATION" }, { "Alternatives": [{ "Confidence": 1, "Content": "Good" }], "BeginAudioTime": 0.61, "EndAudioTime": 0.92, "Type": "PRONUNCIATION" }, { "Alternatives": [{ "Confidence": 1, "Content": "afternoon" }], "BeginAudioTime": 0.92, "EndAudioTime": 1.54, "Type": "PRONUNCIATION" }, { "Alternatives": [{ "Confidence": 0, "Content": "." }], "BeginAudioTime": 0, "EndAudioTime": 0, "Type": "PUNCTUATION" }, { "Alternatives": [{ "Confidence": 0.9924, "Content": "You" }], "BeginAudioTime": 1.55, "EndAudioTime": 1.88, "Type": "PRONUNCIATION" }, { "Alternatives": [{ "Confidence": 1, "Content": "lost" }], "BeginAudioTime": 1.88, "EndAudioTime": 2.19, "Type": "PRONUNCIATION" }, { "Alternatives": [{ "Confidence": 1, "Content": "one" }], "BeginAudioTime": 2.19, "EndAudioTime": 2.4, "Type": "PRONUNCIATION" }, { "Alternatives": [{ "Confidence": 1, "Content": "lb" }], "BeginAudioTime": 2.4, "EndAudioTime": 2.97, "Type": "PRONUNCIATION" } ], "TranscriptSegments": [{ "BeginAudioTime": 0.16, "Content": "Okay.", "EndAudioTime": 0.6, "ParticipantDetails": { "ParticipantRole": "CLINICIAN_0" }, "SectionDetails": { "SectionName": "SUBJECTIVE" }, "SegmentId": "uuid1" }, { "BeginAudioTime": 0.61, "Content": "Good afternoon.", "EndAudioTime": 1.54, "ParticipantDetails": { "ParticipantRole": "CLINICIAN_0" }, "SectionDetails": { "SectionName": "OTHER" }, "SegmentId": "uuid2" }, { "BeginAudioTime": 1.55, "Content": "You lost one lb.", "EndAudioTime": 2.97, "ParticipantDetails": { "ParticipantRole": "CLINICIAN_0" }, "SectionDetails": { "SectionName": "SUBJECTIVE" }, "SegmentId": "uuid3" }, { "BeginAudioTime": 2.98, "Content": "Yeah, I think it, uh, do you feel more energy?", "EndAudioTime": 6.95, "ParticipantDetails": { "ParticipantRole": "CLINICIAN_0" }, "SectionDetails": { "SectionName": "SUBJECTIVE" }, "SegmentId": "uuid4" }, { "BeginAudioTime": 6.96, "Content": "Yes.", "EndAudioTime": 7.88, "ParticipantDetails": { "ParticipantRole": "CLINICIAN_0" }, "SectionDetails": { "SectionName": "SUBJECTIVE" }, "SegmentId": "uuid5" }, { "BeginAudioTime": 7.89, "Content": "Uh, how about craving for the carbohydrate or sugar or fat or anything?", "EndAudioTime": 17.93, "ParticipantDetails": { "ParticipantRole": "CLINICIAN_0" }, "SectionDetails": { "SectionName": "SUBJECTIVE" }, "SegmentId": "uuid6" }] } } ``` ### 範例臨床文件輸出 以下是來自串流工作階段的 a AWS HealthScribe 臨床文件洞察檔案範例。 ``` { "ClinicalDocumentation": { "Sections": [ { "SectionName": "CHIEF_COMPLAINT", "Summary": [ { "EvidenceLinks": [ { "SegmentId": "uuid1" }, { "SegmentId": "uuid2" }, { "SegmentId": "uuid3" }, { "SegmentId": "uuid4" }, { "SegmentId": "uuid5" }, { "SegmentId": "uuid6" } ], "SummarizedSegment": "Weight loss." } ] }, { "SectionName": "HISTORY_OF_PRESENT_ILLNESS", "Summary": [ { "EvidenceLinks": [ { "SegmentId": "uuid7" }, { "SegmentId": "uuid8" }, { "SegmentId": "uuid9" }, { "SegmentId": "uuid10" } ], "SummarizedSegment": "The patient is seen today for a follow-up of weight loss." }, { "EvidenceLinks": [ { "SegmentId": "uuid11" }, { "SegmentId": "uuid12" }, { "SegmentId": "uuid13" } ], "SummarizedSegment": "They report feeling more energy and craving carbohydrates, sugar, and fat." }, { "EvidenceLinks": [ { "SegmentId": "uuid14" }, { "SegmentId": "uuid15" }, { "SegmentId": "uuid16" } ], "SummarizedSegment": "The patient is up to date on their mammogram and pap smear." }, { "EvidenceLinks": [ { "SegmentId": "uuid17" }, { "SegmentId": "uuid18" }, { "SegmentId": "uuid19" }, { "SegmentId": "uuid20" } ], "SummarizedSegment": "The patient is taking phentermine and would like to continue." } ] }, { "SectionName": "REVIEW_OF_SYSTEMS", "Summary": [ { "EvidenceLinks": [ { "SegmentId": "uuid21" }, { "SegmentId": "uuid22" } ], "SummarizedSegment": "Patient reports intermittent headaches, occasional chest pains but denies any recent fevers or chills." }, { "EvidenceLinks": [ { "SegmentId": "uuid23" }, { "SegmentId": "uuid24" } ], "SummarizedSegment": "No recent changes in vision, hearing, or any respiratory complaints." } ] }, { "SectionName": "PAST_MEDICAL_HISTORY", "Summary": [ { "EvidenceLinks": [ { "SegmentId": "uuid25" }, { "SegmentId": "uuid26" } ], "SummarizedSegment": "Patient has a history of hypertension and was diagnosed with Type II diabetes 5 years ago." }, { "EvidenceLinks": [ { "SegmentId": "uuid27" }, { "SegmentId": "uuid28" } ], "SummarizedSegment": "Underwent an appendectomy in the early '90s and had a fracture in the left arm during childhood." } ] }, { "SectionName": "ASSESSMENT", "Summary": [ { "EvidenceLinks": [ { "SegmentId": "uuid29" }, { "SegmentId": "uuid30" } ], "SummarizedSegment": "Weight loss" } ] }, { "SectionName": "PLAN", "Summary": [ { "EvidenceLinks": [ { "SegmentId": "uuid31" }, { "SegmentId": "uuid32" }, { "SegmentId": "uuid33" }, { "SegmentId": "uuid34" } ], "SummarizedSegment": "For the condition of Weight loss: The patient was given a 30-day supply of phentermine and was advised to follow up in 30 days." } ] } ], "SessionId": "sampleSessionUUID" } } ```