View a markdown version of this page

擷取應用程式遙測 - Amazon OpenSearch Service

本文為英文版的機器翻譯版本,如內容有任何歧義或不一致之處,概以英文版為準。

擷取應用程式遙測

若要在 Amazon OpenSearch Service 中使用可觀測性功能,您需要擷取應用程式追蹤、日誌和指標。此頁面涵蓋設定 OpenTelemetry Collector 和 OpenSearch Ingestion 管道,以處理遙測資料並將其路由至 OpenSearch 和 Amazon Managed Service for Prometheus。

設定 OpenTelemetry 收集器

OpenTelemetry (OTel) 收集器是所有應用程式遙測的進入點。它會透過 OTLP 接收資料,並將追蹤和日誌路由至 OpenSearch Ingestion,同時將指標傳送至 Prometheus。

您可以使用下列其中一種方法設定 OTel 收集器:

OTel Collector 會使用 SigV4 身分驗證將追蹤和日誌匯出至 OpenSearch Ingestion 端點,並使用 Prometheus 遠端寫入匯出程式將指標匯出至 Amazon Managed Service for Prometheus。OpenSearch Ingestion 會處理處理、擴充和路由至 OpenSearch。

使用 OpenSearch Ingestion 的 OTel 收集器組態

下列範例組態使用 SigV4 身分驗證,將追蹤和日誌匯出至 OpenSearch 擷取端點,並將指標匯出至 Prometheus:

extensions: sigv4auth: region: us-west-2 service: osis exporters: otlphttp/osis-traces: traces_endpoint: ${OSIS_ENDPOINT}/v1/traces auth: { authenticator: sigv4auth } compression: none otlphttp/osis-logs: logs_endpoint: ${OSIS_ENDPOINT}/v1/logs auth: { authenticator: sigv4auth } compression: none # Amazon Managed Service for Prometheus via Prometheus Remote Write with SigV4 auth prometheusremotewrite/amp: endpoint: "https://aps-workspaces.region.amazonaws.com/workspaces/workspace-id/api/v1/remote_write" auth: authenticator: sigv4auth service: extensions: [sigv4auth] pipelines: traces: receivers: [otlp] processors: [batch] exporters: [otlphttp/osis-traces] logs: receivers: [otlp] processors: [batch] exporters: [otlphttp/osis-logs] metrics: receivers: [otlp] processors: [batch] exporters: [prometheusremotewrite/amp]
注意

管道 ARN 上的 IAM 主體傳送資料需求osis:Ingestaps:RemoteWrite許可。

設定 OpenSearch 擷取管道

OpenSearch Ingestion (或自我管理的資料準備程式) 會從 OTel 收集器接收遙測,並處理它以進行應用程式效能監控 (APM)。

管道架構

管道會在下列階段處理遙測資料:

  1. 進入管道會接收所有遙測,並將日誌和追蹤路由至個別的子管道。

  2. 日誌管道會使用log-analytics-plain索引類型將日誌資料寫入 OpenSearch。

  3. 追蹤管道會分佈到原始儲存管道和服務映射管道。

  4. 原始追蹤管道程序會跨越otel_traces處理器,並將它們存放在trace-analytics-plain-raw索引類型中。

  5. 服務映射管道使用otel_apm_service_map處理器來產生拓撲和 RED (速率、錯誤、持續時間) 指標。它會透過遠端寫入寫入 OpenSearch 和 Prometheus。

管道組態

下列範例顯示 OpenSearch Ingestion 的完整管道組態,涵蓋所有可觀測性訊號類型 – 日誌、追蹤和指標。您可以包含所有管道,或僅包含與您使用案例相關的管道。使用您的資訊取代預留位置的值

version: '2' # Main OTLP pipeline - receives all telemetry and routes by signal type otlp-pipeline: source: otlp: logs_path: '/pipeline-name/v1/logs' traces_path: '/pipeline-name/v1/traces' metrics_path: '/pipeline-name/v1/metrics' route: - logs: 'getEventType() == "LOG"' - traces: 'getEventType() == "TRACE"' - metrics: 'getEventType() == "METRIC"' processor: [] sink: - pipeline: name: otel-logs-pipeline routes: - logs - pipeline: name: otel-traces-pipeline routes: - traces - pipeline: name: otel-metrics-pipeline routes: - metrics # Log processing pipeline otel-logs-pipeline: source: pipeline: name: otlp-pipeline processor: - copy_values: entries: - from_key: "time" to_key: "@timestamp" sink: - opensearch: hosts: - 'https://opensearch-endpoint' index_type: log-analytics-plain aws: serverless: false region: 'region' sts_role_arn: "arn:aws:iam::account-id:role/pipeline-role" # Trace fan-out pipeline otel-traces-pipeline: source: pipeline: name: otlp-pipeline processor: [] sink: - pipeline: name: traces-raw-pipeline routes: [] - pipeline: name: service-map-pipeline routes: [] # Raw trace storage pipeline traces-raw-pipeline: source: pipeline: name: otel-traces-pipeline processor: - otel_traces: sink: - opensearch: hosts: - 'https://opensearch-endpoint' index_type: trace-analytics-plain-raw aws: serverless: false region: 'region' sts_role_arn: "arn:aws:iam::account-id:role/pipeline-role" # Service map generation pipeline (APM) service-map-pipeline: source: pipeline: name: otel-traces-pipeline processor: - otel_apm_service_map: group_by_attributes: - telemetry.sdk.language # Add any resource attribute to group by window_duration: 30s route: - otel_apm_service_map_route: 'getEventType() == "SERVICE_MAP"' - service_processed_metrics: 'getEventType() == "METRIC"' sink: - opensearch: hosts: - 'https://opensearch-endpoint' aws: serverless: false region: 'region' sts_role_arn: "arn:aws:iam::account-id:role/pipeline-role" routes: - otel_apm_service_map_route index_type: otel-v2-apm-service-map - prometheus: url: 'https://aps-workspaces.region.amazonaws.com/workspaces/workspace-id/api/v1/remote_write' aws: region: 'region' routes: - service_processed_metrics # Metrics processing pipeline otel-metrics-pipeline: source: pipeline: name: otlp-pipeline processor: - otel_metrics: sink: - prometheus: url: 'https://aps-workspaces.region.amazonaws.com/workspaces/workspace-id/api/v1/remote_write' aws: region: 'region'

驗證擷取

設定 OTel 收集器和管道之後,請確認遙測資料是否正確流動。

  • 驗證 OpenSearch 索引 – 確認您的網域中存在下列索引:otel-v1-apm-span-*otel-v2-apm-service-maplogs-otel-v1-*

  • 驗證 Prometheus 目標 – 確認 Prometheus 遠端寫入目標正在從服務映射管道接收指標。

  • OpenSearch UI 中驗證 - 導覽至可觀測性,然後進行應用程式監控以確認您的服務出現。

後續步驟

確認擷取遙測資料後,請探索下列主題: