本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。 # Apache Flink 設定 Managed Service for Apache Flink 是 Apache Flink 框架的實作。Managed Service for Apache Flink 使用本節所述的預設值。其中一些值可由 Managed Service for Apache Flink 應用程式在程式碼中設定，其他則無法變更。 使用本節中的連結，進一步了解 Apache flink 設定和可修改的設定。 **Topics** + [Apache Flink 組態](#apache-flink-configuration) + [狀態後端](#reference-defaults-state-backend) + [檢查點](#reference-defaults-checkpoint) + [儲存點](#reference-defaults-savepoint) + [堆積大小](#reference-defaults-heap) + [緩衝區消脹](#reference-defaults-buffer-debloating) + [可修改的 Flink 組態屬性](reference-modifiable-settings.md) + [程式設計 Flink 組態屬性](programmatic-configuration.md) + [檢視設定的 Flink 屬性](viewing-modifiable-settings.md) ## Apache Flink 組態 Managed Service for Apache Flink 提供預設的 Flink 組態，其中包含大多數屬性的 Apache Flink 建議值，少數一些基於常用應用程式設定檔。如需 Flink 組態的詳細資訊，請參閱[組態](https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/)。服務提供的預設組態適用於大多數應用程式。不過，若要調整 Flink 組態屬性，以改善具有高平行處理、高記憶體和狀態用量的特定應用程式效能，或在 Apache Flink 中啟用新的偵錯功能，您可以透過請求支援案例來變更特定屬性。如需詳細資訊，請參閱 [AWS 支援中心](https://console.aws.amazon.com/support/home#/)。您可以使用 [Apache Flink 儀表板](https://docs.aws.amazon.com/managed-flink/latest/java/how-dashboard.html)檢查應用程式的目前組態。 ## 狀態後端 Managed Service for Apache Flink 將暫時性資料儲存在狀態後端。Managed Service for Apache Flink 使用 **RocksDBStateBackend**。呼叫 `setStateBackend` 來設定不同的後端沒有任何效果。 我們在狀態後端啟用以下功能： + 增量狀態後端快照 + 非同步狀態後端快照 + 檢查點本機復原 如需狀態後端的詳細資訊，請參閱 Apache Flink 文件中的[狀態後端](https://nightlies.apache.org/flink/flink-docs-release-1.19/ops/state/state_backends.html)。 ## 檢查點 Managed Service for Apache Flink 使用具有下列值的預設檢查點組態。其中一些值可以使用 [CheckpointConfiguration](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_CheckpointConfiguration.html) 進行變更。您必須`CheckpointConfiguration.ConfigurationType`將 `CUSTOM` Managed Service for Apache Flink 設定為 ，才能使用修改後的檢查點值。 **** | 設定 | 可以修改嗎？ | 方法 | 預設值 | | --- | --- | --- | --- | | CheckpointingEnabled | 可修改 | [建立應用程式](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_CreateApplication.html) [更新應用程式](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_UpdateApplication.html) [CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/AWS_KinesisAnalyticsV2.html) | True | | CheckpointInterval | 可修改 | [建立應用程式](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_CreateApplication.html) [更新應用程式](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_UpdateApplication.html) [CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/AWS_KinesisAnalyticsV2.html) | 60000 | | MinPauseBetweenCheckpoints | 可修改 | [建立應用程式](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_CreateApplication.html) [更新應用程式](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_UpdateApplication.html) [CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/AWS_KinesisAnalyticsV2.html) | 5000 | | 未對齊的檢查點 | 可修改 | [支援案例](https://console.aws.amazon.com/support/home#/) | False | | 並行檢查點的數量 | 不可修改 | N/A | 1 | | 檢查點模式 | 不可修改 | N/A | 恰好一次 | | 檢查點保留政策 | 不可修改 | N/A | 失敗時 | | 檢查點逾時 | 不可修改 | N/A | 60 分鐘 | | 保留的最大檢查點 | 不可修改 | N/A | 1 | | 檢查點和儲存點位置 | 不可修改 | N/A | 我們將持久的檢查點和儲存點資料儲存到服務擁有的 S3 儲存貯體。 | ## 儲存點 依預設，從儲存點還原時，恢復操作會嘗試將儲存點的所有狀態映射回您要還原的程式。如果您捨棄某個運算子，依預設，從具有對應於遺失運算子之資料的儲存點還原將會失敗。您可以透過將應用程式 [FlinkRunConfiguration](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_FlinkRunConfiguration.html) 的 *AllowNonRestoredState* 參數設定為 `true`，以允許操作成功。這將允許恢復操作跳過無法對應至新程式的狀態。 如需詳細資訊，請參閱 [Apache Flink 文件](https://nightlies.apache.org/flink/flink-docs-release-1.15/)中的[允許非還原的狀態](https://nightlies.apache.org/flink/flink-docs-release-1.15/ops/state/savepoints.html#allowing-non-restored-state)。 ## 堆積大小 Managed Service for Apache Flink 會為每個 KPU 分配 3 GiB 的 JVM 堆積，並為原生程式碼配置保留 1 GiB。如需增加應用程式容量的相關資訊，請參閱[在 Managed Service for Apache Flink 中實作應用程式擴展](how-scaling.md)。 如需 JVM 堆積大小的詳細資訊，請參閱 [Apache Flink 文件](https://nightlies.apache.org/flink/flink-docs-release-1.15/)中的[組態](https://nightlies.apache.org/flink/flink-docs-release-1.15/ops/config.html)。 ## 緩衝區消脹 緩衝區消脹可以幫助應用程式處理高背壓。如果應用程式遇到檢查點/儲存點失敗，啟用此功能可能會很有用。要做到這一點，可請求[支援案例](https://console.aws.amazon.com/support/home#/)。 如需詳細資訊，請參閱 [Apache Flink 文件](https://nightlies.apache.org/flink/flink-docs-release-1.15/)中的[緩衝區消脹機制](https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/memory/network_mem_tuning/#the-buffer-debloating-mechanism)。 # 可修改的 Flink 組態屬性 以下是您可以使用[支援案例](https://console.aws.amazon.com/support/home#/)修改的 Flink 組態設定。您可以一次修改多個屬性，也可以透過指定應用程式前綴來同時修改多個應用程式。如果您想要修改此清單以外的其他 Flink 組態屬性，請在您的案例中指定確切的屬性。 ## 重新啟動策略 從 Flink 1.19 及更新版本，預設會使用`exponential-delay`重新啟動策略。根據預設，所有先前的版本都會使用`fixed-delay`重新啟動策略。 `restart-strategy:` `restart-strategy.fixed-delay.delay:` `restart-strategy.exponential-delay.backoff-muliplier:` `restart-strategy.exponential-delay.initial-backoff:` `restart-strategy.exponential-delay.jitter-factor:` `restart-strategy.exponential-delay.reset-backoff-threshold:` ## 檢查點和狀態後端 `state.backend.type:` `state.backend.fs.memory-threshold:` `state.backend.incremental:` ## 檢查點 `execution.checkpointing.unaligned:` `execution.checkpointing.interval-during-backlog:` ## RocksDB 原生指標 RocksDB 原生指標不會運送到 CloudWatch。啟用後，您可以從 Flink 儀表板或使用自訂工具從 Flink REST API 存取這些指標。 Managed Service for Apache Flink 可讓客戶使用 [CreateApplicationPresignedUrl](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_CreateApplicationPresignedUrl.html) API，以唯讀模式存取最新的 Flink [REST API](https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/ops/rest_api/) (或您正在使用的受支援版本)。此 API 由 Flink 自己的儀表板使用，但也可以由自訂監控工具使用。 `state.backend.rocksdb.metrics.actual-delayed-write-rate:` `state.backend.rocksdb.metrics.background-errors:` `state.backend.rocksdb.metrics.block-cache-capacity:` `state.backend.rocksdb.metrics.block-cache-pinned-usage:` `state.backend.rocksdb.metrics.block-cache-usage:` `state.backend.rocksdb.metrics.column-family-as-variable:` `state.backend.rocksdb.metrics.compaction-pending:` `state.backend.rocksdb.metrics.cur-size-active-mem-table:` `state.backend.rocksdb.metrics.cur-size-all-mem-tables:` `state.backend.rocksdb.metrics.estimate-live-data-size:` `state.backend.rocksdb.metrics.estimate-num-keys:` `state.backend.rocksdb.metrics.estimate-pending-compaction-bytes:` `state.backend.rocksdb.metrics.estimate-table-readers-mem:` `state.backend.rocksdb.metrics.is-write-stopped:` `state.backend.rocksdb.metrics.mem-table-flush-pending:` `state.backend.rocksdb.metrics.num-deletes-active-mem-table:` `state.backend.rocksdb.metrics.num-deletes-imm-mem-tables:` `state.backend.rocksdb.metrics.num-entries-active-mem-table:` `state.backend.rocksdb.metrics.num-entries-imm-mem-tables:` `state.backend.rocksdb.metrics.num-immutable-mem-table:` `state.backend.rocksdb.metrics.num-live-versions:` `state.backend.rocksdb.metrics.num-running-compactions:` `state.backend.rocksdb.metrics.num-running-flushes:` `state.backend.rocksdb.metrics.num-snapshots:` `state.backend.rocksdb.metrics.size-all-mem-tables:` ## RocksDB 選項 `state.backend.rocksdb.compaction.style:` `state.backend.rocksdb.memory.partitioned-index-filters:` `state.backend.rocksdb.thread.num:` ## 進階狀態後端選項 `state.storage.fs.memory-threshold:` ## 完整 TaskManager 選項 `task.cancellation.timeout:` `taskmanager.jvm-exit-on-oom:` `taskmanager.numberOfTaskSlots:` `taskmanager.slot.timeout:` `taskmanager.memory.network.fraction:` `taskmanager.memory.network.max:` `taskmanager.network.request-backoff.initial:` `taskmanager.network.request-backoff.max:` `taskmanager.network.memory.buffer-debloat.enabled:` `taskmanager.network.memory.buffer-debloat.period:` `taskmanager.network.memory.buffer-debloat.samples:` `taskmanager.network.memory.buffer-debloat.threshold-percentages:` ## 記憶體組態 `taskmanager.memory.jvm-metaspace.size:` `taskmanager.memory.jvm-overhead.fraction:` `taskmanager.memory.jvm-overhead.max:` `taskmanager.memory.managed.consumer-weights:` `taskmanager.memory.managed.fraction:` `taskmanager.memory.network.fraction:` `taskmanager.memory.network.max:` `taskmanager.memory.segment-size:` `taskmanager.memory.task.off-heap.size:` ## RPC / Akka `akka.ask.timeout:` `akka.client.timeout:` `akka.framesize:` `akka.lookup.timeout:` `akka.tcp.timeout:` ## 用戶端 `client.timeout:` ## 進階叢集選項 `cluster.intercept-user-system-exit:` `cluster.processes.halt-on-fatal-error:` ## 檔案系統組態 `fs.s3.connection.maximum:` `fs.s3a.connection.maximum:` `fs.s3a.threads.max:` `s3.upload.max.concurrent.uploads:` ## 進階容錯能力選項 `heartbeat.timeout:` `jobmanager.execution.failover-strategy:` ## 記憶體組態 `jobmanager.memory.heap.size:` ## 指標 `metrics.latency.interval:` ## REST 端點和用戶端的進階選項 `rest.flamegraph.enabled:` `rest.server.numThreads:` ## 進階 SSL 安全選項 `security.ssl.internal.handshake-timeout:` ## 進階排程選項 `slot.request.timeout:` ## Flink Web UI 的進階選項 `web.timeout:` # 程式設計 Flink 組態屬性 以下是您可以直接在應用程式程式碼中修改的 Flink 組態屬性。從 MSF 2.2 開始，如果您嘗試修改應用程式程式碼中此頁面未列出的屬性，則會擲回例外狀況。 **注意** 這些屬性與透過 AWS 支援案例修改的設定不同。如需 TaskManager 記憶體、狀態後端、RocksDB 調校和重新啟動策略等基礎設施層級設定，請參閱 [可修改的 Flink 組態屬性](reference-modifiable-settings.md)。 ## 管道組態 `execution.runtime-mode` `pipeline.auto-generate-uids` `pipeline.auto-watermark-interval` `pipeline.cached-files` `pipeline.classpaths` `pipeline.closure-cleaner-level` `pipeline.force-avro` `pipeline.force-kryo` `pipeline.generic-types` `pipeline.global-job-parameters` `pipeline.jars` `pipeline.jobvertex-parallelism-overrides` `pipeline.max-parallelism` `pipeline.name` `pipeline.object-reuse` `pipeline.operator-chaining.chain-operators-with-different-max-parallelism` `pipeline.operator-chaining.enabled` `pipeline.serialization-config` `pipeline.vertex-description-mode` `pipeline.vertex-name-include-index-prefix` `pipeline.watermark-alignment.allow-unaligned-source-splits` ## Python API `python.execution-mode` `python.operator-chaining.enabled` `python.job-options` `python.internal.archives-key-map` `python.internal.files-key-map` `python.internal.requirements-file-key` ## 資料表 API/SQL `table.exec.async-lookup.buffer-capacity` `table.exec.async-lookup.key-ordered-enabled` `table.exec.async-lookup.output-mode` `table.exec.async-lookup.timeout` `table.exec.async-ml-predict.max-concurrent-operations` `table.exec.async-ml-predict.output-mode` `table.exec.async-ml-predict.timeout` `table.exec.async-scalar.max-concurrent-operations` `table.exec.async-scalar.max-attempts` `table.exec.async-scalar.retry-delay` `table.exec.async-scalar.retry-strategy` `table.exec.async-scalar.timeout` `table.exec.async-state.enabled` `table.exec.async-table.max-concurrent-operations` `table.exec.async-table.max-retries` `table.exec.async-table.retry-delay` `table.exec.async-table.retry-strategy` `table.exec.async-table.timeout` `table.exec.async-vector-search.max-concurrent-operations` `table.exec.async-vector-search.output-mode` `table.exec.async-vector-search.timeout` `table.exec.deduplicate.insert-update-after-sensitive-enabled` `table.exec.deduplicate.mini-batch.compact-changes-enabled` `table.exec.delta-join.cache-enabled` `table.exec.disabled-operators` `table.exec.iceberg.use-v2-sink` `table.exec.interval-join.min-cleanup-interval` `table.exec.legacy-cast-behaviour` `table.exec.local-hash-agg.adaptive.distinct-value-rate-threshold` `table.exec.local-hash-agg.adaptive.enabled` `table.exec.local-hash-agg.adaptive.sampling-threshold` `table.exec.mini-batch.allow-latency` `table.exec.mini-batch.enabled` `table.exec.mini-batch.size` `table.exec.operator-fusion-codegen.enabled` `table.exec.simplify-operator-name-enabled` `table.exec.sink.keyed-shuffle` `table.exec.sink.nested-constraint-enforcer` `table.exec.sink.not-null-enforcer` `table.exec.sink.rowtime-inserter` `table.exec.sink.type-length-enforcer` `table.exec.sink.upsert-materialize` `table.exec.sink.upsert-materialize-strategy.adaptive.threshold.high` `table.exec.sink.upsert-materialize-strategy.adaptive.threshold.low` `table.exec.sink.upsert-materialize-strategy.type` `table.exec.sort.async-merge-enabled` `table.exec.sort.default-limit` `table.exec.sort.max-num-file-handles` `table.exec.source.cdc-events-duplicate` `table.exec.source.idle-timeout` `table.exec.spill-compression.block-size` `table.exec.spill-compression.enabled` `table.exec.state.ttl` `table.exec.uid.format` `table.exec.uid.generation` `table.exec.unbounded-over.version` `table.exec.window-agg.buffer-size-limit` `table.optimizer.agg-phase-strategy` `table.optimizer.adaptive-broadcast-join.strategy` `table.optimizer.bushy-join-reorder-threshold` `table.optimizer.delta-join.strategy` `table.optimizer.distinct-agg.split.bucket-num` `table.optimizer.distinct-agg.split.enabled` `table.optimizer.dynamic-filtering.enabled` `table.optimizer.incremental-agg-enabled` `table.optimizer.join-reorder-enabled` `table.optimizer.multi-join.enabled` `table.optimizer.multiple-input-enabled` `table.optimizer.non-deterministic-update.strategy` `table.optimizer.reuse-optimize-block-with-digest-enabled` `table.optimizer.reuse-sink-enabled` `table.optimizer.reuse-source-enabled` `table.optimizer.reuse-sub-plan-enabled` `table.optimizer.runtime-filter.enabled` `table.optimizer.skewed-join-optimization.skewed-factor` `table.optimizer.skewed-join-optimization.skewed-threshold` `table.optimizer.skewed-join-optimization.strategy` `table.optimizer.source.report-statistics-enabled` `table.optimizer.union-all-as-breakpoint-enabled` `table.builtin-catalog-name` `table.builtin-database-name` `table.catalog-modification.listeners` `table.column-expansion-strategy` `table.display.max-column-width` `table.dml-sync` `table.dynamic-table-options.enabled` `table.generated-code.max-length` `table.legacy-nested-row-nullability` `table.local-time-zone` `table.plan.compile.catalog-objects` `table.plan.force-recompile` `table.plan.restore.catalog-objects` `table.rtas-ctas.atomicity-enabled` `table.sql-dialect` # 檢視設定的 Flink 屬性 您可以透過 Apache Flink 儀表板檢視您自己設定或請求透過[支援案例](https://support.console.aws.amazon.com/support/home#/)修改的 Apache Flink 屬性，並遵循下列步驟進行操作： 1. 前往 Flink 儀表板 1. 在左側導覽窗格中選擇**作業管理員**。 1. 選擇**組態**以檢視 Flink 屬性的清單。