class MetadataExtractionProcessor

Language	Type name
.NET	Amazon.CDK.AWS.KinesisFirehose.MetadataExtractionProcessor
Go	github.com/aws/aws-cdk-go/awscdk/v2/awskinesisfirehose#MetadataExtractionProcessor
Java	software.amazon.awscdk.services.kinesisfirehose.MetadataExtractionProcessor
Python	aws_cdk.aws_kinesisfirehose.MetadataExtractionProcessor
TypeScript (source)	aws-cdk-lib » aws_kinesisfirehose » MetadataExtractionProcessor

Implements IDataProcessor

The data processor for dynamic partitioning with inline parsing.

See also: https://docs.aws.amazon.com/firehose/latest/dev/dynamic-partitioning-partitioning-keys.html

Example

declare const bucket: s3.Bucket;
const s3Destination = new firehose.S3Bucket(bucket, {
  dynamicPartitioning: { enabled: true },
  processors: [
    firehose.MetadataExtractionProcessor.jq16({
      customer_id: '.customer_id',
      device: '.type.device',
      year: '.event_timestamp|strftime("%Y")',
    }),
  ],
  dataOutputPrefix: '!{partitionKeyFromQuery:year}/!{partitionKeyFromQuery:device}/!{partitionKeyFromQuery:customer_id}/',
});
new firehose.DeliveryStream(this, 'DeliveryStream', {
  destination: s3Destination,
});

Initializer

new MetadataExtractionProcessor(options: MetadataExtractionProcessorOptions, keys: string[])

Parameters

• options MetadataExtractionProcessorOptions
• keys string[]

Properties

Name	Type	Description
props	DataProcessorProps	The constructor props of the DataProcessor.

---

props

Type: DataProcessorProps

The constructor props of the DataProcessor.

Methods

Name	Description
bind(scope, options)	Binds this processor to a destination of a delivery stream.
static jq16(query)	Creates the inline parsing configuration with JQ 1.6 engine.

---

bind(scope, options)

public bind(scope: Construct, options: DataProcessorBindOptions): DataProcessorConfig

Parameters

• scope Construct
• options DataProcessorBindOptions

Returns

• DataProcessorConfig

Binds this processor to a destination of a delivery stream.

Implementers should use this method to grant processor invocation permissions to the provided stream and return the necessary configuration to register as a processor.

---

static jq16(query)

public static jq16(query: { [string]: string }): MetadataExtractionProcessor

Parameters

• query { [string]: string } — A map of partition key to jq expression.

Returns

• MetadataExtractionProcessor

Creates the inline parsing configuration with JQ 1.6 engine.