# Logging and monitoring in MediaPackage Logging and monitoring Monitoring is an important part of maintaining the reliability, availability, and performance of MediaPackage and your other AWS solutions. AWS provides the following monitoring tools to watch MediaPackage, report when something is wrong, and take automatic actions when appropriate: + *Amazon CloudWatch* monitors your AWS resources and the applications that you run on AWS in real-time. You can collect and track metrics, create customized dashboards, and set alarms that notify you or take actions when a specified metric reaches a threshold that you specify. For example, you can have CloudWatch track CPU usage or other metrics of your Amazon EC2 instances and automatically launch new instances when needed. For more information, see the [Amazon CloudWatch User Guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/). + *AWS CloudTrail* captures API calls and related events made by or on behalf of your AWS account and delivers the log files to an Amazon S3 bucket that you specify. You can identify which users and accounts called AWS, the source IP address from which the calls were made, and when the calls occurred. For more information, see the [AWS CloudTrail User Guide](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/). **Topics** + [ # Monitoring AWS Elemental MediaPackage with Amazon CloudWatch metrics ](monitoring-cloudwatch.md) + [ # Monitoring AWS Elemental MediaPackage with EventBridge events ](monitoring-eventbridge-events.md) + [ # Logging AWS Elemental MediaPackage API calls with AWS CloudTrail ](logging-using-cloudtrail.md) + [ # Access logging ](access-logging.md) + [ # Monitoring manifest update time in AWS Elemental MediaPackage ](monitoring-manifest-last-updated.md) + [ # MediaPackage response headers ](response-headers.md) # Monitoring AWS Elemental MediaPackage with Amazon CloudWatch metrics Monitoring with CloudWatch metrics You can monitor MediaPackage using CloudWatch, which collects raw data and processes it into readable, near real-time metrics. These statistics are kept for 15 months, so that you can access historical information and gain a better perspective on how your web application or service is performing. You can also set alarms that watch for certain thresholds, and send notifications or take actions when those thresholds are met. For more information, see the [Amazon CloudWatch User Guide](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/). ------ #### [ MediaPackage console ] MediaPackage displays metrics throughout the console. **To view metrics using the MediaPackage console** 1. Open the MediaPackage console at [https://console.aws.amazon.com/mediapackage/](https://console.aws.amazon.com/mediapackage/). 1. Navigate to the appropriate page to view metrics: + For metrics on all channel groups, go to the **Channel groups** page. + For metrics on all channels and origin endpoints associated with your channel group in the AWS Region, go to the channel group's details page. + For metrics on a specific channel and all of its origin endpoints, go to the channel's details page. + For metrics on a specific origin endpoint, go to the origin endpoint's details page. 1. (Optional) To refine the metrics view, choose **Open in CloudWatch**. ------ #### [ CloudWatch console ] Metrics are grouped first by the service namespace, and then by the various dimension combinations within each namespace. **To view metrics using the CloudWatch console** 1. Sign in to the AWS Management Console and open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/). 1. In the navigation pane, choose **Metrics**. 1. Under **All metrics**, choose the **AWS/MediaPackage** namespace. 1. Choose the metric dimension to view the metrics (for example, choose `channel` to view metrics per channel). ------ #### [ AWS CLI ] **To view metrics using the AWS CLI** At a command prompt, enter the following command: + ``` aws cloudwatch list-metrics --namespace "AWS/MediaPackage" ``` ------ ## MediaPackage live content metrics Live content metrics The `AWS/MediaPackage` namespace includes the following metrics for live content. MediaPackage publishes metrics to CloudWatch every minute, if not sooner. | Metric | Description | | --- | --- | | ChannelMQCS | Segment-level quality score as calculated by MediaPackage for the active input to this channel.Units: NumericValid statistics:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html)Valid dimensions:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | ChannelMQCSSequence | Aggregated quality score for all segments in the sequence for the active input to this channel, as calculated by MediaPackage.Units: NumericValid statistics:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html)Valid dimensions:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | EgressBytes | Number of bytes that MediaPackage successfully sends for each request. If MediaPackage doesn't receive any requests for output in the specified interval, then no data is given. Units: Bytes Valid statistics: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html)Valid dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | EgressRequestCount | Number of content requests that MediaPackage receives. If MediaPackage doesn't receive any requests for output in the specified interval, then no data is given. Units: Count Valid statistics:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html)Valid dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | EgressResponseTime | The time that it takes MediaPackage to process each output request. If MediaPackage doesn't receive any requests for output in the specified interval, then no data is given. Units: Milliseconds Valid statistics:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html)Valid dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | IngressBytes | Number of bytes of content that MediaPackage receives for each input request. If MediaPackage doesn't receive any requests for input in the specified interval, then no data is given. Units: Bytes Valid statistics:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html)Valid dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | IngressMQCS | Segment-level quality score as communicated by AWS Elemental MediaLive for this input.Units: NumericValid statistics:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html)Valid dimensions:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | IngressMQCSSequence | Aggregated quality scores for all segments in the sequence, as communicated by AWS Elemental MediaLive.Units: NumericValid statistics:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html)Valid dimensions:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | IngressRequestCount | Number of input requests that MediaPackage receives. If MediaPackage doesn't receive any requests for input in the specified interval, then no data is given. Units: Count Valid statistics:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html)Valid dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | IngressResponseTime | The time that it takes MediaPackage to process each input request. If MediaPackage doesn't receive any requests for input in the specified interval, then no data is given. Units: Milliseconds Valid statistics:[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html)Valid dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | ## MediaPackage live dimensions You can filter the `AWS/MediaPackage` data using the following dimensions. | Dimension | Description | | --- | --- | | No Dimension | Metrics are aggregated and shown for all channels, endpoints, or status codes. | | `CDNAuthorizationStatus` | Value: `Authorized` or `Unauthorized` Can be used in combination with `ChannelGroup`, `Channel`, `OriginEndpoint`, and `CDNAuthorizationStatusDetails` to show metrics for the CDN authorization requests to the specified endpoint. | | `CDNAuthorizationStatusDetails` | Value when `CDNAuthorizationStatus` is `Authorized`: `HeaderSecretMatched` Value when `CDNAuthorizationStatus` is `Unauthorized`: `HeaderSecretMismatched`, `MissingCdnAuthHeaderAndConfiguration`, `MissingCdnAuthHeader`, `MissingCdnAuthConfiguration`, `MissingCdnIdentifierSecretArns`, `MissingSecretsRoleArn`, `SecretsManagerInvalidParameterError`, `SecretsManagerInternalServiceError`, `SecretsManagerThrottlingError`, `MediaPackageSecretsValidationError`, or `OtherErrors`. Can be used in combination with `ChannelGroup`, `Channel`, `OriginEndpoint`, and `CDNAuthorizationStatus` to show the CDN authorization request details for the specified endpoint. | | `Channel` | Metrics are shown only for the specified channel. Value: The auto-generated name of the channel. Can be used alone or with other dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | `ChannelGroup` | Metrics are shown only for the specified channel group. Value: The name of the channel group. Can be used alone or with other dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | `IngestEndpoint` | Metrics are shown only for the specified ingest endpoint on a channel. Value: The auto-generated GUID of the ingest endpoint. Can be used with the following dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | `OriginEndpoint` | Metrics are shown for the specified channel and endpoint combination. Value: The auto-generated name of the endpoint. Must be used with the `channel` dimension. | | `RequestType` | Metrics are shown only for the specified request type. Value: Either `manifest` or `segment`, signifying the type of content being filtered in the metric. Can be used alone or with other dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | `StatusCode` | Metrics are shown for the specified status code range. Value: `2xx`, `3xx`, `4xx`, or `5xx`. Can be used alone or with other dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | | `TrackType` | Metrics are shown for the specified track type. Value: `Video`, `Audio`, or `Subtitle`. Can be used alone or with other dimensions: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-cloudwatch.html) | # Monitoring AWS Elemental MediaPackage with EventBridge events Monitoring with EventBridgeAdded EventBridge events information Added the topic that describes how to use Amazon EventBridge and Amazon Simple Notification Service to notify you of success or failure of harvest jobs. WIth Amazon EventBridge, you can automate your AWS services and respond automatically to system events such as application availability issues or error conditions. AWS services deliver events to EventBridge in near real-time. You can write simple rules to indicate which events are of interest to you, and what automated actions to take when an event matches a rule. The actions that can be automatically triggered include the following: + Invoking an AWS Lambda function + Invoking AWS Systems Manager Run Command + Relaying the event to Amazon Kinesis Data Streams + Activating an AWS Step Functions state machine For MediaPackage V2, you can use EventBridge and Amazon SNS to be automatically notified when a live-to-VOD harvest job succeeds or fails. MediaPackage emits events on a best-effort basis. For more information about creating rules in EventBridge, see [Creating rules that react to events in Amazon EventBridge](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-create-rule.html) in the *Amazon EventBridge User Guide*. The following sections describe MediaPackage V2 event types and how to set up notifications of events. ## MediaPackage V2 events The following topics describe the EventBridge events that MediaPackage V2 emits. ### Harvest job notification events You get harvest job status events when you export a clip from a live stream to create a live-to-VOD asset. MediaPackage creates notifications when the harvest job succeeds or fails. For information about harvest jobs and live-to-VOD assets, see [Creating live-to-VOD assets with MediaPackage](live-to-vod.md). **Example Successful harvest job event** ``` { "id": "8f9b8e72-0b31-e883-f19c-aec84742f3ce", "detail-type": "MediaPackageV2 HarvestJob Notification", "source": "aws.mediapackagev2", "account": "739874020183", "time": "2024-11-07T17:32:36Z", "region": "us-west-2", "resources": [ "arn:aws:mediapackagev2:us-west-2:111122223333:channelGroup/channel_group_name/channel/channel_name/originEndpoint/origin_endpoint_name/harvestJob/MyHarvestJob" ], "detail": { "harvestJob": { "harvestJobName": "MyHarvestJob", "arn": "arn:aws:mediapackagev2:us-west-2:111122223333:channelGroup/channel_group_name/channel/channel_name/originEndpoint/origin_endpoint_name/harvestJob/MyHarvestJob", "status": "COMPLETED", "channelGroupName": "channel_group_name", "channelName": "channel_name", "originEndpointName": "endpoint_name", "scheduleConfiguration": { "startTime": "2024-11-07T17:29:36Z", "endTime": "2024-11-07T17:32:36Z", }, "destination": { "s3Destination": { "bucketName": "amzn-s3-demo-bucket", "destinationPath": "V2Harvests/my-event/" }, }, "harvestedManifests": { "hlsManifests": [ { "manifestName": "examplemanifest.m3u8" } ], "dashManifests": [ { "manifestName": "examplemanifest2.mpd" } ], "lowLatencyHlsManifests": [ { "manifestName": "examplemanifest3.m3u8" }, { "manifestName": "examplemanifest4.m3u8" } ] }, "message": "Your harvest job is complete." } } } ``` **Example Failed harvest job event** ``` { "id": "8f9b8e72-0b31-e883-f19c-aec84742f3ce", "detail-type": "MediaPackageV2 HarvestJob Notification", "source": "aws.mediapackagev2", "account": "111122223333", "time": "2024-11-07T17:32:36Z", "region": "us-west-2", "resources": [ "arn:aws:mediapackagev2:us-west-2:111122223333:channelGroup/channel_group_name/channel/channel_name/originEndpoint/origin_endpoint_name/harvestJob/MyHarvestJob" ], "detail": { "harvestJob": { "harvestJobName": "MyHarvestJob", "arn": "arn:aws:mediapackagev2:us-west-2:111122223333:channelGroup/channel_group_name/channel/channel_name/originEndpoint/origin_endpoint_name/harvestJob/MyHarvestJob", "status": "FAILED", "channelGroupName": "channel_group_name", "channelName": "channel_name", "originEndpointName": "endpoint_name", "scheduleConfiguration": { "startTime": "2024-11-07T17:29:36Z", "endTime": "2024-11-07T17:32:36Z", }, "destination": { "s3Destination": { "bucketName": "amzn-s3-demo-bucket", "destinationPath": "V2Harvests/my-event/" }, }, "harvestedManifests": { "hlsManifests": [ { "manifestName": "manifestexample1.m3u8" } ], "dashManifests": [ { "manifestName": "manifestexample2.mpd" } ], "lowLatencyHlsManifests": [ { "manifestName": "manifestexample3.m3u8" }, { "manifestName": "manifestexample4.m3u8" } ] }, "message": "There was no content available for the specified time window." } } } ``` ## Creating event notifications You can use Amazon EventBridge and Amazon Simple Notification Service (Amazon SNS) to notify you of new events. In EventBridge, the rule describes which events you're notified about. In Amazon SNS, the topic describes what kind of notification you receive. This section provides high-level steps for creating a topic and rule for events from AWS Elemental MediaPackage. For detailed information about topics and rules, see the following: + [Create a topic](https://docs.aws.amazon.com/sns/latest/dg/sns-getting-started.html#CreateTopic) and [Subscribe to a topic](https://docs.aws.amazon.com/sns/latest/dg/sns-getting-started.html#SubscribeTopic) in the *Amazon Simple Notification Service Developer Guide* + [Getting started with Amazon EventBridge](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-get-started.html) in the *Amazon EventBridge User Guide* **To create notifications of EventBridge events** 1. Access [Amazon SNS](https://console.aws.amazon.com/sns/v2/home) and create a topic. Give the topic a descriptive name that you will later recognize. 1. Subscribe to the topic that you just created. Choose what kind of notification you want to receive, and where that notification is sent. For example, for email notifications, choose the **Email** protocol and enter the email address to receive notifications for the endpoint. 1. Access [EventBridge](https://console.aws.amazon.com/eventbridge) and create a rule that uses a **Custom event pattern**. In the pattern preview space, enter the following: ``` { "source": [ "aws.mediapackagev2" ], "detail-type": [ "detail-type from event" ] } ``` For `detail-type`, enter the value for the `detail-type` field from the event: `MediaPackageV2 HarvestJob Notification` 1. Add a target to the rule that you just created. Choose **SNS topic**, and then choose the topic that you created in step 1. 1. Configure the details of the rule, and give it a descriptive name. To start using the rule, make sure it's enabled, and then save it. # Logging AWS Elemental MediaPackage API calls with AWS CloudTrail Logging is available with only live workflows in MediaPackage. MediaPackage is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or an AWS service in MediaPackage. CloudTrail captures all API calls for MediaPackage as events. These include calls from the MediaPackage console and code calls to the MediaPackage API operations. If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for MediaPackage. If you don't configure a trail, you can still view the most recent events in the CloudTrail console in **Event history**. Using the information collected by CloudTrail, you can determine the request that was made to MediaPackage, the IP address from which the request was made, who made the request, when it was made, and additional details. To learn more about CloudTrail, see the [AWS CloudTrail User Guide](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/). ## MediaPackage information in CloudTrail CloudTrail is enabled on your AWS account when you create the account. When activity occurs in MediaPackage, that activity is recorded in a CloudTrail event along with other AWS service events in **Event history**. You can view, search, and download recent events in your account. For more information, see [Viewing events with CloudTrail event history](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events.html). For an ongoing record of events in your account, including events for MediaPackage, create a trail. A *trail* enables CloudTrail to deliver log files to an Amazon S3 bucket. By default, when you create a trail in the console, the trail applies to all AWS Regions. The trail logs events from all AWS Regions in the AWS partition and delivers the log files to the Amazon S3 bucket that you specify. Additionally, you can configure other AWS services to further analyze and act upon the event data collected in CloudTrail logs. For more information, see the following: + [Overview for creating a trail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html) + [CloudTrail supported services and integrations](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-aws-service-specific-topics.html#cloudtrail-aws-service-specific-topics-integrations) + [Configuring Amazon SNS notifications for CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/getting_notifications_top_level.html) + [Receiving CloudTrail log files from multiple regions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/receive-cloudtrail-log-files-from-multiple-regions.html) and [Receiving CloudTrail log files from multiple accounts](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-receive-logs-from-multiple-accounts.html) All MediaPackage actions are logged by CloudTrail and are documented in the [MediaPackage Live API reference ](https://docs.aws.amazon.com/mediapackage/latest/apireference/what-is.html). For example, calls to the `CreateChannel`, `CreateOriginEndpoint`, and `RotateIngestEndpointCredentials` operations generate entries in the CloudTrail log files. Every event or log entry contains information about who generated the request. The identity information helps you determine the following: + Whether the request was made with root user or IAM user credentials + Whether the request was made with temporary security credentials for a role or federated user + Whether the request was made by another AWS service For more information, see the [CloudTrail userIdentity element](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-user-identity.html). ## Understanding MediaPackage log file entries A trail is a configuration that enables delivery of events as log files to an Amazon S3 bucket that you specify. CloudTrail log files contain one or more log entries. An event represents a single request from any source and includes information about the requested action, the date and time of the action, request parameters, and so on. CloudTrail log files aren't an ordered stack trace of the public API calls, so they don't appear in any specific order. The following example shows a CloudTrail log entry that demonstrates the `UpdateChannel` operation: ``` { "eventVersion": "1.08", "userIdentity": { "type": "AssumedRole", "principalId": "ABCDEFGHIJKL123456789", "arn": "arn:aws:sts::444455556666:assumed-role/Admin/testUser", "accountId": "444455556666", "accessKeyId": "AKIAIOSFODNN7EXAMPLE", "sessionContext": { "attributes": { "mfaAuthenticated": "false", "creationDate": "2023-03-17T00:50:58Z" }, "sessionIssuer": { "type": "Role", "principalId": "ABCDEFGHIJKL123456789", "arn": "arn:aws:iam::444455556666:role/Admin", "accountId": "444455556666", "userName": "Admin" } } }, "eventTime": "2023-03-17T00:50:59Z", "eventSource": "mediapackagev2.amazonaws.com", "eventName": "UpdateChannel", "awsRegion": "us-west-2", "sourceIPAddress": "203.0.113.17", "userAgent": "aws-cli/1.18.147 Python/3.6.5 Darwin/17.7.0 botocore/1.10.70", "requestParameters": { "description": "updated cloudtrail description", "id": "cloudtrail-test" }, "responseElements": { "description": "updated cloudtrail description", "hlsIngest": { "ingestEndpoints": [ { "username": "***", "url": "https://mediapackagev2.us-west-2.amazonaws.com/in/v2/8d0ca97840d94b18b37ad292c131bcad/8d0ca97840d94b18b37ad292c131bcad/channel", "password": "***", "id": "8d0ca97840d94b18b37ad292c131bcad" }, { "username": "***", "url": "https://mediapackagev2.us-west-2.amazonaws.com/in/v2/8d0ca97840d94b18b37ad292c131bcad/9c17f979598543b9be24345d63b3ad30/channel", "password": "***", "id": "9c17f979598543b9be24345d63b3ad30" } ] }, "id": "cloudtrail-test", "arn": "arn:aws:mediapackagev2:us-west-2:444455556666:channelGroup/ChannelGroupName/channel/8d0ca97840d94b18b37ad292c131bcad" }, "requestID": "fc158262-025e-11e9-8360-6bff705fbba5", "eventID": "e9016b49-9a0a-4256-b684-eed9bd9073ab", "readOnly": false, "eventType": "AwsApiCall", "managementEvent": true, "recipientAccountId": "444455556666", "eventCategory": "Management" } ``` # Access logging Added support for access logging MediaPackage v2 supports access logging and sending your logs to the log destination that you specify. AWS Elemental MediaPackage v2 provides access logs that capture detailed information about requests sent to your channels. MediaPackage generates two log types: + Ingress access logs are for requests sent to the channel's input endpoints + Egress access logs are for requests sent to the channel's endpoints or packaging group's assets Each log contains information such as the time the request was received, the client's IP address, latencies, request paths, and server responses. You can use these access logs to analyze service performance and troubleshoot issues. They can also help you learn about your customer base and understand your MediaPackage bill. Access logging is an optional feature of MediaPackage that's disabled by default. After you enable access logging, MediaPackage captures the logs and sends them to a destination. Amazon CloudWatch vending log charges apply. For more information, see [CloudWatch pricing](https://aws.amazon.com/cloudwatch/pricing/#Vended_Logs). **Topics** + [ ## Permissions ](#permissions) + [ ## Enable access logging ](#enable-access-logging) + [ ## Manage and disable access logging ](#disable-access-logging) + [ ## Read access logs ](#read-access-logs) ## Permissions MediaPackage uses CloudWatch vended logs to deliver access logging. To deliver access logs, you need permissions to the logging destination that you specify. To see the required permissions for each logging destination, choose from the following AWS services in the *Amazon CloudWatch Logs User Guide*. + [Amazon CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html#AWS-logs-infrastructure-V2-CloudWatchLogs) + [Amazon Simple Storage Service (Amazon S3)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html#AWS-logs-infrastructure-V2-S3) + [Amazon Data Firehose](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html#AWS-logs-infrastructure-V2-Firehose) To create, view, or modify logging configuration in MediaPackage, you must have the required permissions. Your IAM role must include the following minimum permissions to manage access logging in the MediaPackage console. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Sid": "ServiceLevelAccessForLogDelivery", "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:aws:mediapackagev2:us-east-1:123456789012:channelGroup/*" } ] } ``` ------ For more information about permissions to manage access logging, see [Enable logging from AWS services](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html) in the *Amazon CloudWatch Logs User Guide*. ## Enable access logging After you set up permissions to the logging destination, you can enable access logging for MediaPackage. For each log type, you can configure up to 3 log deliveries. **To enable access logs for an existing channel (console)** 1. Open the MediaPackage console at [https://console.aws.amazon.com/mediapackage/](https://console.aws.amazon.com/mediapackage/). 1. Select your channel group from the list of channel groups. 1. Under **Access Logging**, do the following: 1. For **Log deliveries – Egress Access Logs** or **Log deliveries – Ingress Access Logs**, choose **Add**, and then do the following: 1. Choose one of the following logging destinations. + Amazon CloudWatch Logs + Amazon S3 + Firehose **Tip** If you choose Amazon S3 or Firehose, you can deliver your logs to a **Cross account** or **In current account**. To enable cross-account delivery, both AWS accounts must have the required permissions. For more information, see the [Cross-account delivery example](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html#vended-logs-crossaccount-example) in the *Amazon CloudWatch Logs User Guide*. 1. For the **Delivery destination ARN**, choose or enter the ARN. If you don't have one already, follow the prompts to create one. 1. For **Additional settings - *optional***, choose the following: 1. For **Field selection**, select the log fields to include in each log record. 1. For **Output format**, choose the output format for the log. 1. For **Field delimiter**, choose how to separate each log field. 1. (Amazon S3) For **Suffix**, specify the suffix path to partition your data. You can use the following fields: \$1accountid\$1, \$1region\$1, \$1channel\$1group\$1id\$1, \$1yyyy\$1, \$1MM\$1, \$1dd\$1, \$1HH\$1 1. (Amazon S3) For **Hive-compatible**, choose **Enable** if you want to use Hive-compatible S3 paths. 1. To apply your changes to all log types, choose **Apply to all log types**. 1. To create another log destination, repeat steps 3 – 5. 1. Follow the prompts to update your channel group. You can also use the CloudWatch API to enable access logs for your channel groups. **To enable access logs for an existing channel (API)** 1. After you a create a channel group by using either the MediaPackage v2 API or the MediaPackage v2 console, get the Amazon Resource Name (ARN) of the channel group. You can find the ARN from the **Channel groups** page in the MediaPackage console or you can use the [GetChannel](https://docs.aws.amazon.com/mediapackage/latest/APIReference/API_GetChannel.html) API operation. A channel group ARN follows this format: `arn:aws:mediapackagev2:region:123456789012:channelGroup/channelGroupName` 1. Next, use the CloudWatch [PutDeliverySource](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutDeliverySource.html) API operation to create a delivery source for the channel group. 1. Pass the `resourceArn`. 1. For `logType`, specify the type of logs that are collected: + `EGRESS_ACCESS_LOGS` for requests sent to the endpoint or asset + `INGRESS_ACCESS_LOGS` for requests sent to channel **Example PutDeliverySource operation** ``` { "logType": "EGRESS_ACCESS_LOGS", "name": "my-channel-group-source", "resourceArn": "arn:aws:mediapackagev2:region:123456789012:channelGroup/channelGroupName" } ``` 1. Use the [PutDeliveryDestination](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutDeliveryDestination.html) API operation to configure where to store your logs. You can choose either CloudWatch Logs, Amazon S3, or Firehose as the destination. You must specify the ARN of one of the destination options for where your logs will be stored. You can choose the `outputFormat` of the logs to be one of the following: json, plain, w3c, raw, parquet. The following is an example of configuring logs to store in an Amazon S3 bucket and in JSON format. ``` { "deliveryDestinationConfiguration": { "destinationResourceArn": "arn:aws:s3:::amzn-s3-demo-bucket-name" }, "name": "string", "outputFormat": "json", "tags": { "key" : "value" } } ``` **Note** If you're delivering logs cross-account, you must use the `PutDeliveryDestinationPolicy` API to assign an AWS Identity and Access Management (IAM) policy to the destination account. The IAM policy allows delivery from one account to another account. 1. Use the [CreateDelivery](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateDelivery.html) API operation to link the delivery source to the destination that you created in the previous steps. This API operation associates the delivery source with the end destination. ``` { "deliveryDestinationArn": "string", "deliverySourceName": "string", "tags": { "string" : "string" } } ``` ## Manage and disable access logging Follow these procedures to manage or disable access logging for MediaPackage. **To manage access logging for a channel (console)** 1. Open the MediaPackage console at [https://console.aws.amazon.com/mediapackage/](https://console.aws.amazon.com/mediapackage/). 1. Select your channel group from the list of channel groups. 1. In the **Access Logging** section, select the destination and choose **Edit**. 1. Modify the log configuration as needed and then choose **Update**. **Note** You can't modify the logging destination. If you need to change the logging destination, delete the current configuration and create a new logging destination. You can disable access logs for your MediaPackage channel at any time. **To disable access logging for a channel (console)** 1. Open the MediaPackage console at [https://console.aws.amazon.com/mediapackage/](https://console.aws.amazon.com/mediapackage/). 1. Select your channel group from the list of channel groups. 1. In the **Access Logging** section, select the destination to disable and choose **Remove**. 1. Review your changes and then choose **Delete**. ## Read access logs MediaPackage v2 uses ingestion hub to deliver your access logs. If you're using CloudWatch Logs as the destination, you can use CloudWatch Logs Insights to read the access logs. Typical CloudWatch Logs charges apply. For more information, see [Analyzing Log Data with CloudWatch Logs Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html) in the *AWS CloudWatch Logs User Guide*. **Note** Access logs can take a few minutes to appear in your destination. If you don't see the logs, wait a few minutes and try again. ### Access log format The access log files consist of a sequence of formatted log records, where each log record represents one request. The order of the fields within the log can vary. **Example: Ingress access log** ``` { "event_timestamp": "2020-07-13T18:59:56.293656Z", "resource_arn": "arn:aws:mediapackagev2:us-east-1:007862077394:123456789012/my_channel_group", "client_ip": "192.0.2.0/24", "time_to_first_byte": 0.445, "status_code": "200", "received_bytes": 468, "sent_bytes": 2587370, "method": "GET", "request": "https://aabbcc-1.ingest.eeffgg.mediapackagev2.us-east-1.amazonaws.com:443/in/v1/my_channel_group/1/my_channel/manifest.m3u8", "protocol": "HTTP/1.1", "user_agent": "sabr/3.0 Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Safari/528.17", "account": "123456789012", "channel_id": "my_channel", "channel_arn": "arn:aws:mediapackage:us-west-2:111122223333:channels/ExampleChannelID", "domain_name": "aaabbbcccdddee.mediapackage.us-east-1.amazonaws.com", "request_id": "aaaAAA111bbbBBB222cccCCC333dddDDD", "channel_group_id": "my_channel_group", "input_type": "HLS", "input_index": "1" } ``` **Example: Egress access log** ``` { "event_timestamp": "2020-07-13T18:59:56.293656Z", "resource_arn": "arn:aws:mediapackagev2:us-east-1:007862077394:123456789012/my_channel_group", "client_ip": "192.0.2.0/24", "time_to_first_byte": 0.445, "status_code": "200", "received_bytes": 468, "sent_bytes": 2587370, "method": "GET", "request": "https://aabbcc.egress.eeffgg.mediapackagev2.us-east-1.amazonaws.com:443/out/v1/my_channel_group/my_channel/my_endpoint/index.mpd?param1=value1¶m2=value2", "request_query_params": "param1=value1¶m2=value2", "protocol": "HTTP/1.1", "user_agent": "sabr/3.0 Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Safari/528.17", "account": "111122223333", "channel_id": "my_channel", "channel_arn": "arn:aws:mediapackage:us-west-2:123456789012:channels/ExampleChannelID", "domain_name": "aaabbbcccdddee.mediapackage.us-east-1.amazonaws.com", "request_id": "aaaAAA111bbbBBB222cccCCC333dddDDD", "endpoint_id": "my_endpoint", "endpoint_arn": "arn:aws:mediapackage:us-west-2:123456789012:origin_endpoints/ExampleEndpointID", "channel_group_id": "my_channel_group", "manifest_name": "index", "manifest_type": "DASH" } ``` The following list describes the log record fields, in order: **event\$1timestamp** The time of day when the request was received. The value is `ISO-8601` date time and is based on the system clock of the host that served the request. **resource\$1arn** The Amazon Resource Name (ARN) of the channel group that received the request. **client\$1ip** The IP address of the requesting client. **time\$1to\$1first\$1byte** The number of seconds that MediaPackage spent processing your request. This value is measured from the time the last byte of your request was received until the time the first byte of the response was sent. **status\$1code** The numeric HTTP status code of the response. **received\$1bytes** The number of bytes in the request body that the MediaPackage server receives. **sent\$1bytes** The number of bytes in the response body that the MediaPackage server sends. This value often is the same as the value of the `Content-Length` header that's included with server responses. **method** The HTTP request method that was used for the request: DELETE, GET, HEAD, OPTIONS, PATCH, POST, or PUT. **request** The request URL. **protocol** The type of protocol used for the request, such as HTTP. **user\$1agent** A user-agent string that identifies the client that originated the request, enclosed in double quotes. The string consists of one or more product identifiers, product/version. If the string is longer than 8 KB, it is truncated. **account** The AWS account ID of the account that was used to make the request. **channel\$1id** The ID of the channel that received the request. **channel\$1arn** The Amazon Resource Name (ARN) of the channel that received the request. **domain\$1name** The server name indication domain provided by the client during the TLS handshake, enclosed in double quotes. This value is set to `-` if the client doesn't support SNI or the domain doesn't match a certificate and the default certificate is presented to the client. **request\$1id** A string that's generated by MediaPackage to uniquely identify each request. **endpoint\$1id** The ID of the endpoint that received the request. **endpoint\$1arn** The Amazon Resource Name (ARN) of the endpoint that received the request. **input\$1type** The ingest file formats and protocol. **input\$1index** The input source index. **manifest\$1name** The name of the egress request manifest request. Remains empty for segment egress request. **manifest\$1type** The type of the egress request manifest request. Remains empty for segment egress request. **request\$1query\$1params** The manifest filtering query parameter names and values. ### Examples The following are queries that you can use to read MediaPackage debug log data. **Example View the HTTP status code responses for a channel** Use this query to view the responses by HTTP status code for a channel. You can use this to view HTTP error code responses to help you to troubleshoot issues. ``` fields @timestamp, @message | filter channelId like my-channel | stats count() by statusCode ``` **Example Get the number of requests per endpoint on a channel** ``` fields @timestamp, @message | filter channelId like my-channel | stats count() by endpointId ``` # Monitoring manifest update time in AWS Elemental MediaPackage Manifest header MediaPackage added manifest header for low-latency HLS manifests. MediaPackage playback responses include the following custom headers that indicate when MediaPackage last modified the manifest in non-dynamic ad insertion workflows. These headers are helpful when troubleshooting issues related to stale manifests. **X-Amzn-Mediapackage-Manifest-Last-Part** This is only provided on requests for low-latency HLS manifests, and is the highest part sequence number in the manifest. **X-MediaPackage-Manifest-Last-Sequence** This is the highest segment sequence number in the manifest. For HLS and CMAF, this is the highest segment number in the media playlist. See the following section for [manifest example](#manifest-last-updated). **X-MediaPackage-Manifest-Last-Updated** The epoch timestamp in milliseconds when MediaPackage generates the segment referred to in `X-MediaPackage-Manifest-Last-Sequence`. ## HLS manifest example ### HLS manifest MediaPackage determines the `X-MediaPackage-Manifest-Last-Sequence` value from the last segment in the manifest. For example, in the following manifest `index_1_3.ts` is the highest segment sequence number, so the value of `X-MediaPackage-Manifest-Last-Sequence` is `3`. The value of `X-MediaPackage-Manifest-Last-Updated` corresponds to the epoch timestamp in milliseconds when MediaPackage generates the last segment in the manifest. ``` #EXTM3U #EXT-X-VERSION:3 #EXT-X-TARGETDURATION:8 #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:7.500, index_1_0.ts?m=1583172400 #EXTINF:7.500, index_1_1.ts?m=1583172400 #EXTINF:7.500, index_1_2.ts?m=1583172400 #EXTINF:7.500, index_1_3.ts?m=1583172400 #EXT-X-ENDLIST ``` # MediaPackage response headers Response headers Added section that explains MediaPackage response headers. Use the following AWS Elemental MediaPackage response headers to help you build your workflows. For more information about response headers associated with manifest monitoring, see [Monitoring manifest update time](https://docs.aws.amazon.com/mediapackage/latest/userguide/monitoring-manifest-last-updated.html). **** | Header name | Value | Description | | --- | --- | --- | | `CMSD-Static` | For accepted key/value pairs, see [CMSD headers from AWS Elemental MediaPackage](cmsd.md). | A variety of Common Media Server Data (CMSD) information about the manifests and segment response objects. | | `X-Amzn-Mediapackage-Active-Input` | `1` or `2` | The active input pipeline when MediaPackage serves a given media segment. | | `X-Amzn-Mediapackage-Channel-Id` | The channel-name value in the API. | Use separately from, or in addition to, `X-Amzn-MediaPackage-Channel-UniqueId`, to identify a given channel in the CDN logs. Channel names are unique only within a given region. | | `X-Amzn-Mediapackage-Channel-UniqueId` | The unique identifier of the channel. | Use separately from, or in addition to, `X-Amzn-MediaPackage-Channel-Id`, to identify a given channel in the CDN logs. Channel names are unique only within a given region. Using `X-Amzn-Mediapackage-Channel-UniqueId` is also helpful for support requests. | | `X-Amzn-Mediapackage-Endpoint-Id` | The manifest-name value in the API. | Use separately from, or in addition to, `X-Amzn-Mediapackage-Endpoint-UniqueId`, to identify a given endpoint in the CDN logs. Endpoint names are unique only within a given channel and region. | | `X-Amzn-Mediapackage-Endpoint-UniqueId` | The unique identifier of the endpoint. | Use separately from, or in addition to, `X-Amzn-MediaPackage-Endpoint-Id`, to identify a given endpoint in the CDN logs. Endpoint names are unique only within a given channel and region. Using `X-Amzn-Mediapackage-Endpoint-UniqueId` is also helpful for support requests. | | `X-Amzn-RequestId` | The unique identifier of the request. | Equivalent to `X-Amzn-MediaPackage-Request-Id` in MediaPackage V1. Using `X-Amzn-RequestId` is helpful for support requests. |