# Creating redundant HLS manifests Manifests – Redundant HLS manifests When you create an HLS output group in a standard MediaLive channel, you can enable redundant manifests. Redundant manifests allow the downstream system (that reads the manifests) to better handle an output failure from MediaLive. When the redundant manifest feature is enabled, the main manifest for each pipeline references both its own child manifests and the child manifests for the other pipeline. The downstream system finds the path to the child manifests for one pipeline. If there is a problem with that pipeline, then there will be a problem with the child manifests for that pipeline. The downstream system can then refer back to the main manifest to find the child manifest for the other pipeline. In this way, the downstream system can always continue with its processing of the manifest and media. To successfully implement redundant manifests, you must be sure that the downstream system can handle redundant manifests in the ways that are described in the HLS specification. **Note** The information in this section on HLS manifests assumes that you are familiar with the general steps for creating a channel, as described in [Creating a channel from scratch](creating-channel-scratch.md). The key fields in the console that relate to this feature are in the **Manifests and segments** grouping of the **HLS output group** section on the **Create channel** page. To review the step where you complete these fields, see [The procedure](creating-hls-output-group.md#hls-create-procedure). **Topics** + [ # Procedure to set up redundant manifests ](hls-rm-procedure.md) + [ # The media contents of an HLS manifest ](hls-rm-manifests-contents.md) + [ # Rules for most downstream systems ](hls-redundant-manif-most-systems.md) + [ # Rules for Akamai CDNs ](hls-redundant-manif-akamai.md) + [ # Combining redundant manifests with other features ](hls-redundant-manif-combine.md) # Procedure to set up redundant manifests Procedure There are two parts to setting up redundant manifests in MediaLive HLS outputs. You must turn on the feature in the output group. You must also make adjustments in the design of the output names and destination paths (compared to HLS outputs that don't implement redundant manifests). The following field relates specifically to redundant manifests: + **HLS output group – Manifests and Segments – Redundant manifests** field **To set up redundant manifests** 1. Speak to the operator of the downstream system to find out if they support redundant manifests. 1. Read the information in [Fields for the output destination – sending to an HTTP server](hls-destinations-http.md). Manifests are considered to be output from MediaLive. Therefore, the general rules about output destinations apply to redundant manifests. 1. Design the URLs for the two pipelines. There are special requirements for the URLs for the HLS files. Read the appropriate section: + [Rules for most downstream systems](hls-redundant-manif-most-systems.md) + [Rules for Akamai CDNs](hls-redundant-manif-akamai.md) These rules supplement the information in [Fields for the output destination – sending to an HTTP server](hls-destinations-http.md). 1. If you also need custom paths for manifests, make sure you read the information in [How custom paths work](hls-manifests-how-work.md#hls-custom-manifest-paths). You must consider the rules for custom paths when you design the URLs. 1. In the **HLS output group** section, for **Manifest and segments**, for **Redundant manifest**, choose **ENABLED**. This field applies to all outputs in the output group. 1. Complete these fields, following your design: + **Output group – HLS group destination** section + **Output group – HLS settings – CDN** section + **Output group – Location – Directory structure ** + **Output group – Location – Segments per subdirectory** + **HLS outputs – Output settings – Name modifier** + **HLS outputs – Output settings – Segment modifier** + **HLS output group – Location –Base URL Manifest** (if you are also setting up custom paths) + **HLS output group – Location – Base URL Content** (if you are also setting up custom paths) For information about how this feature changes the contents of the HLS manifests, see [The media contents of an HLS manifest](hls-rm-manifests-contents.md). ## The results of this setup Following is information about how redundant manifests work in three failure scenarios. ### Scenario A – Input loss action is to emit output If the input is lost on one of the pipelines and the [**Input loss action** field](hls-other-features.md#hls-resiliency) is set to **EMIT\$1OUTPUT**, MediaLive continues to update the parent and child manifests. From the point of view of the downstream system, there is no change to the parent or child manifests for either pipeline. The content inside the media files is filler content, but that doesn't affect how the downstream system reads the manifests. ### Scenario B – Input loss action is to pause output If the input is lost on one of the pipelines (for example, on pipeline 0) and the **Input loss action** field is set to **PAUSE\$1OUTPUT**, MediaLive does the following: + It removes the listing for the child manifests for pipeline 0. + It sends a request to the child manifest location for pipeline 0 to delete the child manifests. The result for the downstream system that is reading the main manifest on pipeline 0: The system will no longer find a listing for the child manifests for pipeline 0. The system will look in the pipeline 0 main manifest for an alternative child manifest. If it finds the child manifest for pipeline 1, it will switch to reading that child manifest. Downstream systems that are reading the main manifest for pipeline 1 are not affected because these systems are probably reading the child manifests for pipeline 1 (because these appear first in the manifest). ### Scenario C – Pipeline failure It is also possible for a pipeline to fail. This failure isn't the same as an input failure. When a pipeline fails (for example, pipeline 0), the following happens: + Output stops. + The main manifest for pipeline 0 doesn't get deleted. It still contains a listing for the child manifests for pipeline 0. + The child manifests are not updated because no new media files are being produced. The child manifests are *stale*. + The main manifest for pipeline 1 doesn't change. It still contains a listing for the child manifests for pipeline 0 (and for pipeline 1). The result for the downstream system that is reading the main manifest for pipeline 0: The system will find a listing for child manifests for pipeline 0, but that manifest will be stale. If the system can detect that the manifest is stale, it can return to the pipeline 0 main manifest and search for an alternative child manifest. If it finds the child manifest for pipeline 1, it will switch to reading that child manifest. Downstream systems that are reading the main manifest for pipeline 1 are not affected. These systems are presumably reading the child manifests for pipeline 1 (because these appear first in the manifest). **Note** If the downstream system for the HLS output is AWS Elemental MediaStore, you can set up MediaStore to delete stale inputs. See [Components of an object lifecycle policy](https://docs.aws.amazon.com/mediastore/latest/ug/policies-object-lifecycle-components.html). After the child manifest has been deleted, MediaStore falls back to following the "manifest has been deleted" logic of scenario B. # The media contents of an HLS manifest When you set up redundant manifests in an HLS output, MediaLive changes the contents of the manifest. It changes the media information (the video, audio, and captions information) inside the manifests. All of this information appears as `#EXT-X-STREAM-INF` tags. The following sections describe the number of these tags and the contents of these tags in a standard (not redundant) manifest and in a redundant manifest. ## What a standard manifest looks like With a standard channel, there are two pipelines. Each pipeline produces its own set of manifests. Therefore, for pipeline 0, there is one main manifest, one set of child manifests, and one set of media files. Similarly, pipeline 1 has the same set of files. The manifests reference only the files for their own pipeline. The video information in the main manifest for each pipeline might look like this: ``` #EXT-X-STREAM-INF:BANDWIDTH=629107 ... curling-high.m3u8 ``` ## What a redundant manifest looks like When the redundant manifest feature is enabled, each main manifest references the child manifests for its own pipeline and for the other pipeline. This feature doesn’t affect child manifests. Child manifests only reference their own media files. Following is an example of how the video information in the manifest might appear. Assume that the baseFilename for pipeline 0 is *first-curling* and for pipeline 1 it is *other-curling*. The manifest for pipeline 0 might look like this (with the child manifest information for pipeline 0 appearing first): ``` #EXT-X-STREAM-INF:BANDWIDTH=629107 ... first-curling-high.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=629107 ... other-curling-high.m3u8 ``` The video information in the manifest for pipeline 1 might look like this (with the child manifest information for pipeline 1 appearing first): ``` #EXT-X-STREAM-INF:BANDWIDTH=629107 ... other-curling-high.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=629107 ... first-curling-high.m3u8 ``` # Rules for most downstream systems Rules for most systems You can set up redundant manifests in a MediaLive HLS output group so long as the downstream system can work with specific rules. Read this section if you are setting up redundant manifests with any downstream system except Akamai. If your downstream system is an Akamai CDN, see [Rules for Akamai CDNs](hls-redundant-manif-akamai.md). You must make sure that the downstream system can work with the following rules. + MediaLive pushes the files from both pipelines to the same location (protocol/domain/path). + Given that the location is the same, the base filenames for the pipelines must be different. + If you are also implementing [custom manifest paths](hls-manifests-how-work.md#hls-custom-manifest-paths), the URL inside the manifests must be identical. | Field | Rule | | --- | --- | | Protocol/domain/path portion of the two destination URIs (A and B) | Must be identical in both fields. | | Base-filename portion of the two destination URIs (A and B) | Must be different in each field. It *cannot* use [variable identifiers](variable-data-identifiers.md) that include the date or time. | | NameModifier for each output | There is only one instance of this field. Both pipelines use the same value. It *cannot* use [variable identifiers](variable-data-identifiers.md) that include the date or time. | | Segment modifier | There is only one instance of this field. Both pipelines use the same value. It *can* use [variable identifiers](variable-data-identifiers.md) that include the date or time. | | Base URL Manifest A and Base URL Manifest B | These fields apply only if you are also implementing [custom manifest paths](hls-manifests-how-work.md#hls-custom-manifest-paths). Complete both fields. | | Base URL Content A and Base URL Content B | These fields apply only if you are also implementing [custom manifest paths](hls-manifests-how-work.md#hls-custom-manifest-paths). Complete both fields. | # Rules for Akamai CDNs Rules for Akamai You can set up redundant manifests in a MediaLive HLS output group so long as the downstream system can work with specific rules. Read this section if you are setting up redundant manifests with an Akamai CDN. If your downstream system is not an Akamai CDN, see [Rules for most downstream systems](hls-redundant-manif-most-systems.md). You must make sure that the downstream system can work with the following rules. | Field | Rule | | --- | --- | | Protocol/domain/path portion of the two destination URIs (A and B) | Can be different from each other, or can be the same. | | BaseFilename portion of the two destination URIs (A and B) | Can be different from each other, or can be the same. It *cannot* use [variable identifiers](variable-data-identifiers.md) that include the date or time. The combination of the protocol/domain/path and the baseFilename must be unique in A and B. This rule ensures that the output files from the two pipelines don’t overwrite each other. | | Name modifier | There is only one instance of this field. Both pipelines use the same value. It *cannot* use [variable identifiers](variable-data-identifiers.md) that include the date or time. | | Segment modifier | There is only one instance of this field. Both pipelines use the same value. It *can* use [variable identifiers](variable-data-identifiers.md) that include the date or time. | | Base URL Manifest A and Base URL Manifest B | These fields apply only if you are also implementing [custom manifest paths](hls-manifests-how-work.md#hls-custom-manifest-paths). Typically, with Akamai CDNs, you do implement custom manifest paths. Complete both fields. | | Base URL Content A and Base URL Content B | These fields apply only if you are also implementing [custom manifest paths](hls-manifests-how-work.md#hls-custom-manifest-paths). Complete both fields. | # Combining redundant manifests with other features ## Combining redundant manifests and custom path feature When you set up redundant manifests in a MediaLive HLS output group, you can also set up custom paths. Make sure that you follow the rules [for custom paths](hls-custom-paths-rules.md) and for redundant manifests for your downstream system—either an [Akamai CDN](hls-redundant-manif-akamai.md) or [another downstream system](hls-redundant-manif-most-systems.md). ## Combining redundant manifests with audio rendition groups **Note** The information in this section assumes that you are familiar with the manifests for audio rendition groups. For more information, see [Sample manifest](sample-manifest.md). Following is information about the processing that MediaLive performs when you set up redundant manifests in an HLS output group that includes an audio rendition group. MediaLive automatically adjusts the references to the audio rendition groups in the parent manifests. In each pair of lines (for example, the `#EXT-X-STREAM-INF` for the high-resolution video), MediaLive adjusts the name of the rendition groups. In this way, the references to the rendition groups are different for each pipeline, which ensures that when the client player reads the manifest, it chooses the video and the audio from the same pipeline. The `#EXT-X-STREAM` for the video for pipeline 0. Note the value for *AUDIO*: ``` #EXT-X-STREAM-INF:BANDWIDTH=541107,...AUDIO="aac-audio-0", ... ``` The `#EXT-X-STREAM` for the video for pipeline 1. Note the value for *AUDIO*: ``` #EXT-X-STREAM-INF:BANDWIDTH =541107, ...AUDIO="aac-audio-1",... ```