What's new in Low-Latency HLS

Highlight

Apple pushes Low-Latency HLS from beta to iOS 14, tvOS 14, and macOS, and replaces HTTP/2 Push with Blocking Preload Hints, allowing low-latency live broadcasts to retain HLS’s bit rate switching, FairPlay, fMP4 CMAF, and CDN expansion capabilities.

Core Content

When doing live video, the delay usually comes from a simple problem: the client has to wait for the new media clip to appear, the server and CDN have to send it to the edge node, and the player starts the request again. Low-Latency HLS in 2019 has given direction, but it relies on HTTP/2 Push to push new clips to clients as playlists are updated. The feedback Apple received this year was clear: many content distribution links are not suitable for Push, and ad-supported content is especially difficult to access.

The focus of this session is to turn Low-Latency HLS into a deployable protocol. It came out of beta in 2020 and entered iOS 14, tvOS 14, and macOS that year. The experimental capabilities that originally required special qualifications have become HLS capabilities that can be directly used by native apps, and continue to support existing features such as code rate switching, FairPlay Streaming, and fMP4 CMAF.

The key change at the protocol level is to replace HTTP/2 Push with Blocking Preload Hints. The client requests the next part in advance, and the server waits until the content is ready before returning. The request is initiated by the client, and the CDN edge node can easily trigger cache fill, without the need for extra round trips between the edge node and the origin site for Push. For live broadcast platforms, this change puts low-latency capabilities back into the common CDN model.

Apple has also simultaneously tightened its playlist update model. Rendition Reports are no longer selected separately through on-demand commands, but are carried in every playlist update to avoid generating a large number of different URLs for the same update. Delta Updates can carry the latest date-range tags, long DVR windows and dense metadata without having to retransmit the entire list. Part and Rendition Report have added gap signaling so that the client can better handle encoding interruptions.

Detailed Content

Low-Latency HLS end of beta

(00:17) Roger Pantos opened with a deployment conclusion: Low-Latency HLS is out of beta and available in iOS 14, tvOS 14, and macOS that year. It belongs to a set of low-latency extensions to the HLS protocol. The goal is to continue using the HLS ecosystem with a stream delay of less than two seconds, rather than replacing the existing player stack.

Low-Latency HLS availability
- iOS 14
- tvOS 14
- macOS in 2020

Supported HLS features
- bit-rate switching
- FairPlay Streaming
- fMP4 CMAF
- native application playback
- no entitlement required

Key points:

• iOS 14、tvOS 14、macOS in 2020are the available platforms clearly listed in this presentation.
• bit-rate switchingIndicates that low-latency live broadcast still retains multi-bitrate adaptive capabilities.
• FairPlay StreamingExplain that protected content does not have to waive DRM (Digital Rights Management).
• fMP4 CMAFAllow content production links to continue to use fragmented MP4 and CMAF (Common Media Application Format).
• no entitlement requiredRemove the trial threshold in 2019, and ordinary native apps can access it.

Specification enters HLS RFC draft

(00:45) Rules for Low-Latency HLS are added to the second revised draft of the HLS RFC on ietf.org. Apple also added two appendices: Low-Latency Server Configuration Profile and CDN tune-in algorithm. Developers no longer need to just look at the ad-hoc beta page, but can check for device support as per the HLS specification and Apple’s authoring spec.

HLS RFC second revision draft
- Low-Latency protocol rules
- Low-Latency Server Configuration Profile appendix
- CDN tune-in algorithm appendix

Apple developer documentation
- HLS authoring specification
- Apple device support information

Key points:

• Low-Latency protocol rulesEntering the draft HLS RFC represents these rules becoming part of the core HLS specification.
• Server Configuration ProfileServer-side configuration helps origin, packager and CDN work according to the same set of constraints.
• CDN tune-in algorithmExplain how the client cooperates with the CDN when entering the live stream to reduce the wait in the broadcast path.
• HLS authoring specificationIs your entry point to check the range of supported Apple devices.

Blocking Preload Hints replace HTTP/2 Push

(01:41) In order to reduce segment delay, the original low-latency design uses HTTP/2 Push to send new segments together with playlist update. Feedback shows that Push is not suitable for many content distribution methods, nor is it suitable for many ad-supported streams. The new approach is called Blocking Preload Hints: the client requests the next part in advance, and the server returns after it is available.

Old delivery path
1. Server creates a new segment.
2. Playlist update becomes available.
3. HTTP/2 Push sends the segment with that playlist update.

Updated delivery path
1. Client asks for the next part before it is ready.
2. Server holds the request.
3. Server responds when the part is ready.
4. The client-driven request triggers CDN cache fill.

Key points:

• Old delivery pathRelying on HTTP/2 Push requires CDN and server links to support Push behavior.
• Client asks for the next part before it is readyIt is the core action of Blocking Preload Hints.
• Server holds the requestCorresponding to blocking behavior, the server does not return an empty result immediately.
• triggers CDN cache fillIt is a CDN-friendly point emphasized by Apple that edge nodes can naturally fill the cache after the request is issued from the client.
• Speech still requires the CDN to support HTTP/2, but it no longer requires the CDN to support Push.

Rendition Reports will be distributed with playlist update instead.

(03:13) The CDN team reported that if the client can request a specific rendition report, the same playlist update will correspond to a large number of different URLs. As the number of URL combinations increases, the cache hit rate decreases. Apple removes the report delivery directive and instead carries all Rendition Reports in each playlist update.

Before
playlist update + requested rendition report set
=> many URLs for the same update
=> lower CDN cache efficiency

Now
playlist update
=> all Rendition Reports included
=> one cacheable update shape

Key points:

• requested rendition report setis the source of URL combination bloat caused by the old scheme.
• many URLs for the same updateThis will cause the CDN to cache the same update as multiple objects.
• all Rendition Reports includedSacrifice a small amount of response body size in exchange for a more stable cache shape.
• This change directly serves the CDN efficiency of large-scale live broadcasts.

Delta Updates supports date-range tag

(03:40) Many live streams have long DVR windows and frequently use date-range tags to represent program, commercial, or event metadata. The full playlist will get longer and longer. Apple adds a date-range tag processing method to Playlist Delta Updates so that updates only carry the latest date-range tag.

Long DVR playlist
- old media segments
- current media segments
- many date-range tags

Playlist Delta Update
- recent playlist changes
- the most recent date-range tag

Key points:

• Long DVR playlistDescribes the full playlist under the long lookback window.
• many date-range tagsis a common burden on metadata-intensive streams.
• Playlist Delta UpdateOnly increments are transmitted, reducing repeated transmissions between the client and CDN.
• the most recent date-range tagThe changes from this lecture solve the problem of carrying date-range metadata in incremental updates.

gap signaling covers Part and Rendition Report

(04:00) Low-latency live broadcasts are more sensitive to encoding interruptions. Apple added new attributes to the Part and Rendition Report tags to use gap signaling to indicate the presence of gaps in the stream. Clients can skip unavailable content based on these signals, reducing the risk of getting stuck on missing segments.

Gap signaling targets
- Part tags
- Rendition Report tags

Client behavior enabled by the signal
- detect encoded outages
- avoid waiting on unavailable low-latency parts
- continue recovery using rendition information

Key points:

• Part tagsCorresponds to the partial segment level in low-latency streams.
• Rendition Report tagsCorresponds to synchronization information between different renditions.
• encoded outagesis a failure scenario explicitly mentioned in the speech.
• continue recovery using rendition informationSpecify the value of the gap signal to restore the path on the client side instead of just logging the error.

HLS toolchain and reference server updates

(04:17) Apple updated the Low-Latency server reference implementation. Last year’s set of tools can generate low-latency streams; the 2020 version adds fragmented MPEG-4 packaging, making the output compatible with CMAF. Web server settings are also simplified: in the past, you had to configure your own web server and then connect PHP scripts. Now you can run a Go script and implement delivery directives and HTTP/2 web server at the same time.

Updated Low-Latency HLS tools
- fragmented MPEG-4 packaging for CMAF compatibility
- Go script for delivery directives
- HTTP/2 web server in the same package
- included in the regular HLS tools download

Key points:

• fragmented MPEG-4 packagingCorresponds to the options added in late 2019 in the speech.
• CMAF compatibilityMake it easier for the same media package to enter standardized distribution links.
• Go scriptReduce the steps to build a reference server locally.
• delivery directivesandHTTP/2 web serverPut them in the same package to facilitate developers to reproduce the experimental flow.
• regular HLS tools downloadIndicates the return of low-latency tools to the unified download entrance.

Core Takeaways

1. Make a low-latency live sports player

• What to do: Superimpose scores, barrages or real-time statistics next to the live broadcast screen to keep the screen and interactive data close to synchronization.
• Why it’s worth doing: Low-Latency HLS targets a stream delay of less than two seconds and retains bit-rate switching and FairPlay Streaming.
• How ​​to start: First use HLS tools to generate a low-latency stream, and then verify the playback path for iOS 14, tvOS 14 and macOS on the player side.

2. Make a CDN back-to-origin efficiency monitoring panel

• What: Monitor playlist updates, Rendition Reports, part requests, and CDN cache fill hits.
• Why it’s worth doing: Blocking Preload Hints is changed to the client initiating the request, and Apple clearly points out that it can trigger CDN cache fill.
• How ​​to start: Aggregate requests by playlist update, part URI, and rendition dimensions in origin and CDN logs, focusing on observing whether the same update generates too many URL variations.

3. Make a long DVR live broadcast metadata verification tool

• What to do: Check date-range tags in the long lookback window to see if only recent changes are passed through Playlist Delta Updates.
• Why it’s worth doing: The speech mentioned that a large number of date-range tags will affect long DVR windows, and the 2020 agreement added related processing to Delta Updates.
• How ​​to start: Use the HLS playlist parser to read the full playlist and delta playlist, and confirm that the most recent date-range tag appears in the incremental update.

4. Make a low-latency stream failure recovery test stream

• What to do: Artificially create a coding gap and observe whether the player can continue to play according to the gap signaling of Part and Rendition Report.
• Why it’s worth doing: Apple adds gap signaling to Part and Rendition Report tags to handle encoded outages.
• How ​​to start: Prepare multiple rendition live streams, create gaps in some outputs, and record player wait, switch, and resume times.

5. Make a low-latency HLS local experimental environment

• What to do: Use the updated HLS tools to quickly build a local origin and generate fMP4 CMAF compatible low-latency streams.
• Why it’s worth doing: The reference implementation has merged delivery directives and HTTP/2 web server into Go script, and the construction cost is lower than the old PHP solution.
• How ​​to start: Download the regular HLS tools package, use the low-latency tools in it to generate fragmented MPEG-4 content, and then test the client request rhythm through the reference server.

Related Sessions

• Discover HLS Blocking Preload Hints — Continue to talk about the implementation details of the client requesting the next part in advance and the server blocking, waiting and returning.
• Reduce latency with HLS Blocking Playlist Reload — Talk about how playlist reload reduces the waiting time for the client to discover new segments.
• Optimize live streams with HLS Playlist Delta Updates — Talk about incremental playlist updates in scenarios with long DVR windows and large amounts of metadata.
• Adapt ad insertion to Low-Latency HLS — Talk about how server-side ad insertion adapts to low-latency HLS.
• Improve stream authoring with HLS Tools — Talk about how the HLS tool chain generates, inspects and improves live and on-demand streams.