The Protocol Powering the Streaming Wars
Watching videos is one of the most ubiquitous activities on the internet, but few understand the technology behind it. It would be nice to understand a little of whats happening under the hood, and I'd like to start with a look at HLS (HTTP Live Streaming), a streaming protocol developed by Apple in 2009 that is used by the majority of video streaming sites today.
A World Without HLS
In a world without HLS, there would be a lot of buffering. Not only during the video, but the time it takes to initally load the video once you click play would be excruciatingly slow (scaling with video length). This of course is dependant on the internet speed of the user, but because most countries have slow or flaky internet, watching videos online would be a luxury few could afford.
There would be no dynamic changing of video quality while watching, which you might not think is a big deal, but having to change the stream's quality constantly and then re-start download every single time your internet is flaky would be a nightmare given what consumers have come to expect.
If this world sounds familiar, its because it is, we used to live it! Internet videos used to be a lot less accessible and much more of an ordeal than they are today, and we have Adaptive Bitrate Streaming to thank for that.
Adaptive Bitrate Streaming with HLS
Note that HLS is not the only protocol in this space, the other popular option is MPEG-DASH. Both are examples of Adaptive Bitrate Streaming protocols. Here are a couple of tricks up their sleeves that make modern video streaming possible:
1. Fragmented Stream
Imagine the wait time for every video if you had to download the entire clip before the player would start playing the video. That might sound fine for a 30 second cat video, but imagine a Netflix movie! One of the great innovations in the video streaming world was the idea of playing a video without having to download the whole video to a user's device.
To experiment with this feature, I created a little archive of my own videos, and below is a screenshot of the network responses that occur when hitting play on one of the videos.
The initial .m3u8 file downloaded is an M3U playlist for the video. An M3U (MP3 URL) playlist is a file that describes a list of media files, including where they can be found (their URL). HLS works by first segmenting the video into .ts files, which are Video Transport Stream files used for compressed video data. A sequence of .ts files numbered in ascending order represent the full video. The .m3u8 file lists .ts files and where they can be found, so if the client has the .m3u8 file they'll be able to reconstruct the whole video!
You've probably noticed when watching Youtube videos that the seek bar has an indicator for how much of the video has already been downloaded ahead of the current timestamp. This feature is noticable when the internet connection goes out while watching a video, and the playback stops abruptly once the seeker reaches the furthest point downloaded.
This "pre-fetching" of video is done with Adaptive Bitrate Streaming, where .ts files ahead of the current timestamp are constantly queued up. An additional side-effect is that if users frequently open videos and then exit out of them without watching the whole video, or maybe skip around different timestamps, the entire video file need not be downloaded, saving plenty of bandwidth!
2. Different Bitrates
I mentioned earlier that one of the benefits of HLS was video quality dynamically changing as the video is viewed. This works via HLS inspecting the bandwidth on the user's device, which is why HLS is not the only streaming protocol that exists. When Apple created HLS, they built out support for all the Apple devices for the Safari web browser and Apple TV, but HLS has been adopted widely across the board. HLS supports intelligent switching of streams based on bandwidth changes, meaning the video won't stop playing when a user's internet slows down, instead a lower quality .ts file is downloaded so the playback is un-interrupted.
The example .m3u8 file shown above is just one of the various .m3u8 files a video will have in HLS. The reality is that there is a master .m3u8 playlist, which has links to many .m3u8 files each at a different bitrates (or video stream quality). HLS intelligently switches between the .m3u8 file to download from, and in turn downloads the .ts files from that .m3u8 playlist.
You can see the various bitrates in the titles of the m3u8 files, for example 0.4Mbps which corresponds to the 480x270 video, versus the 6.5Mbps video which has a resolution of 1280x720.
Demystifying Streaming
I wanted this post to be a brief non-technical introduction to HLS, so I won't go into how the different components actually work to make it all possible. I'll leave some of that magic intact 😉 (or if you're still curious, the apple docs do a good job explaining).
I hope this brief intro was enough to get you curious the next time you watch a TV show on your Mac in Safari, or a Youtube video on your Android device! The full list of usages and supported players can be found here.