---
title:
- HTTP Live Streaming (HLS)
author:
- Sanchayan Maity
theme:
- default
classoption:
- aspectratio=169
---

# Who

- Who am I?
  * Embedded Systems background
  * Prefer C, Haskell and Rust
  * Organize and speak at Rust and Haskell meet-ups in Bangalore
- Work?
  * Software Engineer @ [asymptotic](https://asymptotic.io/)
  * Open source consulting firm based out of Bangalore and Toronto
  * Work on low level systems software centred around multimedia
  * GStreamer, PipeWire, PulseAudio
  * Language Polyglots


# Open source contributions

- [GStreamer](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests?scope=all&state=all&author_username=SanchayanMaity)
- [gst-plugins-rs](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests?scope=all&state=all&author_username=SanchayanMaity)
- [PipeWire](https://gitlab.freedesktop.org/pipewire/pipewire/-/merge_requests?scope=all&state=all&author_username=SanchayanMaity)
- [PulseAudio](https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests?scope=all&state=all&author_username=SanchayanMaity)
- [Linux](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?qt=author&q=Sanchayan+Maity)
- [u-boot](https://source.denx.de/u-boot/u-boot/-/commits/master?search=Sanchayan%20Maity)

# Agenda

- Whirlwind tour of GStreamer
- What's HTTP Live Streaming (HLS)
- HLS implementation

# GStreamer

- Multiplatform Pipeline based multimedia framework
- Bindings for various languages
- Allows building complex media processing workflows
- Some applications
  * PiTiVi (Video Editor)
  * amaroK, Banshee, Clementine (audio players)
  * Empathy (VOIP and video conferencing)
  * GstLAL (gravitational wave data analysis)
  * Rygel (DLNA streaming server and renderer)
  * Totem (movie player for the GNOME desktop)


# Simple pipeline

  ```bash
  gst-launch-1.0 videotestsrc ! autovideosink
  gst-launch-1.0 audiotestsrc ! autoaudiosink
  ```

# Media pipeline[^1]

![*_Simple Player_*](simple-player.png){width=80%}

[^1]: [Dynamic Pipelines](https://gstreamer.freedesktop.org/documentation/tutorials/basic/dynamic-pipelines.html?gi-language=c)

# HLS

- HTTP-based adaptive bit-rate streaming communications protocol
- Developed by Apple and released in 2009
- Standardised in [RFC 8216](https://datatracker.ietf.org/doc/html/rfc8216)
- HTTP traffic, unlike UDP-based protocols such as RTP
- Content can be offered from conventional HTTP servers
- Delivered over widely available HTTP-based content delivery networks

# Motivation

- Video on demand content
  * HLS is better than just serving media directly over HTTP
  * Can describe metadata
  * Variants using a bit rate ladder
  - Alternate media renditions
- Live content
  * Reuse the same mechanism
  * Trading off lower cost distribution via CDN at the cost of latency over real time

# Playlist[^2]

```m3u8
   #EXTM3U
   #EXT-X-TARGETDURATION:10
   #EXT-X-VERSION:3
   #EXTINF:9.009,
   http://media.example.com/first.ts
   #EXTINF:9.009,
   http://media.example.com/second.ts
   #EXTINF:3.003,
   http://media.example.com/third.ts
   #EXT-X-ENDLIST
```

[^2]: [RFC 8216](https://datatracker.ietf.org/doc/html/rfc8216#page-50)

# Master playlist[^3]

```m3u8
   #EXTM3U
   #EXT-X-STREAM-INF:BANDWIDTH=1280000,AVERAGE-BANDWIDTH=1000000
   http://example.com/low.m3u8
   #EXT-X-STREAM-INF:BANDWIDTH=2560000,AVERAGE-BANDWIDTH=2000000
   http://example.com/mid.m3u8
   #EXT-X-STREAM-INF:BANDWIDTH=7680000,AVERAGE-BANDWIDTH=6000000
   http://example.com/hi.m3u8
   #EXT-X-STREAM-INF:BANDWIDTH=65000,CODECS="mp4a.40.5"
   http://example.com/audio-only.m3u8
```
[^3]: [RFC 8216](https://datatracker.ietf.org/doc/html/rfc8216#section-8.4)

# Master playlist[^4]

```m3u8
   #EXTM3U
   #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",NAME="English", \
      DEFAULT=YES,AUTOSELECT=YES,LANGUAGE="en", \
      URI="main/english-audio.m3u8"
   #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",NAME="Deutsch", \
      DEFAULT=NO,AUTOSELECT=YES,LANGUAGE="de", \
      URI="main/german-audio.m3u8"
   #EXT-X-STREAM-INF:BANDWIDTH=1280000,CODECS="...",AUDIO="aac"
   low/video-only.m3u8
   #EXT-X-STREAM-INF:BANDWIDTH=2560000,CODECS="...",AUDIO="aac"
   mid/video-only.m3u8
   #EXT-X-STREAM-INF:BANDWIDTH=7680000,CODECS="...",AUDIO="aac"
   hi/video-only.m3u8
   #EXT-X-STREAM-INF:BANDWIDTH=65000,CODECS="mp4a.40.5",AUDIO="aac"
   main/english-audio.m3u8
```
[^4]: [RFC 8216](https://datatracker.ietf.org/doc/html/rfc8216#section-8.6)

# Implementation

- New GStreamer plugin
- Written in Rust
- Uses [m3u8-rs](https://docs.rs/m3u8-rs/latest/m3u8_rs/) and existing [`hlssink3`](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/tree/main/net/hlssink3?ref_type=heads)
- Open MR: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1515

# Demo

Using

- [videojs](https://videojs.com/)
- `python3 -m http.server`
- Sample GStreamer Rust Code[^5]

[^5]: [HLS sample app](https://git.sanchayanmaity.net/sanchayanmaity/gst-hlssink4)

# References

- [RFC 8216](https://datatracker.ietf.org/doc/html/rfc8216)
- [HLS in depth](https://dyte.io/blog/hls-in-depth/)
- [LL-HLS in depth](https://dyte.io/blog/ll-hls-in-depth/)
- [HTTP Live Streaming - A Practical Guide](https://hlsbook.net/)
- [HTTP Live Streaming - Wikipedia](https://en.wikipedia.org/wiki/HTTP_Live_Streaming)

# Questions

- Reach out on
  * Email: sanchayan@sanchayanmaity.net
  * Mastodon: [sanchayanmaity.com](https://sanchayanmaity.com/@sanchayan)
  * Telegram: https://t.me/SanchayanMaity
  * Blog: [sanchayanmaity.net](https://sanchayanmaity.net/)