173 lines
5.5 KiB
Markdown
173 lines
5.5 KiB
Markdown
|
---
|
||
|
|
title:
|
||
|
|
- HTTP Live Streaming (HLS)
|
||
|
|
author:
|
||
|
|
- Sanchayan Maity
|
||
|
|
theme:
|
||
|
|
- default
|
||
|
|
classoption:
|
||
|
|
- aspectratio=169
|
||
|
|
---
|
||
|
|
|
||
|
|
# Who
|
||
|
|
|
||
|
|
- Who am I?
|
||
|
|
* Embedded Systems background
|
||
|
|
* Prefer C, Haskell and Rust
|
||
|
|
* Organize and speak at Rust and Haskell meet-ups in Bangalore
|
||
|
|
- Work?
|
||
|
|
* Software Engineer @ [asymptotic](https://asymptotic.io/)
|
||
|
|
* Open source consulting firm based out of Bangalore and Toronto
|
||
|
|
* Work on low level systems software centred around multimedia
|
||
|
|
* GStreamer, PipeWire, PulseAudio
|
||
|
|
* Language Polyglots
|
||
|
|
|
||
|
|
|
||
|
|
# Open source contributions
|
||
|
|
|
||
|
|
- [GStreamer](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests?scope=all&state=all&author_username=SanchayanMaity)
|
||
|
|
- [gst-plugins-rs](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests?scope=all&state=all&author_username=SanchayanMaity)
|
||
|
|
- [PipeWire](https://gitlab.freedesktop.org/pipewire/pipewire/-/merge_requests?scope=all&state=all&author_username=SanchayanMaity)
|
||
|
|
- [PulseAudio](https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests?scope=all&state=all&author_username=SanchayanMaity)
|
||
|
|
- [Linux](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?qt=author&q=Sanchayan+Maity)
|
||
|
|
- [u-boot](https://source.denx.de/u-boot/u-boot/-/commits/master?search=Sanchayan%20Maity)
|
||
|
|
|
||
|
|
# Agenda
|
||
|
|
|
||
|
|
- Whirlwind tour of GStreamer
|
||
|
|
- What's HTTP Live Streaming (HLS)
|
||
|
|
- HLS implementation
|
||
|
|
|
||
|
|
# GStreamer
|
||
|
|
|
||
|
|
- Multiplatform Pipeline based multimedia framework
|
||
|
|
- Bindings for various languages
|
||
|
|
- Allows building complex media processing workflows
|
||
|
|
- Some applications
|
||
|
|
* PiTiVi (Video Editor)
|
||
|
|
* amaroK, Banshee, Clementine (audio players)
|
||
|
|
* Empathy (VOIP and video conferencing)
|
||
|
|
* GstLAL (gravitational wave data analysis)
|
||
|
|
* Rygel (DLNA streaming server and renderer)
|
||
|
|
* Totem (movie player for the GNOME desktop)
|
||
|
|
|
||
|
|
|
||
|
|
# Simple pipeline
|
||
|
|
|
||
|
|
```bash
|
||
|
|
gst-launch-1.0 videotestsrc ! autovideosink
|
||
|
|
gst-launch-1.0 audiotestsrc ! autoaudiosink
|
||
|
|
```
|
||
|
|
|
||
|
|
# Media pipeline[^1]
|
||
|
|
|
||
|
|
{width=80%}
|
||
|
|
|
||
|
|
[^1]: [Dynamic Pipelines](https://gstreamer.freedesktop.org/documentation/tutorials/basic/dynamic-pipelines.html?gi-language=c)
|
||
|
|
|
||
|
|
# HLS
|
||
|
|
|
||
|
|
- HTTP-based adaptive bit-rate streaming communications protocol
|
||
|
|
- Developed by Apple and released in 2009
|
||
|
|
- Standardised in [RFC 8216](https://datatracker.ietf.org/doc/html/rfc8216)
|
||
|
|
- HTTP traffic, unlike UDP-based protocols such as RTP
|
||
|
|
- Content can be offered from conventional HTTP servers
|
||
|
|
- Delivered over widely available HTTP-based content delivery networks
|
||
|
|
|
||
|
|
# Motivation
|
||
|
|
|
||
|
|
- Video on demand content
|
||
|
|
* HLS is better than just serving media directly over HTTP
|
||
|
|
* Can describe metadata
|
||
|
|
* Variants using a bit rate ladder
|
||
|
|
- Alternate media renditions
|
||
|
|
- Live content
|
||
|
|
* Reuse the same mechanism
|
||
|
|
* Trading off lower cost distribution via CDN at the cost of latency over real time
|
||
|
|
|
||
|
|
# Playlist[^2]
|
||
|
|
|
||
|
|
```m3u8
|
||
|
|
#EXTM3U
|
||
|
|
#EXT-X-TARGETDURATION:10
|
||
|
|
#EXT-X-VERSION:3
|
||
|
|
#EXTINF:9.009,
|
||
|
|
http://media.example.com/first.ts
|
||
|
|
#EXTINF:9.009,
|
||
|
|
http://media.example.com/second.ts
|
||
|
|
#EXTINF:3.003,
|
||
|
|
http://media.example.com/third.ts
|
||
|
|
#EXT-X-ENDLIST
|
||
|
|
```
|
||
|
|
|
||
|
|
[^2]: [RFC 8216](https://datatracker.ietf.org/doc/html/rfc8216#page-50)
|
||
|
|
|
||
|
|
# Master playlist[^3]
|
||
|
|
|
||
|
|
```m3u8
|
||
|
|
#EXTM3U
|
||
|
|
#EXT-X-STREAM-INF:BANDWIDTH=1280000,AVERAGE-BANDWIDTH=1000000
|
||
|
|
http://example.com/low.m3u8
|
||
|
|
#EXT-X-STREAM-INF:BANDWIDTH=2560000,AVERAGE-BANDWIDTH=2000000
|
||
|
|
http://example.com/mid.m3u8
|
||
|
|
#EXT-X-STREAM-INF:BANDWIDTH=7680000,AVERAGE-BANDWIDTH=6000000
|
||
|
|
http://example.com/hi.m3u8
|
||
|
|
#EXT-X-STREAM-INF:BANDWIDTH=65000,CODECS="mp4a.40.5"
|
||
|
|
http://example.com/audio-only.m3u8
|
||
|
|
```
|
||
|
|
[^3]: [RFC 8216](https://datatracker.ietf.org/doc/html/rfc8216#section-8.4)
|
||
|
|
|
||
|
|
# Master playlist[^4]
|
||
|
|
|
||
|
|
```m3u8
|
||
|
|
#EXTM3U
|
||
|
|
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",NAME="English", \
|
||
|
|
DEFAULT=YES,AUTOSELECT=YES,LANGUAGE="en", \
|
||
|
|
URI="main/english-audio.m3u8"
|
||
|
|
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",NAME="Deutsch", \
|
||
|
|
DEFAULT=NO,AUTOSELECT=YES,LANGUAGE="de", \
|
||
|
|
URI="main/german-audio.m3u8"
|
||
|
|
#EXT-X-STREAM-INF:BANDWIDTH=1280000,CODECS="...",AUDIO="aac"
|
||
|
|
low/video-only.m3u8
|
||
|
|
#EXT-X-STREAM-INF:BANDWIDTH=2560000,CODECS="...",AUDIO="aac"
|
||
|
|
mid/video-only.m3u8
|
||
|
|
#EXT-X-STREAM-INF:BANDWIDTH=7680000,CODECS="...",AUDIO="aac"
|
||
|
|
hi/video-only.m3u8
|
||
|
|
#EXT-X-STREAM-INF:BANDWIDTH=65000,CODECS="mp4a.40.5",AUDIO="aac"
|
||
|
|
main/english-audio.m3u8
|
||
|
|
```
|
||
|
|
[^4]: [RFC 8216](https://datatracker.ietf.org/doc/html/rfc8216#section-8.6)
|
||
|
|
|
||
|
|
# Implementation
|
||
|
|
|
||
|
|
- New GStreamer plugin
|
||
|
|
- Written in Rust
|
||
|
|
- Uses [m3u8-rs](https://docs.rs/m3u8-rs/latest/m3u8_rs/) and existing [`hlssink3`](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/tree/main/net/hlssink3?ref_type=heads)
|
||
|
|
- Open MR: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1515
|
||
|
|
|
||
|
|
# Demo
|
||
|
|
|
||
|
|
Using
|
||
|
|
|
||
|
|
- [videojs](https://videojs.com/)
|
||
|
|
- `python3 -m http.server`
|
||
|
|
- Sample GStreamer Rust Code[^5]
|
||
|
|
|
||
|
|
[^5]: [HLS sample app](https://git.sanchayanmaity.net/sanchayanmaity/gst-hlssink4)
|
||
|
|
|
||
|
|
# References
|
||
|
|
|
||
|
|
- [RFC 8216](https://datatracker.ietf.org/doc/html/rfc8216)
|
||
|
|
- [HLS in depth](https://dyte.io/blog/hls-in-depth/)
|
||
|
|
- [LL-HLS in depth](https://dyte.io/blog/ll-hls-in-depth/)
|
||
|
|
- [HTTP Live Streaming - A Practical Guide](https://hlsbook.net/)
|
||
|
|
- [HTTP Live Streaming - Wikipedia](https://en.wikipedia.org/wiki/HTTP_Live_Streaming)
|
||
|
|
|
||
|
|
# Questions
|
||
|
|
|
||
|
|
- Reach out on
|
||
|
|
* Email: sanchayan@sanchayanmaity.net
|
||
|
|
* Mastodon: [sanchayanmaity.com](https://sanchayanmaity.com/@sanchayan)
|
||
|
|
* Telegram: https://t.me/SanchayanMaity
|
||
|
|
* Blog: [sanchayanmaity.net](https://sanchayanmaity.net/)
|