53 lines
4.3 KiB
Markdown
53 lines
4.3 KiB
Markdown
|
+++
|
||
|
|
title = "Notes from GStreamer spring hackfest of 2024"
|
||
|
|
date = 2024-07-30
|
||
|
|
+++
|
||
|
|
|
||
|
|
Some time has passed since the GStreamer spring hackfest took place in Thessaloniki, Greece in the month of May. Second time attending the GStreamer hackfest and thought about summarizing some of my thoughts this time around.
|
||
|
|
|
||
|
|
## Thanks
|
||
|
|
|
||
|
|
Before getting into the details, want to send out a thank you to:
|
||
|
|
|
||
|
|
- The GStreamer foundation for sponsoring the event as a whole
|
||
|
|
- Sebastian, Vivia and Jordan for making all the arrangements
|
||
|
|
- Asymptotic, for sponsoring my presence at the event
|
||
|
|
|
||
|
|
## The event
|
||
|
|
|
||
|
|
[At the hackfest](https://social.cologne/@francisv/112518409917632684).
|
||
|
|
|
||
|
|
It was good to see some familiar faces at the event, folks whom I had met at the previous hackfest and conference. Also nice when you finally meet people you have only conversed with online and get to put a face on the online persona you have been conversing with.
|
||
|
|
|
||
|
|
## Work
|
||
|
|
|
||
|
|
Originally the plan was to work on adding stream multiplexing support to [QUIC](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/tree/main/net/quinn/src?ref_type=heads) elements. However, missed pushing some of the work to GitLab which was on desktop and decided to work on that later.
|
||
|
|
|
||
|
|
### HTTP Live Streaming (HLS)
|
||
|
|
|
||
|
|
A merge request for adding [multi-variant playlist support with HLS](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1515) has been pending review for a while. One of the features missing from that merge request was support for codec string generation when using MPEG-TS with H-264 and H-265. Decided to work on that.
|
||
|
|
|
||
|
|
H-264 or H-265 has what are known as `stream-formats`. H-264 or H-265 can be stream oriented or packet oriented. In the case of former, `stream-format` is said to be `byte-stream`, while in the case of latter, `stream-format` is said to be `avc`. For `byte-stream`, the required parameter sets are sent in-band with the video, but for `avc` in GStreamer, the video metadata is conveyed via an additional caps field named `codec_data` which can be considered as out-of-band. `codec_data` is only present when the video is packet oriented, that's when `stream-format` is `avc`, this value represents an `AVCDecoderConfigurationRecord` structure.
|
||
|
|
|
||
|
|
GStreamer already has helper functions in codec utilities which can provide information like `profile-level` which are required for constructing codec strings. However, these helper functions require the existence of `codec_data`.
|
||
|
|
|
||
|
|
When using MPEG-TS as the container, the only possible `stream-format` is `byte-stream` with H-264 or H-265. In this case, one needs to parse the in-band information for getting information like `profile-level` or other video metadata. In Rust, there is the [`cros-codecs`](https://docs.rs/cros-codecs/latest/cros_codecs/codec/index.html) crate which has a parser module. Using this, it was easy to parse the in-band data and then generate the codec string required for HLS playlist.
|
||
|
|
|
||
|
|
### Threadshare
|
||
|
|
|
||
|
|
Before the hackfest, had spend some time on understanding the [`threadshare`](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/tree/main/generic/threadshare/src?ref_type=heads) elements. Met François Laignel at the hackfest who helped with clearing doubts I had with how some of the code was laid out in `threadshare`.
|
||
|
|
|
||
|
|
If you are interested in understanding about what makes the `threadshare` elements different, highly recommend going through the blog post [here](https://coaxion.net/blog/2018/04/improving-gstreamer-performance-on-a-high-number-of-network-streams-by-sharing-threads-between-elements-with-rusts-tokio-crate/).
|
||
|
|
|
||
|
|
There was some `end-of-stream` handling missing with the `threadshare`, `tcpclientsrc` and `udpsrc` elements. Spend some time working on adding support for that, which has now been [merged](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1581#d18be81ee1a92c24cf3786f98c51bc93bd1ce661) upstream.
|
||
|
|
|
||
|
|
## Play
|
||
|
|
|
||
|
|
After the three days of hackfest, a day trip was planned to the Palace of Aigai.
|
||
|
|
|
||
|
|
[GStreamer hackers & co. exploring the Palace of Aigai](https://social.cologne/@francisv/112530301636663783).
|
||
|
|
|
||
|
|
## Conclusion
|
||
|
|
|
||
|
|
All in all, this hackfest turned out to be a productive and fun filled hackfest. Also, have to add that Greek cuisine is excellent and look forward to the next hackfest and visiting Thessaloniki/Greece again.
|