notes/gstreamer/quic.md

---
title: quic
---

## GStreamer

Existing implementations which support stream and datagram multiplexing are

- https://github.com/bbc/gst-quic-transport
- https://github.com/bbc/gst-roq

## gst-quic-transport

Few points to think about before hand

- Flow control
- Stream credits
- Priority among streams

### rtpquicmux

- Adds the flow identifier
- Maps RTP and RTCP packets to QUIC streams or datagram
- Each payload type and SSRC is send on a unique QUIC stream (uses a hash table to track this)

### quicmux

- Multiplex QUIC streams and datagrams on the same QUIC transport connection
- Adds meta to buffers to distinguish between stream and datagrams at the transport layer
- New stream (uni/bi-directional) or datagram connection is opened depending on the requested pad
- Buffers on stream pads don't map directly to frames at QUIC layer and are payloaded by the library.
- Uses custom query events for handling connections with `quicsink` downstream

### quicsrc

- Uses the meta on buffers to decide if the buffer is from a stream or datagram

# Media over QUIC

- Optimised for QUIC protocol either directly or via WebTransport.
- Utilises a publish subscribe workflow
- Designed to work with multiple MoQ streaming formats which define how content is encoded, packaged and mapped to MOQT objects along with policies for discovery and subscription.

## Terms

**Group**: a temporal sequence of objects. A group represents a join point in a track.

**Track**: an encoded bitstream. Tracks contain a sequential series of one or more groups and are the subscribable entity with MOQT.

**Object**: an object is an addressable unit whose payload is a sequence of bytes. Objects form the base element in MOQT model.

#### Objects

- Basic data element of MOQT is an object.
- An objects is an addressable unit whose payload is a sequence of bytes.
- All objects belong to a group (indicates ordering and potential dependencies)
- Uniquely identified by track namespace, track name, group and object ID.
- Can become unavailable but contents must not change over time.
- Comprises of metadata (never encrypted, visible to relays) and payload.
- Application alone responsible for content of object payload.
- Relay must not combine, split or otherwise modify object payloads.

#### Groups

- Collection of objects, sub-unit of track.
- Objects within groups shouldn't depend on objects in other groups.
- Behaves as joint point for subscriptions.

#### Track

- Sequence of groups.
- Entity against which a consumer issues a subscription request.
- Subscriber can request to receive individual tracks starting at group boundary.

### Session initialisation

- First stream opened is a client initiated bi-directional control stream.
- Peers exchange setup messages on this stream.
- Control messages must not be sent on any other stream, if received on a non-control stream, it'll be a protocol violation.
- Control stream is the only bi-directional stream. Objects are sent on uni-directional streams. A second bi-directional stream would be a protocol violation.
- Objects are sent on uni-directional streams.

### Stream cancellation

- Streams aside from the control stream may be cancelled due to congestion or any other reason.

### Termination

The transport session can be terminated at any point.


# WARP

- WARP streaming format is a media format designed to deliver CMAF compliant media content over MOQT.
- Fragments the bitstream into objects that can be independently transmitted.
- Uses a prioritisation strategy of assigning newer content a higher delivery order and video over audio when facing congestion.
- Either complete GOPS or individual frames are mapped to MoQTransport objects.
- Targeted at interactive levels of latency.

### Time alignment

WARP tracks may be time aligned. (*Don't understand this here*)

- Time aligned tracks must be advertised in the catalogue as belonging to a common render group.
- Presentation time of the first media sample contained within the first MOQT object of each equally numbered MOQT group must be identical.

### Media transmission

MOQT groups and objects need to be mapped to MOQ transport streams. Each MOQT object must be mapped to a new moq-transport stream.