Proof of concept of an open and standardized WebRTC based streaming
In this blog post we will describe a proof of concept that we are working on with the aim to find an open and standardized way of using WebRTC for real-time low-latency streaming. This is our contribution to the technical work in the context of standardizing WebRTC-based streaming, and all source code related to this proof of concept are made available as open source.
There are a number of use cases that are made possible by integrating WebRTC streaming. A report developed by DASH-IF describes some of the use cases and the technical considerations that needs to be taken into account [1]. Use cases that ranges from interactive live concerts / music events to, in-stadium interactivity and lower latency broadcasting.
WebRTC is a set of standards and support for WebRTC is today included in all modern browsers, and even some more modern Smart TVs. Original focus of WebRTC was around video conferencing but its ultra-low latency features (demanded by two-way communication) is an attractive feature in real-time streaming. Reducing the glass-to-glass latency is one of the most discussed topics in the streaming industry. But adopting and integrating WebRTC in the streaming industry has been a slow process and the lack of standards on how to integrate WebRTC in this context we believe has contributed to that slow pace.
There are two main areas that needs to be standardized as a start: ingest and playback. When we started this proof of concept we decided to reduce the scope to cover these two fundamental areas. On the ingest side there is already an initiative to standardize ingesting of media into a streaming service or CDN using WebRTC called WHIP (WebRTC-HTTP Ingestion Protocol [2]). A draft standard that was published in October 2021 and written by S. Murillo and A. Gouaillard from CoSMo Software.
We found that a relevant use case that would cover these two areas was to develop a simple web-based application that allows you to instantly start a broadcast from either your computer or mobile just using the in-built browser. When the broadcast is live you will get a link to another web page that you can share with your viewers. As for the sender / broadcaster a viewer would not need any additional application installed to watch the broadcast, and should only need to open the provided link in a web browser. A perfect use case that also demonstrates the benefit of using WebRTC as all modern browsers have in-built support for it.
We developed a Javascript library [3] that implements the WHIP protocol and can be integrated into a web application. This means that a web based ingest application based on this library and the WHIP protocol is not tightly tied to a specific WebRTC media server as long as the media server has WHIP support. As part of this project we developed a basic media server that provides a WHIP endpoint and a broadcaster (SFU) [4]. Now we have a standardized way for a sender to ingest media using WebRTC.
Next thing to tackle was the playback of this channel. At the time of writing this post and this project it did not yet exist a standard for consuming and playing a channel using WebRTC. Here we had to take a bit of a different approach due to this fact. We developed a player library for WebRTC playback [5] that was designed to be media server independent. We achieved this by designing the player using media server adapters.
The adapter is responsible for the SDP exchange (offer/answer) while the other commonalities are handled outside of the adapter and in the player. The player attach the WebRTC stream to an HTML video element but does not render any type of skin or controls. That was intentionally left outside of the scope of this library. We started by implementing an adapter that worked with our media server that we developed for this prototype. The SDP exchange in this case is inspired by WHIP as it uses HTTP as the transport protocol with a resource based API design. A channel resource is located with a URL and the media server adapter issues an HTTP POST with an application/json payload including the SDP offer. The reason for using JSON was in case we wanted to extend it with other data than just the SDP. In response the adapter receives the SDP answer from the media server.
Then we added this WebRTC player library into our Eyevinn Web Player [6]. The Eyevinn Web Player is a modular player framework that incorporates other player libraries such as HLS.js, Shaka Player and DASH.js for supporting playback of the most common HTTP streaming protocols. Adding the WebRTC support by using the WebRTC player library was a breeze and we could utilize the skin and controls already available in the Web Player.
We now have a player that is not bound to a specific media server however how do we signal what adapter the player should be using? The way we solved this, and hope this could be an inspiration to the standardization work that is ongoing around DASH and WebRTC, was to extend an MPEG-DASH manifest a small bit. Our media server can return an MPEG-DASH manifest for a channel, for example:
GET https://broadcaster-wrtc.dev.eyevinn.technology:443/broadcaster/mpd/60dc933b-0738-485d-8f81-01d8fae7402cwould return this MPEG-DASH:
<MPD profiles="urn:mpeg:dash:profile:webrtc-live:2022">
<Period
xlink:href="https://broadcaster-wrtc.dev.eyevinn.technology:443/broadcaster/channel/60dc933b-0738-485d-8f81-01d8fae7402c"
xlink:rel="urn:ietf:params:whip:eyevinn-wrtc-channel"
xlink:actuate="onRequest"></Period>
</MPD>An MPEG-DASH manifest that contains a single period with an xlink:href and the channel resource locator. For the WebRTC player to understand which adapter to use we added an xlink:rel attribute describing what type of resource it points to. We could then parse this manifest and know which adapter to use in the player.
With this prototype we have a proof-of-concept where a browser based WebRTC ingest application can ingest to any type of WHIP compatible media server and a way to signal to a media server independent WebRTC player what adapter to use. Everything is open source and you are welcome to use it in the standardization efforts around WebRTC and playback. The end goal is to not have to have media server specific adapters and instead one that implements a common standard by all media servers.
If you want to try it out the prototype is available online at https://webcast.eyevinn.technology/
Contributors to this project so far are Jonas Birmé, Benjamin Wallberg and Marcus Spangenberg, all from Eyevinn Technology AB.
If you want to know more about this prototype and proof-of-concept you can contact Jonas Birmé, VP R&D Eyevinn Technology.
Eyevinn Technology is leading specialists in video technology and media distribution, and proud organizer of the yearly nordic conference Streaming Tech Sweden.
Related developer blogs:
- https://dev.to/video/media-server-independent-webrtc-player-1gn3
- https://dev.to/video/open-source-browser-application-for-webcast-496d
Links in this article:
[1] https://dashif.org/webRTC/report
[2] https://www.ietf.org/archive/id/draft-ietf-wish-whip-01.html
[3] https://www.npmjs.com/package/@eyevinn/whip-web-client
[4] https://www.npmjs.com/package/@eyevinn/whip-endpoint
[5] https://github.com/Eyevinn/webrtc-player
[6] https://web.player.eyevinn.technology/
