Simulcast

Introduction to Simulcast

The basic SFU architecture solves several issues with other architecture types but also introduces some issues. One of those issues is introduced by sending multiple video streams to all users. If the user’s internet connection does not have enough bandwidth to handle multiple streams and upload its own, it can lead to a worse call experience for all users. To solve this, a technique called Simulcast is used.

Simulcast allows a connected user to send multiple versions of the same video stream at different resolutions and bitrates. This allows viewers with different bandwidths to receive the best possible video quality. For example, the SFU could send a high-resolution, high-bitrate stream to viewers with fast internet connections and a low-resolution, low-bitrate stream to users with slow internet connections. This ensures that everyone can watch the video without any buffering or lag.

Here are some of the benefits of simulcast:

• Allows users with different bandwidths to receive the best possible video quality
• Reduce the amount of bandwidth used
• Improves the reliability of video streaming

Problems with the basic SFU architecture

The basic SFU architecture, as described in the previous lesson describes all connected users sending a single video stream and every individual user receiving all video streams available. This does not consider that users with lower bandwidth may be available than other users. Due to this, SFUs can face an issue named the ‘slow user problem’.

The slow user problem is a challenge that can occur in SFUs when a participant has a slow internet connection. This can cause the SFU to buffer the video stream from the slow participant, delaying the video for all participants. WebRTC also contains error resiliency mechanisms such as Picture Loss Indication (PLIs) and Negative Acknowledgement (NACKs) that help deal with packet loss when sending video streams. However, too many requests can slow down the call for all users.

How Does Simulcast Work?

Simulcast can be used with an SFU to improve the quality of video conferencing calls, even when participants have different internet connections. The SFU can receive multiple versions of the video stream from each participant and then forward the appropriate version to each participant based on their bandwidth. This ensures that everyone can see the video without any buffering or lag. Simulcast is also good because you can scale and send different resolutions and quality based on the user’s device - for example, if a user has a video in Picture-in-picture mode, the smaller window can be sent a lower resolution.

Here is an example of how simulcast can be used with an SFU in a call with three participants:

• Participant A has a fast internet connection
• Participant B has a slow internet connection
• Participant C has an average internet connection

The video encoder for Participant A will send three versions of the video stream: a high-resolution, high-bitrate stream, a medium-resolution, medium-bitrate stream, and a low-resolution, low-bitrate stream.

The SFU will receive the three versions of the video stream from Participant A and then forward the appropriate version to each participant:

• The high-resolution, high-bitrate stream will be forwarded to Participant A.
• The medium-resolution, medium-bitrate stream will be forwarded to Participant B.
• The low-resolution, low-bitrate stream will be forwarded to Participant C.

This ensures that everyone can see video streams without buffering or lag, even though they have different internet connections.