How Does Simulcast Let One Video Call Serve Every Viewer's Bandwidth?

TL;DR

• Every viewer on a call has different bandwidth, so a single video encode always looks wrong for someone. Too heavy for one connection, too soft for another.
• Simulcast has the sender encode the same camera feed at two or three qualities at once; the SFU forwards whichever layer each viewer's bandwidth and screen size call for, with no re-encoding on the server.
• It costs the sender roughly 30-40% more upload and encoding work than a single top-quality stream. It's cheap because each lower layer has a fraction of the pixels.
• It's distinct from SVC (one-layered encoding instead of separate streams), server-side transcoding (heavier and slower), and the other "simulcast," sending one encode to multiple platforms like Twitch or YouTube.

Video calls are presented as a single entity. Everyone on the call is sending video and receiving from everyone else (except those who always have their cameras off).

But under the hood, that's not true. Everyone on a video call is on a different connection. One person could be on a great fiber connection, another on home Wi-Fi, and a third on a train with a mobile signal that comes and goes.

If the sender produces a single encode, you have to choose which of those viewers gets a bad experience. A 720p stream at 1.5 Mbps stalls on the train, and a 300 kbps stream looks very bad for the person on fiber.

What Does Simulcast Mean in a Video Call?

In a WebRTC call, simulcast means the sending device encodes its camera feed two or three times in parallel, at different resolutions and bitrates. It then uploads every one of those encodings to the call's Selective Forwarding Unit (SFU).

A typical three-layer setup looks like this:

Each layer is a separate, self-contained encode. The 360p stream doesn't depend on the 720p stream in any way, which is what lets the SFU hand any single layer to a viewer on its own. Most implementations also split each layer into temporal sub-layers, so the SFU can lower the frame rate within a layer, say from 30 fps to 15, without changing resolution.

The viewer never sees any of this. On the receiving side, there's one video track per participant, and the SFU quietly swaps which encoding it feeds. All the extra work sits with the sender and the server.

Simulcast only matters when there's a server in the path. In a 1:1 peer-to-peer call, the sender talks directly to one receiver, so it encodes once and lets congestion control adjust that single stream. Add a third participant through an SFU, and the mismatch problem shows up right away.

How Does the SFU Decide Which Layer Each Viewer Gets?

The SFU makes this choice for each viewer and participant, and revisits it constantly. Two inputs drive it.

• Bandwidth. Receivers send feedback about what's arriving, mostly through transport-wide congestion control reports, and the SFU turns that into a running estimate of how much each connection can carry. A viewer on Wi-Fi with heavy interference, or on a congested mobile network, sees that estimate drop, and the SFU responds by forwarding a lower layer.
• Layout. A participant drawn as a small tile doesn't need 720p even when the viewer has bandwidth to spare, so the client tells the SFU what size it's rendering each participant at, and the SFU picks the smallest layer that fills it. When that participant becomes the active speaker and switches to full screen, the client requests a higher layer.

So, a single point in time on a call might look like this:

Say you're in a 20-person call in grid view, getting everyone at 360p, and you click on Kyle to make him full screen.

• Your client tells the SFU it's now drawing Kyle at full size and everyone else as small thumbnails.
• The SFU wants to move you from Kyle's 360p stream to his 720p one. It can't just start forwarding 720p packets, though. Every frame in that stream is encoded as a set of changes from the frame before it, so your decoder needs a keyframe, a complete picture, to start from.
• The SFU requests a keyframe from Kyle's device on the 720p layer. This is a small RTCP message called a PLI (picture loss indication), and Kyle's encoder usually answers within a frame or two.
• When the keyframe arrives, the SFU stops sending you Kyle's 360p packets and starts sending 720p packets instead. It rewrites the packet numbers and timestamps as it goes, so as far as your decoder can tell it's the same stream, just sharper.
• The other 19 people just got small, so the SFU moves you down to their 180p layers the same way, which needs a keyframe from each of them, too. Anyone who's now off-screen stops being forwarded at all.

On your screen, Kyle's tile grows and for a moment shows the 360p picture stretched up, a little soft. Then the 720p keyframe lands, and it snaps sharp.

Connection changes are handled with some deliberate caution because every layer switch costs a keyframe, and keyframes are big.

• When your connection gets worse, the SFU reacts quickly. Its first move is often to drop frame rate rather than resolution, because the encoder arranges frames so that some can be skipped without affecting the rest. The SFU just stops forwarding those frames, no keyframe needed.
• When your connection gets better, it waits. It wants to see that you can sustain the next layer's bitrate for a while before stepping up; otherwise, a flaky connection would keep flipping between layers and every flip would trigger another keyframe.
• Active speaker changes get a short delay for the same reason, so a two-second interjection doesn't promote someone to full screen and back.

Through all of this, the SFU never actually looks at the video. It reads packet headers, decides which packets to pass on, and forwards them. That's why one server can handle hundreds of viewers cheaply and without adding noticeable delay. A server that decoded and re-encoded video for each viewer would require far more CPU and add latency to every stream.

Does Simulcast Triple the Sender's Upload?

Not triple, but more than a single stream. The example ladder above is 150 + 500 + 1,500 = 2,150 kbps, against 1,500 kbps for the 720p layer on its own. That's about 43% more upload. On the wire, it's slightly higher once packet headers and retransmissions are accounted for.

The extra layers are cheap because of pixel counts. Each step down the ladder halves the width and height, leaving a quarter of the pixels.

So the sender is encoding 31% more pixels than it would for a single 720p stream. Adding a fourth or fifth quarter-size layer barely moves that, since the series 1 + 1/4 + 1/16 + 1/64 never gets past 4/3. The bitrate overhead (43%) is higher than the pixel overhead (31%) because small pictures require more bits per pixel to remain watchable. At 30 fps, the 180p layer uses about 0.09 bits per pixel, and the 720p layer about 0.05 bits per pixel.

However, that also means the budget grows quickly in the other direction. People now expect 1080p on calls because that's what streaming services give them, and on a call the sender's device pays for it. Keep the same quarter-size steps. A 1080p ladder is 1080p at around 4,000 kbps, 540p at about 1,200 kbps, and 270p at about 350 kbps, which adds up to 5,550 kbps.

That's 2.25 times the pixels of the 720p ladder and 2.6 times the upload, and encoding work and battery drain grow with the number of pixels, too. A steady 5.5 Mbps up is more than many home connections can handle, and every participant with a camera on has to sustain it, not just whoever is on screen. Go to 4K, and the top layer alone lands somewhere around 15-20 Mbps, which is why 4K calls are still rare (thankfully).

Most viewers never receive the expensive layer either. In a grid, their tile of you is small, so the SFU hands them 540p or 270p. The 1080p encode only reaches someone who has you full screen on a fast connection, so the sender is paying for a layer that often goes unwatched.

How Is Simulcast Different From SVC, Transcoding, and Adaptive Bitrate Streaming?

All four solve the same mismatch problem. They differ in who produces the quality versions and who picks one for each viewer.

SVC is the closest relative. The encoder produces a single stream in which each layer builds on the one below it, and the SFU strips layers from packets rather than choosing between separate streams. That saves upload bandwidth and encoding work on the sender, and layer switches don't need keyframes.

The catch with SVC is video codec support. VP9 and AV1 support it in Chromium-based browsers, while H.264 only gets temporal layers in browsers, not spatial ones, so simulcast is still the safe default when hardware encoders or older devices are involved. Plenty of SFUs do both and choose per call.

Server-side transcoding is what an MCU does. The server decodes everything, composes or rescales it, and encodes a fresh stream for each viewer. That gives the sender the easiest job, but it costs a lot of CPU per viewer and adds latency, which is why SFUs took over for interactive calls.

Adaptive bitrate streaming (ABR) is the same idea applied to live streams and on-demand video content. The origin transcodes one input into a ladder of renditions, chops them into segments, and the player picks a rendition segment by segment. It scales to millions of viewers through a CDN, but it runs seconds behind live, so it suits watching more than talking.

Is Simulcast the Same as Multistreaming to Twitch, YouTube Live, and Facebook Live?

No. Outside WebRTC, "simulcast" is short for simultaneous broadcast: sending the same program over several channels at once.

The word comes from radio stations and TV networks. A network like CBS or ESPN might broadcast simultaneously on its main channel and a secondary one, or air a live broadcast on TV while a radio station carries the audio. Anime uses it the same way. A simulcast episode streams internationally the same day it airs in Japan, rather than months later.

For content creators and streaming platforms, the term has become interchangeable with multistreaming. One live stream goes out from OBS or a similar encoder to a service like Restream, StreamYard, or Vimeo, and that service restreams it to Twitch, YouTube Live, Facebook Live, and LinkedIn Live at the same time. Cloudflare labels its version of this "simulcast (restream)".

Every destination receives the same encode, and each platform, whether that's LinkedIn or Twitch, then transcodes it into its own quality ladder for viewers.

So the two meanings point in opposite directions:

• Multistreaming sends a single live video encode to multiple social media platforms.
• WebRTC simulcast sends multiple encodes of a single feed to a single server.

Twitch's Enhanced Broadcasting sits somewhere in between. OBS encodes the stream at several resolutions on the streamer's GPU and sends them all to Twitch, so the platform doesn't need to transcode. Mechanically, that's close to WebRTC simulcast, but it's still a one-to-many live broadcast with seconds of latency, and it only works for one destination.

How Does Simulcast Work in Large Meetings and Livestreams?

In a 100-person meeting, no viewer downloads 99 video streams at full quality, and no laptop could decode them if it did. Simulcast is half of what makes large calls workable. The other half is that the client only subscribes to what's on screen.

• Tiles in the visible grid receive the layer that matches their rendered size, which is usually the low or medium one.
• The active speaker, or whoever is pinned, receives the top layer.
• Participants scrolled off-screen get paused entirely, so the SFU stops forwarding their video until they scroll back into view.
• Screen share is usually published as one high-resolution encode at a low frame rate, since text needs sharp pixels more than smooth motion, so it typically skips simulcast altogether.

That's why per-viewer download and decode costs track the layout rather than the headcount. A viewer looking at a 3x3 grid receives nine small streams, whether the meeting has 12 people or 500.

Stream Video's SFU works this way. Dynascale decides which resolutions and codecs participants publish and which layer each viewer receives, based on the video size, network conditions, and the device, and it pauses the video for participants who aren't visible. Screen sharing publishes a single resolution. The quality and latency guide has the details.

Livestreams are where the two meanings of simulcast come together. During the call, hosts still send simulcast layers to the SFU, and viewers watching over WebRTC have a layer picked for them in the same way. To reach a much larger audience, the SFU output can be composed into a single stream and packaged as HLS for CDN delivery, or pushed over RTMP to YouTube, Twitch, and Facebook, which is the multistreaming kind of simulcast.

Stream's RTMP broadcasts do exactly that, and a single call can reach several platforms at once. Each platform then transcodes the feed into its own ladder, so a viewer on a slow connection still gets a version they can watch, a few seconds behind live.