Selective Forwarding Unit (SFU)

Introduction to SFU Architecture

The Selective Forwarding Unit (SFU) represents the most popular architecture for modern WebRTC applications, striking a balance between the centralized processing of MCUs and the distributed nature of P2P. Unlike its predecessor MCU, which processes and mixes all media streams, an SFU acts as a smart traffic controller, efficiently routing media streams between participants without the need for computationally expensive transcoding.

The Core Concept

Think of an SFU as a post office. Each participant sends their package (media stream) to the post office, which then distributes copies to all other participants. Unlike an MCU that would open each package, combine contents, and repackage them, the SFU simply forwards the original packages unchanged. This approach distributes the processing load between the server and clients, leading to better scalability and flexibility.

The Technical Foundation: How SFU Works

Architecture Overview

In the SFU architecture, every connected device emits a single set of outgoing media streams to the backend. The SFU then forwards these streams to every other device without modification. Each individual device receives all streams from other participants via the SFU and is responsible for decoding and processing these streams to create a coherent video call interface.

Stream Management

The SFU maintains a routing table of all active participants and their streams, managing stream subscriptions dynamically based on client requests and network conditions. It handles stream quality adaptation through simulcast and SVC support while monitoring network conditions for each participant to make intelligent forwarding decisions.

javascript

            class SFUServer {
  constructor() {
    this.participants = new Map();
    this.streamRouter = new StreamRouter();
  }

  async handleNewParticipant(connection) {
    const participant = {
      id: generateId(),
      connection: connection,
      publishedStreams: new Map(),
      subscribedStreams: new Map()
    };

    this.participants.set(participant.id, participant);

    // Setup event handlers
    connection.on('publish', stream => {
      this.handleStreamPublish(participant.id, stream);
    });

    connection.on('subscribe', streamId => {
      this.handleStreamSubscribe(participant.id, streamId);
    });
  }

  async handleStreamPublish(publisherId, stream) {
    const publisher = this.participants.get(publisherId);
    publisher.publishedStreams.set(stream.id, stream);

    // Notify all other participants about the new stream
    this.broadcastStreamAvailability(publisherId, stream.id);
  }

  async handleStreamSubscribe(subscriberId, streamId) {
    const subscriber = this.participants.get(subscriberId);
    const stream = this.findStream(streamId);

    if (stream) {
      // Create forwarding path
      this.streamRouter.createRoute(stream, subscriber.connection);
      subscriber.subscribedStreams.set(streamId, stream);
    }
  }
}

Bandwidth Management

Simulcast Support

One of the key advantages of SFU architecture is its support for simulcast (Simulcast, seen later), where clients send multiple versions of their video at different resolutions and bitrates:

javascript

            class SimulcastManager {
  constructor() {
    this.qualityLevels = ['high', 'medium', 'low'];
  }

  async setupSimulcast(participant) {
    const encodings = [
      { rid: 'high', maxBitrate: 2500000, scaleResolutionDownBy: 1 },
      { rid: 'medium', maxBitrate: 500000, scaleResolutionDownBy: 2 },
      { rid: 'low', maxBitrate: 150000, scaleResolutionDownBy: 4 }
    ];

    // Configure participant's encoder for simulcast
    await participant.configureEncodings(encodings);
  }

  selectQualityLevel(subscriber, publisher, networkConditions) {
    // Dynamic quality selection based on:
    // - Subscriber's bandwidth
    // - Display size requirements
    // - Current network conditions
    if (networkConditions.bandwidth > 1000000) {
      return 'high';
    } else if (networkConditions.bandwidth > 400000) {
      return 'medium';
    } else {
      return 'low';
    }
  }
}

Bandwidth Estimation

SFUs continuously monitor network conditions to optimize stream forwarding:

javascript

            class BandwidthEstimator {
  constructor() {
    this.measurements = new Map();
  }

  async measureBandwidth(participant) {
    const stats = await participant.getStats();

    // Analyze RTCP feedback
    const packetLoss = this.calculatePacketLoss(stats);
    const jitter = this.calculateJitter(stats);
    const rtt = this.calculateRTT(stats);

    // Estimate available bandwidth
    const estimatedBandwidth = this.estimateBandwidth({
      packetLoss,
      jitter,
      rtt,
      previousMeasurements: this.measurements.get(participant.id)
    });

    this.measurements.set(participant.id, estimatedBandwidth);
    return estimatedBandwidth;
  }
}

Scalability Analysis

Connection Complexity

SFU architecture scales more efficiently than P2P while requiring less server resources than MCU:

Participants	Network Connections	Server Processing	Client Processing
2	4	Minimal	1x decode
5	10	Minimal	4x decode
10	20	Minimal	9x decode
25	50	Minimal	24x decode
100	200	Minimal	99x decode

Resource Requirements

The resource demands in SFU architecture are distributed between server and clients:

Server Requirements: The SFU primarily needs strong network capabilities with high bandwidth capacity for stream forwarding, low-latency network interfaces, and efficient packet routing algorithms. CPU usage remains relatively low since there's no transcoding involved.

Client Requirements: Connected devices must handle multiple stream decoding, with requirements scaling with the number of visible participants. Modern devices can typically handle 4-9 simultaneous video decodes efficiently, though this varies based on resolution and device capabilities.

Performance Optimization

Selective Forwarding

SFUs implement intelligent forwarding strategies to optimize performance:

javascript

            class SelectiveForwarder {
  constructor() {
    this.activeStreams = new Map();
    this.viewportManager = new ViewportManager();
  }

  async optimizeStreamDelivery(subscriber) {
    // Determine which streams are actually visible
    const visibleParticipants = await this.viewportManager
      .getVisibleParticipants(subscriber);

    // Subscribe only to necessary streams
    for (const participant of visibleParticipants) {
      if (!this.activeStreams.has(subscriber.id)?.has(participant.id)) {
        await this.subscribeToStream(subscriber, participant);
      }
    }

    // Unsubscribe from non-visible streams
    for (const [participantId, stream] of this.activeStreams.get(subscriber.id) || []) {
      if (!visibleParticipants.includes(participantId)) {
        await this.unsubscribeFromStream(subscriber, participantId);
      }
    }
  }
}

Last-N Architecture

Many SFUs implement Last-N strategy to limit the number of forwarded streams:

javascript

            class LastNManager {
  constructor(n = 5) {
    this.maxStreams = n;
    this.activeSpeakers = new Map();
  }

  updateActiveSpeakers(audioLevels) {
    // Sort participants by audio activity
    const sortedParticipants = Array.from(audioLevels.entries())
      .sort((a, b) => b[1] - a[1])
      .slice(0, this.maxStreams);

    // Update active speakers list
    this.activeSpeakers.clear();
    sortedParticipants.forEach(([id, level]) => {
      this.activeSpeakers.set(id, level);
    });
  }

  shouldForwardStream(participantId) {
    return this.activeSpeakers.has(participantId);
  }
}

Cost Analysis

Infrastructure Costs

SFU architecture offers a cost-effective balance between server infrastructure and client requirements. Server costs are significantly lower than MCU since there's no transcoding, though bandwidth costs are higher due to multiple stream forwarding. The infrastructure can scale horizontally by adding more SFU instances as needed.

Cost Comparison

Aspect	MCU	SFU	P2P
Server CPU	Very High	Low	Minimal
Server Bandwidth	Medium	High	Minimal
Client CPU	Low	Medium	High
Client Bandwidth	Low	Medium	High
Scalability	Medium	High	Low
Cost at Scale	High	Medium	Low

Advanced SFU Features

Simulcast and SVC

Modern SFUs support advanced encoding techniques:

javascript

            class EncodingManager {
  constructor() {
    this.simulcastLayers = ['high', 'medium', 'low'];
    this.svcSupported = true;
  }

  async setupAdvancedEncoding(participant) {
    if (this.svcSupported && participant.supportsSVC()) {
      // Configure Scalable Video Coding
      await this.setupSVC(participant);
    } else {
      // Fallback to simulcast
      await this.setupSimulcast(participant);
    }
  }

  async setupSVC(participant) {
    const svcConfig = {
      spatialLayers: 3,
      temporalLayers: 3,
      mode: 'L3T3'
    };

    await participant.configureEncoder(svcConfig);
  }
}

Recording and Analytics

SFUs can efficiently implement recording and analytics:

javascript

            class RecordingManager {
  constructor() {
    this.recordings = new Map();
  }

  async startRecording(roomId) {
    const recorder = new MediaRecorder();

    // Tap into stream forwarding pipeline
    const streamTap = await this.createStreamTap(roomId);

    recorder.ondataavailable = (data) => {
      this.saveRecordingChunk(roomId, data);
    };

    recorder.start();
    this.recordings.set(roomId, recorder);
  }
}

When to Use SFU Architecture

Ideal Use Cases

SFU architecture excels in modern web applications where devices have reasonable processing power and bandwidth is available. It's particularly well-suited for medium to large group calls (5-100 participants), applications requiring flexible layouts and quality adaptation, and scenarios where end-to-end encryption is important. The architecture also works well for applications needing recording capabilities and real-time analytics.

When to Consider Alternatives

You might want to consider other architectures when working with very large conferences (>100 participants without cascading), legacy devices with limited processing power, extremely bandwidth-constrained environments, or when server-side composition is required. Applications needing guaranteed quality across all clients regardless of their capabilities might be better served by MCU architecture.

SFU Deployment Strategies

Single-Region Deployment

For applications with geographically concentrated users:

javascript

            class SingleRegionSFU {
  constructor(region) {
    this.region = region;
    this.loadBalancer = new LoadBalancer();
  }

  async routeParticipant(participant) {
    // Find least loaded SFU instance in region
    const instance = await this.loadBalancer
      .findOptimalInstance(this.region);

    return instance.connect(participant);
  }
}

Multi-Region with Cascading

For global applications, cascading SFUs provide better performance. See the lesson about SFU Cascading further in this module. In general, you don't want to run a SFU without cascading:

javascript

            class CascadingSFU {
  constructor() {
    this.regions = new Map();
    this.meshConnections = new Map();
  }

  async setupCascading(regions) {
    // Create mesh network between regional SFUs
    for (const regionA of regions) {
      for (const regionB of regions) {
        if (regionA !== regionB) {
          await this.createMeshConnection(regionA, regionB);
        }
      }
    }
  }

  async routeParticipant(participant) {
    // Connect to nearest regional SFU
    const nearestRegion = await this.findNearestRegion(participant);
    const regionalSFU = this.regions.get(nearestRegion);

    await regionalSFU.connect(participant);

    // Setup cascading for cross-region communication
    await this.setupParticipantCascading(participant, nearestRegion);
  }
}

Conclusion

SFU architecture has become the de facto standard for modern WebRTC applications due to its excellent balance of scalability, cost-effectiveness, and flexibility. While it requires more client-side processing than MCU, the benefits in terms of server efficiency and scalability make it the preferred choice for most real-time communication applications.

The key to successful SFU implementation lies in understanding the trade-offs between server and client resources, implementing appropriate optimizations like simulcast and selective forwarding, and choosing the right deployment strategy for your specific use case.