Did you know? All Video & Audio API plans include a $100 free usage credit each month so you can build and test risk-free. View Plans ->

Stream logo
Start Coding Free
WebRTC For The Brave

Why Transcoding is a Bad Idea in WebRTC

In this lesson, you'll explore why transcoding is generally discouraged in WebRTC applications.

WebRTC, renowned for its innate capability to facilitate real-time communication, is a foundational technology for applications that demand swift audio and video transmission. Traditionally, transcoding is used to adapt media streams to the varying capabilities of devices and network conditions. Despite the robust features WebRTC brings to the table, integrating transcoding within this ecosystem presents several challenges and often turns out not to be the best choice. In this lesson, you'll explore why transcoding is generally discouraged in WebRTC applications and review more effective alternatives that align better with WebRTC's design principles.

Before we dive into the challenges of transcoding, let's clarify what transcoding involves in a WebRTC context. Media transcoding refers to the process of decoding a media stream encoded in one format and re-encoding it into another format. This might involve changing codecs (e.g., VP8 to H.264), altering bitrates, resolutions, or frame rates to accommodate different network conditions or device capabilities.

The Challenges of Transcoding in WebRTC

1. Increased Latency: The Primary Concern

In WebRTC applications, low latency is not just desirable—it's essential. The introduction of transcoding creates a significant latency bottleneck that undermines one of WebRTC's core strengths.

How Transcoding Increases Latency

  1. Additional Processing Steps: Transcoding requires:

    • Decoding the original media stream
    • Processing the decoded data
    • Re-encoding into the target format
    • Each step adds incremental delay

  2. Buffering Requirements: Transcoding typically requires buffering frames before processing, which adds further delay to the pipeline.

  3. Measurable Impact:

    • With hardware acceleration (NVENC/ASIC/dedicated hardware): Benchmarks show that an H.264→H.264 transcode of 720p30 typically adds 40-120ms one-way latency
    • With pure software transcoding on a single vCPU: Latency typically ranges from 80-200ms
    • Additional network hops or software mixing can push the total latency well above 300ms

    This additional delay is problematic for real-time applications where target latencies are already tightly constrained.

Latency Impact on User Experience

The human perception of communication becomes noticeably disrupted when latency exceeds certain thresholds. According to ITU-T G.114 recommendations for one-way latency:

  • < 150ms one-way: Considered transparent for most users
  • 150-300ms one-way: Noticeable but acceptable for most applications
  • > 300ms one-way: Creates a significant "out of sync" feeling in conversations
  • > 400ms one-way: ITU-T G.114 planning limit for acceptable voice quality
  • > 500ms one-way: Makes natural conversation extremely difficult

Transcoding often pushes WebRTC applications beyond these acceptable thresholds, degrading the user experience dramatically.

2. Quality Degradation: The Inevitable Trade-off

Every time media content is re-encoded, some quality is lost—a phenomenon known as generation loss or transcoding artifacts.

Causes of Quality Loss During Transcoding

  1. Lossy Compression: Most codecs used in WebRTC (H.264, VP8, VP9, AV1) employ lossy compression. Each encoding cycle compounds quality loss.

  2. Different Codec Architectures: Converting between fundamentally different codec architectures (e.g., VP8 to H.264) can introduce more significant artifacts than staying within the same codec family.

  3. Resolution and Color Space Transformations: Changes in resolution or color space during transcoding can result in blurring, color shifts, and loss of detail. A real-world example is Safari's HDR to SDR tone-mapping path, which requires re-encoding and introduces visible color space conversion artifacts.

Visual Examples of Quality Degradation

In practice, you'll see artifacts most clearly in:

  • Fine details such as text or small objects
  • High-frequency components like hair, grass, or complex textures
  • Gradient areas that may develop banding artifacts
  • Moving objects that can exhibit compression artifacts

3. Resource Consumption: The Scalability Bottleneck

Transcoding is computationally expensive, demanding significant hardware resources that can limit the scalability of WebRTC services.

CPU and Memory Requirements

  1. High CPU Utilization: Modern video codecs are designed to be asymmetric, with encoding requiring substantially more processing power than decoding. Transcoding involves both operations, multiplying the CPU burden.

  2. Memory Footprint: Transcoding buffers often require storing uncompressed frames temporarily, which can consume significant amounts of RAM for high-resolution video.

  3. GPU Acceleration: While GPU acceleration can help, it introduces additional complexity and hardware dependencies that may not be available in all deployment environments.

Impact on Server Costs and Scalability

For services with many concurrent users, the resource requirements can quickly become substantial:

  • A single 720p 30fps video stream might require 0.5-1 CPU core for software transcoding (x264/VP8)
  • With hardware acceleration (NVENC/QuickSync), this can drop to about 0.1-0.2 vCPU per stream
  • On Apple Silicon and newer dedicated hardware, efficiency improves even further
  • A service with 100 concurrent streams could require 10-100 CPU cores depending on hardware acceleration
  • Cloud infrastructure costs generally scale linearly with transcode load, making high-volume services economically challenging without specialized hardware

4. Complexity and Maintenance: The Hidden Costs

Implementing transcoding introduces significant complexity to WebRTC architectures, creating numerous potential failure points and maintenance challenges.

Architectural Complexity

  1. Media Server Requirements: Transcoding typically requires dedicated media servers inserted into the communication path, breaking the peer-to-peer model that makes WebRTC efficient.

  2. Stream Routing Logic: Applications must implement complex logic to determine what streams require transcoding and how to route media through transcoding servers.

  3. Format Negotiation: Managing codec preferences, capabilities, and fallbacks adds another layer of complexity to the signaling process.

Operational Challenges

  1. Monitoring and Debugging: Transcoding introduces many more potential failure points, making it more difficult to diagnose quality issues.

  2. Version Management: Codec implementations evolve over time, requiring ongoing updates to transcoding modules.

  3. Different Platform Behaviors: Various browsers and platforms implement WebRTC differently, requiring additional testing and adaptation of transcoding solutions.

Better Alternatives to Transcoding

Given the challenges associated with transcoding, WebRTC applications typically employ more efficient approaches for handling varying network conditions and device capabilities.

1. Simulcast: Multiple Qualities at Source

We've already touched on Simulcast in Module 3, but let's recap. Simulcast stands out as a compelling alternative to transcoding in WebRTC applications. The term "simulcast" is derived from "simultaneous broadcast" and refers to sending multiple concurrent signals of the same video stream simultaneously, each at a different quality and bitrate.

How Simulcast Works

  1. Multiple Encodings: The sender encodes the source video at different quality levels (e.g., 720p, 480p, 240p).

  2. Separate Transmission: Each encoded version is sent as a separate RTP stream.

  3. Receiver Selection: Receivers can select the appropriate quality based on their network conditions and device capabilities.

  4. Dynamic Switching: Quality can be changed dynamically without requiring re-negotiation.

Implementation in WebRTC

javascript
            // Example of configuring simulcast in WebRTC
const transceiver = peerConnection.addTransceiver('video', {
  direction: 'sendonly'
});

// Configure three encoding layers (high, medium, low quality)
const parameters = transceiver.sender.getParameters();
parameters.encodings = [
  { rid: 'high', maxBitrate: 900000, scaleResolutionDownBy: 1 },
  { rid: 'medium', maxBitrate: 300000, scaleResolutionDownBy: 2 },
  { rid: 'low', maxBitrate: 100000, scaleResolutionDownBy: 4 }
];

await transceiver.sender.setParameters(parameters);

// Note: For simulcast to work properly, the SDP answer must include the RID
// parameters and the offer/answer must carry a=simulcast. Switching between 
// pre-announced layers doesn't require renegotiation, but adding or removing 
// layers does require a new offer/answer cycle.

Browser Support (as of 2025)

  • Simulcast (VP8/H.264):
    • Stable in all Chromium-based browsers (Chrome, Edge, etc.)
    • Safari 17+ supports H.264 simulcast (RID-based)
  • VP9/AV1 SVC:
    • Enabled by default in Chrome/Edge; commonly used in production for screen sharing
    • Firefox support remains experimental
  • Safari SVC Support: Only temporal scalability (not spatial) exposed via WebCodecs

Advantages Over Transcoding

  • No Additional Latency: In pure P2P scenarios, no server-side processing is needed to adapt stream quality
  • SFU Efficiency: Even in multiparty calls using a Selective Forwarding Unit (SFU), the server simply forwards the chosen layer rather than re-encoding
  • Preserved Quality: Each stream is encoded only once, avoiding generation loss
  • Reduced Server Load: Processing burden is shifted to the sender's device
  • Client-Side Adaptation: Receivers can adapt to changing network conditions without server involvement

Simulcast offers a compelling solution that allows recipients to select the most suitable video stream according to their current network conditions and device capabilities without the need for intensive server-side transcoding. Here are some key benefits of using Simulcast:

  • Reduced Latency: Unlike transcoding, Simulcast removes the requirement for server-side processing, substantially reducing overall latency and improving the real-time communication experience. Latency is critical to user experience.
  • Network Adaptability: Simulcast's provision of multiple stream qualities automatically adjusts video quality in response to available bandwidth, ensuring an optimal balance between quality and network conditions.
  • Resource Efficiency: Unlike transcoding, which incurs considerable overhead and high resource consumption on the server side, Simulcast effectively shifts the task of selecting stream quality to the client side. This transition significantly lightens the server's computational burden, enhancing resource efficiency.

2. Scalable Video Coding (SVC): Layered Approach

Scalable Video Coding (SVC), first standardized as an extension for H.264 but now equally relevant for VP9 and AV1, offers a sophisticated method of encoding video streams in layers. These layers include a base layer representing the lowest video quality and additional enhancement layers progressively improving the video quality.

How SVC Works

  1. Base Layer + Enhancement Layers: Video is encoded into a base layer (lowest quality) and multiple enhancement layers that progressively improve quality.

  2. Bitstream Organization: For spatial scalability, SVC typically produces a single encoded bitstream containing all layers. However, with Chromium's K-SVC mode (optimized for screen sharing), separate SSRCs might be used for different layers.

  3. Selective Decoding: Receivers can choose how many layers to decode based on their capabilities and network conditions.

  4. Efficient Bandwidth Usage: SVC is typically more bandwidth-efficient than simulcast because enhancement layers can build upon information in the base layer.

Thus, SVC encodes raw video data into layers, enabling the file to be examined or adjusted to match different bitrates without the need for decompression. This layer-based approach eliminates the necessity for transcoding, streamlining the entire process. Here are some key benefits of using SVC:

  • Flexible Quality Selection: SVC enables receivers to dynamically adjust the number of layers they decode, facilitating real-time adaptation to changes in network conditions without the need for multiple stream versions. This flexibility greatly enhances user experiences by allowing users to select the optimal decoding levels based on their specific needs and circumstances.
  • Improved Error Resilience: The layered architecture of SVC streams significantly boosts error resilience. Even if enhancement layers are lost, the base layer remains unaffected, preserving the core video quality and ensuring a consistent viewing experience.
  • Bandwidth Efficiency: Unlike resource-intensive transcoding, which can lead to bandwidth constraints, SVC optimizes bandwidth usage by transmitting only the essential layers required to maintain the desired video quality. This approach ensures efficient use of network resources.

Implementation with VP9 and AV1

javascript
            // Example of configuring SVC with VP9 in WebRTC
const transceiver = peerConnection.addTransceiver('video', {
  direction: 'sendonly'
});

// Configure SVC with spatial and temporal layers
const parameters = transceiver.sender.getParameters();
parameters.encodings = [{
  // L2T3 means 2 spatial layers and 3 temporal layers
  scalabilityMode: 'L2T3'
}];

await transceiver.sender.setParameters(parameters);

Advantages Over Transcoding

  • Bandwidth Efficiency: SVC typically requires less total bandwidth than simulcast for the same quality options
  • Graceful Degradation: Loss of enhancement layer packets still allows for playback using the base layer
  • Fine-Grained Adaptation: Offers more granular quality adaptation than simulcast
  • No Server-Side Processing: Like simulcast, avoids the latency and resource overhead of transcoding

3. Adaptive Bitrate Switching

Even without simulcast or SVC, WebRTC includes mechanisms for adapting to changing network conditions by dynamically adjusting encoding parameters.

How Adaptive Bitrate Works in WebRTC

  1. Bandwidth Estimation: WebRTC continuously estimates available bandwidth using techniques like REMB (Receiver Estimated Maximum Bitrate) and Transport-CC.

  2. Encoder Parameter Adjustment: Based on bandwidth estimates, the sender can adjust encoder parameters like bitrate, framerate, and resolution.

  3. Real-time Adaptation: These adjustments happen in real-time without requiring session renegotiation.

Implementation Example

javascript
            // Example of implementing adaptive bitrate control
const sender = peerConnection.getSenders()[0]; // Video sender

// Set initial encoding parameters
let encodingParams = sender.getParameters();
encodingParams.encodings[0].maxBitrate = 1000000; // 1 Mbps
await sender.setParameters(encodingParams);

// Adjust parameters based on network conditions
peerConnection.addEventListener('connectionstatechange', async () => {
  if (peerConnection.connectionState === 'connected') {
    // Get stats to estimate available bandwidth
    const stats = await peerConnection.getStats(sender);
    let availableBandwidth = 0;

    stats.forEach(report => {
      if (report.type === 'outbound-rtp' && report.kind === 'video') {
        availableBandwidth = report.availableOutgoingBitrate;
      }
    });

    // Adjust encoding parameters based on available bandwidth
    if (availableBandwidth > 0) {
      const newParams = sender.getParameters();
      // Set to 80% of available bandwidth
      newParams.encodings[0].maxBitrate = Math.floor(availableBandwidth * 0.8);
      await sender.setParameters(newParams);
    }
  }
});

Selective Use of Transcoding: When It Makes Sense

While generally discouraged, there are specific scenarios where transcoding might be justified in WebRTC applications:

Recording and Archiving

When permanent storage of WebRTC sessions is required, transcoding may be necessary to:

  • Convert to formats better suited for storage (e.g., MP4 with H.264)
  • Reduce storage requirements through higher compression
  • Create standardized archives compatible with various playback systems

Broadcasting to Non-WebRTC Platforms

When broadcasting WebRTC streams to CDNs, social media, or traditional broadcasting systems, transcoding may be required to:

  • Convert to formats compatible with HLS, DASH, or RTMP delivery
  • Comply with platform-specific requirements (e.g., YouTube, Facebook Live)
  • Optimize for one-to-many distribution rather than real-time communication

Legacy System Integration

When integrating with legacy video conferencing or telephony systems, transcoding might be necessary to:

  • Bridge between different codec ecosystems (e.g., WebRTC to SIP)
  • Support hardware endpoints with limited codec capabilities
  • Maintain compatibility with older standards

Best Practices for Minimizing Transcoding Impact

When transcoding cannot be avoided, several strategies can help minimize its negative effects:

1. Edge Transcoding

Position transcoding servers as close as possible to the content origin to minimize network latency:

  • Use edge computing services in multiple geographic regions
  • Consider WebAssembly-based transcoding for browser-side processing where appropriate
  • Implement intelligent routing to minimize hop counts

2. Hardware Acceleration

Leverage hardware acceleration to reduce the CPU burden and improve transcoding efficiency:

  • GPU-based encoding/decoding (NVENC, QuickSync, etc.)
  • FPGA or ASIC solutions for high-volume applications
  • Cloud instances with dedicated media processing hardware

3. Adaptive Transcoding Policies

Implement smart policies to minimize unnecessary transcoding:

  • Only transcode when absolutely required by receiver capabilities
  • Use progressive enhancement approaches for non-critical applications
  • Consider hybrid approaches that combine simulcast/SVC with selective transcoding

Conclusion

Throughout this lesson, you've explored the drawbacks of transcoding and two alternative strategies: Simulcast and Scalable Video Coding (SVC).

For scenarios characterized by a wide variation in client device processing power and network conditions, Simulcast is often the preferred choice due to its simplicity and extensive compatibility. Conversely, SVC stands out in contexts requiring high-quality video alongside efficient bandwidth utilization, provided there's infrastructure in place to handle its more sophisticated stream processing requirements.

It's crucial to evaluate which strategy aligns best with your service's needs, as making the right decision can significantly enhance user experiences in your application.

Additional Resources