Video Quality, Latency and Lag Guide

Stream provides a detailed analytics report for all calls. If you encounter any issues with latency, quality of the video, or lag, be sure to reach out to support. Contact support and send them your API key and call ID together with a description of your issue.

This guide will go into more detail about how to optimize quality and latency of your video. To start, let’s recap how Stream provides a high-quality video service.

How Stream Ensures Quality Video

Video Edge Network: We run an edge network of servers around the world. By having your users closer to our infrastructure, we reduce lag and, more importantly, reduce packet loss. This improves the quality of video.

Dynascale: Stream will automatically tell SDKs to start uploading different codecs or different resolutions of video depending on how the video is used. If you’re showing the video in a small thumbnail screen, it will automatically select the lower quality. And if you switch back to displaying it full screen, it will switch back to high quality.

Codecs: We dynamically switch between AV1, VP9, H.264, and VP8 based on the hardware that’s connected to the call.

SDKs & Reconnects: We implement a fast failover and reconnect protocol. So if your connection breaks, the reconnect should happen very quickly.

Video Quality and Resolution

The max target quality is configured at the call type level. You can edit this in the dashboard. Typically you want to target 720p or 1080p. It is possible to target higher resolutions, but often the camera, CPU for encoding, and/or bandwidth are not ready for more than 1080p.

The max target defines the ideal maximum video resolution. There are several reasons why users can receive video at lower resolutions. Let’s say you’re aiming at 1080p.

Video quality target → 1080p

Publisher: Is the publisher able to publish at 1080p? The camera, the CPU, and the network can all cause the quality to go down on the publisher side of things.
Subscriber target resolution: For the subscriber, the target changes based on what resolution the video is shown at. If the video is displayed in a small area, we will often subscribe to 25% or 50% of the full video quality.
Subscriber degradation: If the network or device CPU are not able to keep up with the current quality of video, the subscriber will ask for a lower quality video. As network or CPU conditions improve, it will try to recover to higher quality.
Codecs: Video codecs play a key role in video quality. AV1 is by far the most advanced video codec widely available and can deliver much higher video quality with the same bitrate as older codecs such as H.264 and VP8. Unfortunately, not all devices support AV1 efficiently. This means that in some cases, less efficient codecs will be used to publish video. The good news is that encoding AV1 is far more expensive than decoding AV1. We automatically allow more powerful devices to publish video using AV1 and have publishers on older devices send video in a codec that’s better supported like H.264 or VP9.

Default target resolutions

Default target resolutions by call type:

Livestream: 1080p
Default: 720p
Development: 720p

AV1 Support Explained

AV1 codec is selected automatically on calls with any of these devices:

iPhone 15 and up supports AV1 well
Galaxy S23 and up
Chrome supports AV1, Firefox 136 added support recently. Safari 17 and up supports AV1 decoding and encoding.

Min SDK versions needed for using AV1:

Android 1.3.0
iOS 1.15.0
React 1.10.0
React Native 1.8.0
Flutter 0.7.0

Video Latency and Lag

The video edge network is great at providing low latency. Often the delay is 50-150ms on receiving the video. But many customers have a livestreaming setup which goes through several steps. For instance, you can have OBS publish to RTMP, which goes to our RTMP ingress and then is published to users. Additional steps can cause delay to the video.

Configuring OBS for Low Latency

Output Settings

You can find these settings under the output section, make sure to use the Advanced mode:

Select a hardware-based encoder if possible (Typically NVIDIA NVENC)
Make sure to use an H.264 encoder
Configure your encoder for low latency
Set Rate Control to “CBR” (Constant Bitrate)
Set keyframe interval to 2s
Disable B-frames
Pick an appropriate bitrate based on your internet connection and the resolution you want to use

Advanced Settings

Network

Set Bind to IP if you have multiple network interfaces
Enable Dynamically change bitrate when dropping frames
Set Network Buffer to 0ms (minimizes buffering)
Disable Optimize Network Usage

Video

Lower your output resolution if needed (1080p is often a good balance)
Use 30 or 60 FPS depending on your needs (if unsure, pick 30fps)
Set Common FPS Values instead of using fractional FPS
If possible, use the same resolution for the canvas and the output

Bitrate Selection

As a rule of thumb, you want your internet connection to have at least twice the upload speed that you configure on OBS as the bitrate.

When streaming content with frequent scene changes, fast motion (like sports, action games, or FPS games), you should aim for the higher end of these ranges or even exceed them slightly if your connection allows.

720p 30fps

Standard content: 2,500-4,000 Kbps
High motion content: 4,000-5,000 Kbps

1080p 30fps

Standard content: 4,500-6,000 Kbps
High motion content: 6,000-8,000 Kbps

If you use 60fps, you will need to increase the bitrate 1.5x to 2x higher than the one needed for 30fps.

RTMP Delay Explained

Our RTMP ingress is configured for low latency. It typically adds between 1 and 2 seconds of delay to convert the video. If you can use a WebRTC or WHIP type of ingress, this will remove one step from the publishing pipeline and reduce latency. WHIP is going to be generally available in March 2025.

SRT Ingress

Similar to RTMP, using SRT can cause a delay of 1 or 2 seconds. Using WebRTC or WHIP reduces these delays. Note: SRT ingress is going to be generally available in March 2025.

Video at 4K

Publishing at 4K video resolution is heavy in terms of encoding the video and requires a camera that can capture at this resolution. Most consumer cameras do not support capturing video above 2K resolution.

Recommended Specs:

GPU: NVIDIA RTX 4080 and up are typically great at low latency AV1 encoding with 4K resolution.
Network: 50Mbps upload is recommended to support an AV1 upload.

In general, the configuration is more complex, so if you are interested in using 4K for your calls, we recommend reaching out to our support team to get more guidance.

Backstage

OpenAI Realtime