How Does Echo Cancellation Work?

TL;DR

• Echo happens because every device on a call plays and records audio at once; the delay added by network transit and buffering is what makes the overlap audible instead of unnoticeable.
• An adaptive filter (commonly NLMS) predicts the echo from a reference signal and subtracts that prediction from the microphone signal. That subtraction is what separates cancellation from suppression.
• Browsers handle this by default: Chrome's AEC3 canceller cut echo incidents by roughly 80% after its 2019 rollout, and it stays on unless you turn it off in your getUserMedia constraints.

When video calls first started taking off, it wasn't uncommon to join a call on a laptop with no headphones and for the person on the other end to start hearing their own voice, delayed by half a second. A simple echo: their voice came out of your speaker, crossed the room, went back in through your microphone, and got sent right back to them.

This is a problem every product with real-time audio has to deal with. And does. Because of echo cancellation, you rarely come across this issue with voice and video calls anymore.

Echo cancellation runs by default in every WebRTC call, and most of the time you don't have to think about it. The times you do are the times it fails, so it's worth knowing how it works, where it breaks, and what you can do when users report echo anyway.

Why Do Calls Echo in the First Place?

Every device in a call plays audio and records audio at the same time. The remote participant's voice comes out of your loudspeaker, bounces around the room, and arrives back at your microphone mixed with your own voice. Your device can't tell those two apart on its own, so it transmits the mixture, and the remote participant hears themselves again.

Delay is what turns that into a problem. A reflection that arrives within a few tens of milliseconds blends into the conversation unnoticed. A real-time call adds network transit, jitter buffers, and audio buffering on both ends, so the reflection comes back hundreds of milliseconds late. At that delay, hearing your own voice is genuinely disorienting, and most people can't keep talking through it.

How Does Acoustic Echo Cancellation Work?

The device already knows exactly what it's about to play. That signal is called the reference. The job is to work out what the speaker, the room, and the microphone did to the reference, synthesize the echo that produced, and subtract it from the capture before anything leaves the device.

The tool for that job is an adaptive filter: a running model of the echo path that starts blank. It predicts, from the recent reference, what echo should be arriving at the microphone right now, subtracts that prediction from the capture, and treats whatever remains as its mistake. Each mistake nudges the model closer to the real echo path, and given far-end speech to learn from, it converges on the room within a second or two.

Subtracting a model is what separates cancellation from suppression, two terms that get mixed up:

• A suppressor attenuates one direction of audio whenever the other is active, which forces the call into half duplex, one side at a time
• A canceller leaves the near-end voice untouched so that both people can speak at once

The adaptation step itself has a standard algorithm: NLMS, normalized least mean squares. After every sample, each coefficient gets nudged in proportion to the error times the reference sample it multiplies. The normalization divides that step by the energy of the recent reference, so the filter learns at the same rate whether the far end whispers or shouts. The step size, mu, sets the tradeoff: bigger converges faster, smaller tracks more steadily once converged.

In code, the core of the canceller is a short loop:

1
2
3
4
5
6
w = np.zeros(L)                     # the echo path model, L taps
for i in range(L, n):
    x = far[i - L:i][::-1]          # last L reference samples
    y_hat = w @ x                   # predicted echo
    out[i] = mic[i] - y_hat         # what goes to the network
    w += (mu / (x @ x + eps)) * out[i] * x   # correct the model

Let's say you're on a call. For the first few seconds, you just listen, so everything reaching your microphone is the other person's voice out of your speaker, and the filter has clean material to learn the room from. Then you start talking over them. A little while after you stop, you pick the laptop up and carry it to the couch.

That's the scenario here, with simulated speech standing in for the voices. The filter runs twice: once adapting continuously, and once freezing whenever it suspects you're talking. This is the microphone signal and both cancellers' outputs. The middle panel is the run that never stops adapting.

Here is an ERLE (echo return loss enhancement) plot of the call. The ratio of echo power in to echo power out. Higher means more echo removed.

The first stretch goes well in both runs. Within a couple of seconds, the model matches the room, and the echo drops by about 25 dB, a factor of a few hundred.

Your interruption is where they split. In the run that keeps adapting, your voice pours into the error signal, the filter mistakes it for echo it failed to predict, and it rewrites its model trying to cancel you. That wrecks the model. for a stretch its output carries more echo than the raw microphone, and once you stop talking it has to relearn the room from scratch. The frozen run just waits, keeps its model, and picks up where it left off.

The move to the couch hits both runs the same way. The model now describes a room that no longer exists, echo leaks through, and the filter spends a second or two re-converging. That dip is the moment of echo people hear on real calls when someone repositions a device.

The freeze needs a trigger: something has to notice you've started talking. The classic cheap test is the Geigel detector:

1
2
3
4
5
6
if abs(mic[i]) > 0.4 * np.max(np.abs(far[i - L:i])):
    hold = hangover          # double-talk suspected: freeze
elif hold > 0:
    hold -= 1                # count down the hangover
if hold == 0:
    w += (mu / (x @ x + eps)) * out[i] * x

Sound loses energy on the trip from speaker to microphone, so echo is always a quieter copy of what your device played. A microphone sample near half the recent reference peak would mean the room amplified the signal. It can't be echo, so it must be you, and adaptation freezes.

The hangover keeps it frozen through the short pauses in your speech so it doesn't restart mid-sentence. Production cancellers use subtler tests built on how well the capture correlates with the reference, but the structure is the same.

What Does WebRTC Do About Echo?

Browsers handle most of this for you. Every WebRTC call runs the captured audio through a processing pipeline, and echo cancellation sits at the center of it. Chrome's current canceller, AEC3, reached all desktop users in early 2019 after a multi-year rebuild, and Google reported roughly 80% fewer echo incidents than the previous generation.

Under the hood, AEC3 is the same design as the last section, run in the frequency domain instead of sample by sample. A delay estimator keeps the reference lined up with the capture as buffers drift. An adaptive filter takes out the linear part of the echo; a suppressor and comfort noise cover the rest, and each frequency band adapts at its own pace, which suits how unevenly speech spreads its energy.

The order is deliberate. Gain control ahead of the canceller would amplify the exact echo it's trying to model, and noise suppression ahead of it would distort the relationship between reference and capture that adaptation depends on.

You control all of this through getUserMedia constraints, and echo cancellation is on unless you turn it off:

1
2
3
4
5
6
7
8
9
10
const stream = await navigator.mediaDevices.getUserMedia({
  audio: {
    echoCancellation: true,   // the default
    noiseSuppression: true,
    autoGainControl: true,
  },
});

// check what was actually applied
stream.getAudioTracks()[0].getSettings().echoCancellation;

Constraints are requests rather than guarantees, which is why checking getSettings() is worth the extra line.

There's one limit to keep in mind: the canceller can only subtract audio it received as reference. Sound your app plays outside that path, through a separate audio element, a Web Audio graph, or shared screen audio, never reaches the filter. The microphone picks it up anyway, and it goes back out as echo while the canceller works exactly as designed.

Why Can Users Still Hear Echo When Cancellation Is On?

Almost every echo report lands in one of a few buckets:

• Audio outside the reference signal. Screen sharing with sound, media played through a separate element or Web Audio, virtual audio devices, soundboards. The canceller never sees that audio, so it can't remove it.
• Bluetooth. Wireless audio adds latency that shifts during a call, which breaks the delay alignment the filter depends on. Many headsets also run their own echo and noise processing on the earpiece, and two cancellers stacked on one signal produce odd artifacts.
• Two devices in one room. Each device cancels its own playback, not the sound arriving from the laptop beside it. Either one participant per room stays unmuted, or everyone wears headphones.
• Loud speakerphone. Small speakers driven hard distort, and distortion sits outside what a linear filter can model. The residual suppressor compensates by turning aggressive, which is why loud speakerphone calls go choppy, with each side cutting the other off.
• The echo path changed. Moving the device, changing volume, or switching outputs invalidates the model, and echo leaks for a second or two while it re-converges. Echo that leaks constantly points at a delay estimate that never stabilizes, common on low-end Android hardware.

Headphones sidestep all of it by removing the acoustic path. With no speaker feeding the microphone, there's nothing to cancel, which is why they stay the first suggestion in every troubleshooting guide, including this one.

How Does Echo Cancellation Work in Stream?

Stream's SDKs run on the WebRTC audio pipeline described above, so every call gets echo cancellation, noise suppression, and gain control without any configuration. There's no switch to hunt for, and for standard voice and video calls you shouldn't need one. Device selection and the rest of capture behavior live in the microphone and camera APIs.

The exception is HiFi mode. For live music, karaoke, podcasts, and other cases where processed audio defeats the purpose, Stream's high-fidelity audio switches to stereo capture at a studio bitrate and turns audio processing off, including echo cancellation:

1
2
3
4
5
6
import { SfuModels } from "@stream-io/video-client";

// set before joining the call
await call.microphone.setAudioBitrateProfile(
  SfuModels.AudioBitrateProfile.MUSIC_HIGH_QUALITY,
);

HiFi is allowed by default on the livestream call type, and you can enable it for other call types in the dashboard. It makes headphones close to mandatory for participants. Stream's docs say directly that without headphones or a dedicated audio interface, users in this mode should expect echo or feedback, because nothing remains in the pipeline to stop it.

One last distinction saves support time. Stream's noise cancellation, powered by Krisp, removes background noise from the microphone. It runs alongside echo cancellation and solves a different problem. Enabling it won't fix an echo complaint, and echo cancellation won't quiet a barking dog. When a user reports echo, work down the buckets in the previous section, starting with whatever audio is playing outside the call.