Voice visualizations

When users talk to your agent instead of typing, VoiceGlow and VoiceWave give the agent a visual presence. VoiceGlow is a soft globe of drifting color, and VoiceWave is a single waveform line. Both show what the agent is doing through the way they move. Pass the agent's state:

import { VoiceGlow } from "stream-chat-react/ai-components";

<VoiceGlow state="listening" style={{ width: "20rem", height: "20rem" }} />;

The Voice agent use case builds a whole voice screen around them: a call with your agent, the globe driven by the agent's state and voice, and what was said written out as a conversation. The Voice agent visuals cookbook adds them to a chat app.

States

The state is a VoiceAgentState:

The states change only the motion, never the colors, and a change of state eases from one motion into the next. Set the state straight from your agent's events, without animating it yourself.

Neither component has a size of its own: each draws in the middle of the box you give it, on a transparent background. The globe is about half as wide as the box's shorter side, and the line about 80% as wide, so give the glow a square and the wave a square or a wide box. Without a size of yours, the stylesheet makes the glow a 12rem square, and the wave 100% wide and 8rem tall:

.voice-panel .str-chat__ai-voice-wave {
  height: 6rem;
}

Props

Both components take the same props:

VoiceGlow also takes rendering. See Draw the glow with CSS.

Any other prop of a div, such as className, style or an event handler, goes to the root element.

Follow the real voice

To drive the motion from the agent's voice, pass audioLevel a function that returns the live level. The component reads it once a frame, so a level that changes many times a second re-renders nothing. Keep it up to date at least 15 times a second to follow the words.

useAudioLevel makes that function from a MediaStream, such as the user's microphone or the agent's audio in a call:

useAudioLevel(stream: MediaStream | null | undefined): () => number | null

The function it returns is stable, and measures the level each time it's called. It returns null while there is no stream or the browser has no Web Audio, so the component animates speech on its own. The hook releases its audio graph when the stream changes and on unmount, and leaves the stream's tracks running, since they're yours.

With Stream Video, that's the audioStream of the agent as a participant of the call. Participant state explains how to read the participants:

import type { StreamVideoParticipant } from "@stream-io/video-react-sdk";
import {
  useAudioLevel,
  VoiceWave,
  type VoiceAgentState,
} from "stream-chat-react/ai-components";

export const AgentWave = ({
  agent,
  state,
}: {
  agent?: StreamVideoParticipant;
  state: VoiceAgentState;
}) => {
  const agentLevel = useAudioLevel(agent?.audioStream);

  return (
    <VoiceWave
      audioLevel={agentLevel}
      colors={["#7c3aed", "#ec4899", "#14b8a6"]}
      expressivity={1.4}
      state={state}
      style={{ height: "6rem" }}
    />
  );
};

To show the user's voice while the agent listens, pass the microphone's stream instead, such as the one from navigator.mediaDevices.getUserMedia({ audio: true }). Stop its tracks yourself when you're done with it.

Colors

Without colors, the components use --str-chat__ai-voice-color, which follows the SDK's accent color. To use the same palette everywhere, set it once in CSS instead of passing colors to each component:

.voice-panel {
  --str-chat__ai-voice-color: #7c3aed;
  --str-chat__ai-voice-backdrop: #0b1020;
}

The components also pick a light or a dark look from --str-chat__ai-voice-backdrop, which defaults to the SDK's page background. Set it when they sit on a surface whose color differs from the page's. They read both about once a second, and when the system switches between light and dark, so a theme switch needs no prop. Theming lists the other variables.

Pointer, accessibility and rendering

• Clicking the globe or the line bounces it, and a hovering mouse or pen ripples it. Only the shape takes the pointer: around it, clicks and hovers reach whatever is behind the box. Pointer events are never stopped, so your own onClick on the root or a parent still fires.
• With prefers-reduced-motion: reduce, the motion slows to a crawl, and clicks and hovers are ignored.
• The root has role="img" and an accessible name made of "Voice agent" and the state, such as "Voice agent: Speaking". The texts go through the SDK's translations, under aiComponents.voiceVisualization.voiceAgent.ariaLabel and the state words such as aiComponents.voiceVisualization.speaking.text, and an aria-label of yours replaces the name. The drawing itself is hidden from assistive technology.
• The component isn't a live region, so screen readers don't announce a new state. If users need to hear it, add a role="status" text next to it.
• The components draw on a canvas: the line in 2D, and the globe with a WebGL shader. All the globes on a page share one WebGL context, so the shader compiles once. They draw only while they're on screen and the tab is visible, and fade in once their first frame is ready, so they never flash empty.
• With server rendering, the markup renders on the server, and drawing starts after hydration.

Draw the glow with CSS

VoiceGlow's rendering sets how the globe is drawn. Either way it moves the same; only the look and the cost differ:

• "auto", the default, draws it with the WebGL shader, the richest look, with color masses marbled with noise and ripples from the pointer. It falls back to "css" when the browser can't create a WebGL context, or loses it.
• "css" composes it from CSS gradients animated only through transform and opacity, so the browser's compositor runs the motion. It keeps moving while the page is busy, such as while a long answer streams in, and runs no script per frame unless you pass audioLevel. It lacks the shader's fine marbling: a click sends out a ring instead of a ripple, and hovering does nothing. It needs @property, container query units and color-mix() (Chrome 111, Safari 16.4 and Firefox 128 or later).

<VoiceGlow
  aria-label="Assistant is thinking"
  rendering="css"
  state="thinking"
  style={{ width: "8rem", height: "8rem" }}
/>