Voice Visualizations

When users talk to your agent instead of typing, VoiceGlow and VoiceWave give the agent a visual presence. VoiceGlow is a soft globe of drifting color, and VoiceWave is a single waveform line. Both show what the agent is doing through the way they move. Pass the agent's state:

import { VoiceGlow } from "stream-chat-react-native/ai-components";

<VoiceGlow state="listening" style={{ width: 240, height: 240 }} />;

The Voice agent use case builds a whole voice screen around them: the globe driven by the agent's state, the wave moved by the user's voice, and the conversation held hands-free.

States

The state is a VoiceAgentState:

The states change only the motion, never the colors, and a change of state eases from one motion into the next. Set the state straight from your agent's events, without animating it yourself.

Neither component has a size of its own: each draws in the middle of the view you give it, so size them with style. The globe is about half as wide as the view's shorter side, and the line about 80% as wide, so give the glow a square and the wave a square or a wide box:

<VoiceWave state="thinking" style={{ height: 96, width: "100%" }} />

Props

Both components take the same props:

Follow the real voice

To drive the motion from a voice, pass audioLevel a Reanimated shared value that holds the live level. The visualization reads it once a frame on the UI thread, so a level that changes many times a second re-renders nothing. Update it at 15 Hz or more to follow the words.

Follow the microphone

useDictationAudioLevel gives the microphone's live level while dictation is listening. It works with the composer's SpeechToTextButton and the useDictation hook, and needs dictation to be set up:

const useDictationAudioLevel: () => SharedValue<number | null>;

The value is null while dictation isn't listening, and the visualization then animates speech on its own. The native side measures the level about 10 to 30 times a second: on Android from the speech recognizer, and on iOS from the microphone input. Show the user's voice while they dictate:

import { Pressable } from "react-native";
import {
  useDictation,
  useDictationAudioLevel,
  useDictationState,
  VoiceWave,
} from "stream-chat-react-native/ai-components";

const DictationWave = () => {
  const { toggle } = useDictation();
  const { isRecording } = useDictationState();
  const level = useDictationAudioLevel();

  return (
    <Pressable onPress={toggle}>
      <VoiceWave
        audioLevel={level}
        state={isRecording ? "listening" : "idle"}
        style={{ height: 96, width: "100%" }}
      />
    </Pressable>
  );
};

Follow another source

For any other source, such as a level you measure from the audio of a call, keep a shared value of your own and write the level into it:

import { useSharedValue } from "react-native-reanimated";

const level = useSharedValue<number | null>(null);

// Wherever the level arrives, 15 times a second or more:
level.value = measuredLevel; // from 0 to 1

<VoiceGlow
  audioLevel={level}
  state="speaking"
  style={{ width: 240, height: 240 }}
/>;

Set it back to null when the voice stops, so the visualization goes back to animating on its own.

Colors

Without colors, the components use the theme's voiceVisualization.colors, which default to the accent color. To use the same palette everywhere, set it once on StreamTheme instead of passing colors to each component. backdropColor is the color they sit on, which tells them to draw for a light or a dark background. Without it, they follow the system color scheme:

<StreamTheme
  style={{
    voiceVisualization: {
      backdropColor: "#0b0b10",
      colors: ["#005fff", "#7b5cff", "#00c2ff"],
    },
  }}
>
  {children}
</StreamTheme>

Touch, accessibility and rendering

• It runs on the UI thread. The motion is advanced in a Reanimated frame callback and drawn by Skia, so it stays smooth while the JS thread is busy, for example while a long answer streams in.
• Only the shape takes touches. Over the string or, for VoiceGlow, the globe, a tap bounces the shape and sends out a ripple, and a hovering mouse or pen ripples it too. Touches are only observed, never claimed, so a Pressable around the visualization still gets its press. Beside the shape, touches reach whatever is behind it.
• Reduced motion. With Reduce Motion on iOS, or animations turned off on Android, the motion slows to a crawl and taps are ignored.
• It fades in. The first frame fades in over 0.3 seconds, so the component never flashes empty while it starts.
• It keeps running while mounted. Unlike on iOS and the web, it doesn't pause while scrolled off screen; it only pauses while the app is in the background. Unmount it when it isn't needed.
• The glow is a shader. VoiceGlow is drawn by a Skia shader that computes every pixel: noise-marbled color masses, concentric waves, light and a halo, with ripples from a tap or a hovering pointer. It's the same shader as Android's, and every glow shares one compiled copy. Skia runs it on every Android version, so there is no Canvas fallback and no rendering option.

The root is an image named "Voice agent", with the state as its value: "Ready", "Connecting", "Listening", "Thinking" or "Speaking", as on iOS and Android. They go through the Chat SDK's translations inside Chat, so they're translated with the rest of your app. To change them, pass stateLabels for the state values and accessibilityLabel for the name. accessibilityValue replaces the value outright. The root has no live region, so a screen reader doesn't announce a new state. If users need to hear it, show the state as text beside it.

<VoiceGlow
  accessibilityLabel="Assistant"
  state={state}
  stateLabels={{ thinking: "Working on it" }}
/>