Voice Visualizations

When users talk to your agent instead of typing, AIVoiceGlowView and AIVoiceWaveView give the agent a visual presence. AIVoiceGlowView is a soft globe of drifting color, and AIVoiceWaveView is a single waveform line. Both show what the agent is doing through the way they move. Pass the agent's state:

AIVoiceGlowView(state: .listening)
    .frame(width: 320, height: 320)

The Voice agent use case builds a whole voice screen around them: a call with your Stream Agents agent, the globe driven by the agent's state and voice, and what was said written out as a conversation.

States

The state is an AIVoiceAgentState:

The states change only the motion, never the colors, and a change of state eases from one motion into the next. Set the state straight from your agent's events, without animating it yourself.

Neither view has a size of its own: each fills the space it's given, on a transparent background. The globe is about half as wide as the shorter side of that space, and the line about 80% as wide, so give either view a square frame.

Parameters

Both views take the same parameters:

• state - what the agent is doing.
• colors - one to five colors to paint it with. Colors past the fifth are ignored. Leave it nil to use the palette of the appearance. The colors keep their full hue, saturation and brightness, so muted brand colors render as they are, and dynamic colors follow light and dark mode.
• expressivity - how lively the motion is, from 0 (nearly still) through 1 (the default) to 2 (twice as animated). It never changes the colors.
• audioLevel - the live level of the voice being heard, from 0 to 1. Leave it nil, and the view animates speech on its own.

Follow the real voice

To drive the motion from the agent's voice, pass a live audioLevel, updated at least 15 times a second. Live audio lifts the motion to its level, whatever the state. The glow follows the level closely, while the wave smooths it, so the line swells with each phrase rather than jittering with every syllable.

With Stream Video, that's the audioLevel of the agent as a participant of the call:

let agent = call.state.remoteParticipants.first { $0.user.id == agentId }

AIVoiceWaveView(
    state: agentState,
    colors: [.purple, .pink, .mint],
    expressivity: 1.4,
    audioLevel: agent.map { Double($0.audioLevel) }
)
.frame(width: 320, height: 320)

Colors

To use the same palette everywhere, set it once on the appearance instead of passing colors to each view. It defaults to the accent color (accentPrimary), and an empty palette falls back to it:

appearance.colors.voiceVisualization = [.systemIndigo, .systemPink, .systemTeal]

Touch, accessibility and rendering

• Tapping the globe or the line bounces it and sends a ripple out from the touch, and a hovering pointer, such as an iPad trackpad, ripples it too. Taps on the empty space around the shape pass through to the views behind it.
• With Reduce Motion turned on, the views barely move and ignore taps.
• VoiceOver reads either view as "Voice agent", with the state as its value, such as "Listening". These texts go through the localization provider, under the voice.accessibility.agent and voice.state.* keys.
• The views render with Metal. The first one to appear compiles the shaders and fades in once its first frame is ready, and the views stop rendering while they're off screen or your app is in the background.