Voice Visualizations

When users talk to your agent instead of typing, AIVoiceGlow and AIVoiceWave give the agent a visual presence. AIVoiceGlow is a soft globe of drifting color, and AIVoiceWave is a single waveform line. Both show what the agent is doing through the way they move. Pass the agent's state:

AIVoiceGlow(
    state = AIVoiceAgentState.Listening,
    modifier = Modifier.size(320.dp),
)

The Voice agent use case builds a whole voice screen around them: a call with your Stream Agents agent, the globe driven by the agent's state, and what was said written out as a conversation.

States

The state is an AIVoiceAgentState:

The states change only the motion, never the colors, and a change of state eases from one motion into the next. Set the state straight from your agent's events, without animating it yourself.

Neither view has a size of its own: each draws in the middle of the space it's given, on a transparent background. The globe is about half as wide as the shorter side of that space, and the line about 80% as wide, so give either view a square, or the wave a wide box.

Parameters

Both views take the same parameters:

• state - what the agent is doing.
• colors - one to five colors to paint it with. Colors past the fifth are ignored, and an empty list falls back to the default. See Colors.
• expressivity - how lively the motion is, from 0 (nearly still) through 1 (the default) to 2 (twice as animated). It never changes the colors.
• audioLevel - a function returning the live level of the voice being heard, from 0 to 1, or null. Leave it out, and the view animates speech on its own.

AIVoiceGlow also takes a rendering, described in Touch, accessibility and rendering.

Follow the real voice

To drive the motion from the agent's voice, pass audioLevel. The view calls it once a frame, so reading a value that changes many times a second recomposes nothing. Update the level at least 15 times a second for the motion to follow the words. The glow follows the level closely, while the wave smooths it, so the line swells with each phrase rather than jittering with every syllable.

With Stream Video, that's the audioLevel of the agent as a participant of the call:

val participants by call.state.remoteParticipants.collectAsState()
val agent = participants.firstOrNull { it.userId.value == agentId }

AIVoiceWave(
    state = agentState,
    modifier = Modifier.size(320.dp),
    colors = listOf(Color(0xFF7C4DFF), Color(0xFFFF4081), Color(0xFF1DE9B6)),
    expressivity = 1.4f,
    audioLevel = { agent?.audioLevel?.value },
)

A participant's audioLevel updates a few times a second, which moves the view with each phrase. For motion that follows every word, measure the agent's audio track yourself, as the Android assistant sample does.

Colors

The default palette, AIVoiceVisualizationDefaults.colors(), is the theme's primary color. To use the same palette everywhere, keep it in one place and pass it as colors to each view:

val VoicePalette: List<Color> = listOf(Color(0xFF3F51B5), Color(0xFFE91E63), Color(0xFF009688))

AIVoiceGlow(state = agentState, colors = VoicePalette, modifier = Modifier.size(320.dp))

Touch, accessibility and rendering

• Tapping the globe or the line bounces it, and the shader glow and the wave also ripple from the touch and under a hovering mouse or stylus. The views don't consume the touches, so a clickable on them or on a parent still works.
• With the system's animations turned off, the views barely move and ignore touches.
• TalkBack reads either view as "Voice agent", with the state as its value, such as "Listening". These texts are the stream_ai_compose_voice_* string resources, which you can override in your app's strings.xml.
• AIVoiceGlow takes a rendering. The default, AIVoiceGlowRendering.Auto, draws with a shader on Android 13 and later, unless the device is in battery saver, and with a few Canvas gradients otherwise. Shader and Canvas force one or the other (Shader still falls back to Canvas below Android 13). The Canvas glow is cheaper, but has no marbling or ripples. The wave always draws on a Canvas.