# Voice Agent

Talking is often faster than typing, and it's the only way to use an app with busy hands. A voice agent listens to the user, answers out loud, and lets them interrupt it, like a phone call with your assistant.

>
> **Note:** **What you'll build**
>
> - A call screen that puts your agent on a call with the user
> - The agent as a glowing globe that moves with its voice
> - What was said, written out under the globe
> - Mute and hang up
> - The question a tool asks before it runs, over the call
>

**Starts from:** the [Quickstart](https://getstream.io/chat/docs/sdk/android/ai-components/), with the user signed in to `agents`.

>
> **Info:** `VoiceSession` does the hard part. It starts your agent on a call, and joins the call from the phone with Stream Video, which comes with the agents SDK. The agent's speech-to-text, model and text-to-speech run on Stream, and when the user talks over the agent, it stops and listens.
>

## Best practices

- Show what the agent is doing at every moment. A silent screen while the agent thinks feels broken.
- Drive the globe from the agent's real voice, so it swells with its words.
- Write out what was said. Users glance at it to check a name, a number or an address.
- Make hanging up obvious, so the agent stops listening.

## Give the agent a voice

Any agent can talk, with the router's default speech models. Make it from the voice template to choose its models, voice and greeting:

```sh
getstream agents init my-agent --template voice
getstream agents sync my-agent
```

The call needs the microphone. `RECORD_AUDIO` is already in your manifest, from Stream Video, but the app has to ask for it at runtime before it joins. [STT](https://getstream.io/agents/docs/stt/?lang=android) and [TTS & Voices](https://getstream.io/agents/docs/tts/?lang=android) list the models and voices.

## Start the call

`VoiceAgentScreen` asks for the microphone, starts a voice session with the agent you name, and shows the call once the agent is on it. Show it from anywhere, such as a destination of your navigation graph:

```kotlin
import android.Manifest
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import io.getstream.chat.android.ai.compose.ui.component.AIVoiceAgentState
import io.getstream.chat.android.ai.compose.ui.component.AIVoiceGlow
import io.getstream.agents.android.core.AgentsException
import io.getstream.agents.android.core.SessionOptions
import io.getstream.agents.android.core.VoiceSession

@Composable
fun VoiceAgentScreen(agent: String, onClose: () -> Unit) {
    val context = LocalContext.current
    var voice by remember { mutableStateOf<VoiceSession?>(null) }
    // The call needs the microphone, so ask for it before the agent starts.
    val microphone = rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
        if (!granted) onClose()
    }
    LaunchedEffect(Unit) { microphone.launch(Manifest.permission.RECORD_AUDIO) }

    val started = voice
    if (started != null) {
        VoiceCallScreen(started, onHangUp = onClose)
    } else {
        AIVoiceGlow(state = AIVoiceAgentState.Connecting, modifier = Modifier.size(220.dp))
        LaunchedEffect(agent) {
            try {
                // The agent joins the session's call right away.
                voice = VoiceSession.start(context, agents, SessionOptions(agent = agent))
            } catch (_: AgentsException) {
                onClose()
            }
        }
    }
}
```

## Show the call

`VoiceCallScreen` joins the call from the phone, with the microphone on, and shows the agent, what was said, and the controls. `voice.session.conversation` holds the conversation as it's said, so the transcript grows as the user and the agent talk:

```kotlin
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.CallEnd
import androidx.compose.material.icons.filled.Mic
import androidx.compose.material.icons.filled.MicOff
import io.getstream.chat.android.ai.compose.conversation.AIMessage
import io.getstream.chat.android.ai.compose.conversation.AIMessageRole
import io.getstream.chat.android.ai.compose.ui.component.AIConversation
import io.getstream.agents.android.core.Turn

@Composable
fun VoiceCallScreen(voice: VoiceSession, onHangUp: () -> Unit) {
    val scope = rememberCoroutineScope()
    val call by voice.call.collectAsStateWithLifecycle()
    val muted by voice.isMuted.collectAsStateWithLifecycle()
    val conversation by voice.session.conversation.collectAsStateWithLifecycle()

    // Joins from this device, with the microphone on and the agent on the speaker.
    LaunchedEffect(voice) { voice.join() }

    Column(Modifier.fillMaxSize(), horizontalAlignment = Alignment.CenterHorizontally) {
        val active = call
        if (active != null) {
            AgentGlow(voice.session, active)
        } else {
            AIVoiceGlow(state = AIVoiceAgentState.Connecting, modifier = Modifier.size(220.dp))
        }
        AIConversation(
            messages = conversation.turns.map(::TurnMessage),
            modifier = Modifier.weight(1f),
        )
        Row(horizontalArrangement = Arrangement.spacedBy(40.dp)) {
            FilledTonalIconButton(onClick = { voice.setMuted(!muted) }) {
                Icon(
                    if (muted) Icons.Filled.MicOff else Icons.Filled.Mic,
                    contentDescription = if (muted) "Unmute" else "Mute",
                )
            }
            FilledIconButton(
                onClick = {
                    scope.launch {
                        voice.leave()
                        onHangUp()
                    }
                },
            ) {
                Icon(Icons.Filled.CallEnd, contentDescription = "Hang up")
            }
        }
    }
}

/** A turn of the call, as the AI components show it. */
data class TurnMessage(val turn: Turn) : AIMessage {
    override val id: String = turn.id
    override val role: AIMessageRole = if (turn.isAgent) AIMessageRole.Assistant else AIMessageRole.User
    override val text: String = turn.text
}
```

`join()` doesn't throw: if it fails, for example because the user denied the microphone, the error is in `voice.failure`. `leave()` leaves the call and ends the session `start` opened.

## Show what the agent is doing

`AgentGlow` shows the agent as an `AIVoiceGlow`. The call says when the agent speaks, and how loud, and the session says when it listens or thinks:

```kotlin
import io.getstream.video.android.core.Call
import io.getstream.agents.android.core.AgentSession
import io.getstream.agents.android.core.Conversation

@Composable
fun AgentGlow(session: AgentSession, call: Call) {
    val conversation by session.conversation.collectAsStateWithLifecycle()
    // The agent is the call's other participant.
    val participants by call.state.remoteParticipants.collectAsStateWithLifecycle()
    val agent = participants.firstOrNull()
    val speaking by remember(agent) { agent?.speaking ?: MutableStateFlow(false) }.collectAsStateWithLifecycle()
    val level = remember(agent) { agent?.audioLevel ?: MutableStateFlow(0f) }

    val state = when {
        speaking -> AIVoiceAgentState.Speaking
        else -> when (conversation.state) {
            Conversation.State.Listening -> AIVoiceAgentState.Listening
            Conversation.State.Responding, is Conversation.State.Working -> AIVoiceAgentState.Thinking
            Conversation.State.Idle, Conversation.State.Ended -> AIVoiceAgentState.Idle
        }
    }
    AIVoiceGlow(
        state = state,
        modifier = Modifier.size(220.dp),
        // Read once a frame, so the glow swells with the agent's words.
        audioLevel = if (speaking) ({ level.value }) else null,
    )
}
```

The glow takes the theme's primary color unless you pass it [`colors`](https://getstream.io/chat/docs/sdk/android/ai-components/voice-visualizations/#colors), and `AIVoiceWave` can show the user's microphone the same way, from `call.state.me`. [Voice Visualizations](https://getstream.io/chat/docs/sdk/android/ai-components/voice-visualizations/) lists the options of both.

## Ask before a tool runs

A tool of your app that asks first, an `AgentTool` with an `approval`, waits in the session's `approvals` until the user answers. On a call, show the question over it with `AIToolApprovalCard`:

```kotlin
import io.getstream.chat.android.ai.compose.parts.AIToolApproval
import io.getstream.chat.android.ai.compose.ui.component.AIToolApprovalCard
import io.getstream.chat.android.ai.compose.ui.component.AIToolApprovalState
import io.getstream.agents.android.core.ToolApprovalRequest

@Composable
fun ApprovalCard(request: ToolApprovalRequest, session: AgentSession) {
    val scope = rememberCoroutineScope()
    var state by remember { mutableStateOf(AIToolApprovalState()) }
    AIToolApprovalCard(
        approval = AIToolApproval(
            title = request.approval.title,
            message = request.approval.message,
            reason = request.reason.ifEmpty { null },
        ),
        state = state,
        decide = { allowed ->
            state = AIToolApprovalState(isSending = true)
            scope.launch {
                try {
                    session.decide(request.id, allowed)
                } catch (_: AgentsException) {
                    state = AIToolApprovalState(failed = true)
                }
            }
        },
    )
}
```

Then add it to the `Column` of `VoiceCallScreen`, above the controls:

```kotlin
val approvals by voice.session.approvals.collectAsStateWithLifecycle()

approvals.firstOrNull()?.let { request ->
    // Keyed by the call, so the next question starts with no answer sending.
    key(request.id) { ApprovalCard(request, voice.session) }
}
```

## Show the agent the camera

An agent with the `vision` skill can look through the phone's camera. Join with the camera on with `voice.join(camera = true)`, or turn it on during the call with `voice.setCameraEnabled(true)`. The camera starts on the back lens, and the agent sees what the call publishes. Ask for `CAMERA` first:

```kotlin
import androidx.compose.material.icons.filled.Videocam

@Composable
fun CameraButton(voice: VoiceSession) {
    // The camera needs its permission before it turns on.
    val camera = rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
        if (granted) voice.setCameraEnabled(true)
    }
    FilledTonalIconButton(onClick = { camera.launch(Manifest.permission.CAMERA) }) {
        Icon(Icons.Filled.Videocam, contentDescription = "Show the agent the camera")
    }
}
```

[Video](https://getstream.io/agents/docs/video/?lang=android) covers what the agent sees.

## Next steps

- [Personal assistant](https://getstream.io/chat/docs/sdk/android/ai-components/use-cases/personal-assistant/#talk-instead-of-typing): switch a typed conversation to voice and back.
- [Sessions](https://getstream.io/agents/docs/sessions/?lang=android#voice-ui-transcript-captions-and-status): the voice UI's states and how a voice session ends.
- [Purchase a phone number](https://getstream.io/agents/docs/telephony/purchase-number/?lang=android): the same agent on the phone.

---

For the most recent version of this documentation, visit [https://getstream.io/chat/docs/sdk/android/ai-components/use-cases/voice-agent/](https://getstream.io/chat/docs/sdk/android/ai-components/use-cases/voice-agent/).