Conversation State

AIConversation shows a conversation, and ChatComposer takes what the user writes. Between them and your agent sits the state of the conversation: its messages, whether the agent is thinking or writing, the message on its way, and why the last request failed. Your app holds that state, usually in a ViewModel, and passes it to both views. With Stream Chat, the conversation is a channel, so most of the state is the channel's, which Stream Chat keeps up to date:

@Composable
fun AssistantScreen(viewModel: AssistantViewModel) {
    val messages by viewModel.messages.collectAsStateWithLifecycle()
    val hasOlderMessages by viewModel.hasOlderMessages.collectAsStateWithLifecycle()
    val indicator by viewModel.indicator.collectAsStateWithLifecycle()

    Column(modifier = Modifier.fillMaxSize()) {
        AIConversation(
            messages = messages,
            modifier = Modifier.weight(1f),
            // The agent is on it, and its reply isn't in the channel yet.
            isThinking = indicator != null && messages.lastOrNull()?.role != AIMessageRole.Assistant,
            loadOlderMessages = if (hasOlderMessages) viewModel::loadOlderMessages else null,
        )
        ChatComposer(
            onSendClick = { data -> viewModel.send(data.text) },
            onStopClick = viewModel::stop,
            isGenerating = indicator != null,
        )
    }
}

AssistantViewModel is written below. With these two views and that state, the conversation:

• Shows the agent thinking until its reply starts, and the reply as it streams.
• Turns the composer's send button into a stop button while the agent answers, and stops the agent when it's tapped.
• Loads older messages as the user scrolls up.

The Conversation View guide adds the rest: a message on its way, reply actions and editing a message.

Make a conversation

With a backend of your own, the ViewModel watches the channel, maps its messages to AIMessages, follows what the agent says it's doing, and sends what the user asks as a message in the channel. StreamAIMessage is the message type from Messages from Stream Chat:

@OptIn(ExperimentalCoroutinesApi::class)
class AssistantViewModel(private val cid: String, currentUserId: String) : ViewModel() {

    private val channel = ChatClient.instance().channel(cid)
    private val channelState = ChatClient.instance()
        .watchChannelAsState(cid = cid, messageLimit = 30, coroutineScope = viewModelScope)

    /** The conversation, oldest message first. */
    val messages: StateFlow<List<AIMessage>> = channelState
        .filterNotNull()
        .flatMapLatest { it.messages }
        .map { messages -> messages.map { StreamAIMessage(it, currentUserId) } }
        .stateIn(viewModelScope, SharingStarted.WhileSubscribed(5_000), emptyList())

    /** Whether the channel has messages before the first one loaded. */
    val hasOlderMessages: StateFlow<Boolean> = channelState
        .filterNotNull()
        .flatMapLatest { it.endOfOlderMessages }
        .map { end -> !end }
        .stateIn(viewModelScope, SharingStarted.WhileSubscribed(5_000), false)

    // What the agent says it's doing, or null while it's idle.
    private val _indicator = MutableStateFlow<AIIndicatorUpdatedEvent?>(null)
    val indicator: StateFlow<AIIndicatorUpdatedEvent?> = _indicator.asStateFlow()

    // Why the latest message couldn't be sent, or null.
    private val _error = MutableStateFlow<String?>(null)
    val error: StateFlow<String?> = _error.asStateFlow()

    // The listener runs on a background thread, which is fine for a StateFlow.
    private val subscription: Disposable = channel.subscribeFor(
        AIIndicatorUpdatedEvent::class.java,
        AIIndicatorClearEvent::class.java,
    ) { event ->
        _indicator.value = when (event) {
            // The Stream backend SDKs send no ai_indicator.clear after an error, so reset here.
            is AIIndicatorUpdatedEvent -> event.takeUnless { it.aiState == "AI_STATE_ERROR" }
            else -> null
        }
    }

    fun send(text: String) {
        _error.value = null
        viewModelScope.launch {
            channel.sendMessage(Message(text = text)).await()
                .onError { error -> _error.value = error.message }
        }
    }

    /** Stops the reply in flight. See "Stop the reply" below. */
    fun stop() {
        val indicator = _indicator.value ?: return
        channel.sendEvent(
            eventType = EventType.AI_TYPING_INDICATOR_STOP,
            extraData = mapOf("message_id" to indicator.messageId),
        ).enqueue()
        _indicator.value = null
    }

    suspend fun loadOlderMessages() {
        ChatClient.instance().loadOlderMessages(cid, messageLimit = 30).await()
    }

    override fun onCleared() {
        subscription.dispose()
    }
}

The channel's state lists its messages oldest first, which is the order the conversation expects. subscribeFor on the channel delivers only the channel's events, but the subscription keeps running until you call dispose(), so the view model does that in onCleared(). The subscribeFor overloads that take a LifecycleOwner dispose it for you, but they also drop the events that arrive while the screen is stopped: if ai_indicator.clear is one of them, the indicator stays on.

isGenerating disables the composer's text field, so make sure the indicator always resets, as the view model does on ai_indicator.clear, on AI_STATE_ERROR, and when the user stops the reply.

With Stream Agents

With Stream Agents, the agent's session gives you its channel, and you ask the session rather than send to the channel:

The Quickstart builds the whole conversation this way, as an AgentChat.

Show what the agent is doing and errors

The conversation's footer shows under the latest turn's messages, and emptyContent while there are none. Show what the agent says it's doing there, and an error with a way to try again:

val indicator by viewModel.indicator.collectAsStateWithLifecycle()
val error by viewModel.error.collectAsStateWithLifecycle()

AIConversation(
    messages = messages,
    footer = {
        val label = when (indicator?.aiState) {
            "AI_STATE_EXTERNAL_SOURCES" -> "Checking external sources"
            else -> null
        }
        if (label != null) {
            AITypingIndicator(label = { Text(label) })
        }
        error?.let { message ->
            Row(verticalAlignment = Alignment.CenterVertically) {
                Text(message)
                TextButton(onClick = { viewModel.send(lastQuestion) }) { Text("Try again") }
            }
        }
    },
    emptyContent = {
        Text("How can I help?")
    },
)

While the agent only thinks, the conversation shows "Thinking... 4s" by itself, as described in Show the agent thinking. An error is one of two kinds:

• The message couldn't be sent. sendMessage fails, and the view model keeps why.
• The agent couldn't answer. It reports AI_STATE_ERROR, and writes what went wrong into its reply, as the Stream backend SDKs do.

What your backend sends

With Stream Chat, your backend writes the agent's reply into the channel, and the conversation shows what it writes. A typical backend posts an empty placeholder message as the agent, updates it as the model streams, and stores the final text. Stream Agents does all of this for you. With a backend of your own, follow the conventions below, which the components and Stream's backend SDKs share. The AI message streaming guide covers the whole backend flow.

Mark the agent's replies

Set two custom fields on the agent's reply:

• ai_generated: true, when the backend creates the placeholder. Your app reads it to tell the agent's replies apart, for example for your message's role, and your backend checks it to skip the agent's own replies instead of answering them.
• generating, true on every update while the reply streams, and false on the final one. Read it as your message's isGenerating. A reply types itself out only while it's true, so a stored reply, saved with false, doesn't type itself out again when the user scrolls back to it.

const { message } = await channel.sendMessage({
  text: "",
  user_id: botId,
  ai_generated: true,
});

placeholder = channel.send_message(
    MessageRequest(text="", user_id=bot_id, custom={"ai_generated": True}),
).data.message

The Android SDK has no dedicated property for them. Like any custom field, they end up in the message's extra data, as message.extraData["ai_generated"] and message.extraData["generating"].

Say what the agent is doing

Before the first word of the reply arrives, the model can spend a while thinking or calling tools, and the message is still empty. The backend says what it's doing with ai_indicator.* custom events on the channel, and the Android SDK parses them into typed events:

The type strings are also available as EventType.AI_TYPING_INDICATOR_UPDATED, EventType.AI_TYPING_INDICATOR_CLEAR and EventType.AI_TYPING_INDICATOR_STOP. aiState is a plain String holding the raw ai_state value, not an enum, so a misspelled state compiles fine and never matches. Compare it with these values:

A typical reply sends AI_STATE_THINKING right after the placeholder, AI_STATE_EXTERNAL_SOURCES while a tool call runs, AI_STATE_GENERATING with the first token, and ai_indicator.clear once the final text is stored.

The indicator events reach only the clients connected and watching the channel when they're sent, and aren't stored, so a user who opens the channel halfway through a reply never gets the earlier ones. Use them for the indicator only, and rely on the message's text and generating, which arrive with every update, for the reply itself.

If you send these events from your own backend, always include message_id in ai_indicator.update. The Android SDK can't parse an update without it: it drops the event and treats the failure as a connection error, so the client reports a disconnect and reconnects. The Stream backend SDKs always include it.

Stop the reply

When the user stops the reply, the app sends ai_indicator.stop with the message_id of the reply, as the view model above does. sendEvent sends the entries of extraData as top-level fields of the event, next to the channel's cid.

On the backend, stop the reply whose id matches message_id, or whatever reply is in flight in the channel if the event has none. Abort the model's stream, then finish the way a normal reply finishes: store the text written so far with a regular partial update and generating: false, then send ai_indicator.clear. Without that final update, the message stays as it was last stored.

Stream with ephemeral updates

A regular partial update writes the message to the database on every call. A long reply flushed every few tokens turns into hundreds of writes, and with several conversations at once, you can hit the rate limits for message updates. While the reply streams, update it with ephemeralUpdateMessage instead. It takes the same payload and sends the same message.updated event, so the reply types itself out the same way, but it stores nothing.

The last update must not be ephemeral. When the model finishes, store the complete text with a regular partial update, and generating: false. Otherwise anyone who opens the channel later loads the last stored version of the message: the empty placeholder, still generating. The same goes for whatever else ends the reply, such as an error or the user stopping it.

// While the model streams: send the latest text without storing it.
await client.ephemeralUpdateMessage(
  messageId,
  { set: { text, generating: true } },
  botId,
);

// Once the model is done: store the final text.
await client.partialUpdateMessage(
  messageId,
  { set: { text, generating: false } },
  botId,
);

Ephemeral updates are server-side only, and every server-side SDK has them: the AI message streaming guide shows each one. Even ephemeral updates reach every device watching the channel, so don't send one per token: flush every 20 or so chunks, or at most every 50 to 100 ms.

Answer on the device

When your agent can't answer, because the user is offline or the agent reached its usage limit, a model on the device still can. AIOnDeviceModel is Gemini Nano, the model in Android's AICore, run through ML Kit's GenAI Prompt API, and the conversation never leaves the device. It ships as a separate artifact, available from 7.15.0, and requires minSdk 26:

dependencies {
    implementation("io.getstream:stream-chat-android-ai-ondevice:$stream_version")
}

Gemini Nano runs only on devices whose AICore offers it, such as recent Pixel, Samsung and other flagship phones, and not on emulators. Even there, the model may need downloading first. status() says where it is (Available, Downloadable, Downloading or Unavailable), and download() downloads it while you collect the flow:

val model: AIOnDeviceModel = AIOnDeviceModel(context)

if (model.status() == AIOnDeviceModel.Status.Downloadable) {
    model.download().collect { download: AIOnDeviceModel.Download ->
        when (download) {
            is AIOnDeviceModel.Download.Started -> showProgress(download.bytesToDownload)
            is AIOnDeviceModel.Download.Progress -> updateProgress(download.bytesDownloaded)
            AIOnDeviceModel.Download.Completed -> hideProgress()
            is AIOnDeviceModel.Download.Failed -> showError(download.cause)
            else -> Unit // More states may be added.
        }
    }
}

Once it's available, the model streams the answer to a list of AIConversationTurns that ends with the user's question, and each emission is the whole answer so far:

try {
    backend.send(text)
} catch (error: IOException) {
    if (!model.isAvailable()) throw error
    val turns: List<AIConversationTurn> = history + AIConversationTurn.user(text)
    model.reply(instructions = "Answer briefly. You have no tools.", turns = turns)
        .collect { answer: String -> localAnswer = answer }
}

Cancelling the collection stops the model. reply throws AIOnDeviceModel.Unavailable when the model can't answer, and AIOnDeviceModel.Failure when it fails while answering. It's a small model, good for short answers and for working with what's in the conversation. Its context holds a few thousand tokens, so the oldest turns of a long conversation are left out, and it writes at most 256 tokens of answer (MODEL_MAXIMUM_RESPONSE_TOKENS). Pass maximumResponseTokens to keep answers shorter. Tell it in the instructions what it can't do, since it has none of your agent's tools. Its answer isn't sent to the channel: keep it on the device, or send it yourself. When you no longer need the model, such as in your ViewModel's onCleared(), call close() to release its resources. To use another model, implement the AILocalModel interface.

Actions

Under each finished reply, the conversation shows what the AIConversationReplyActions slot of ChatAiComponentFactory draws, nothing by default. Return an AIReplyActionsRow there to offer copy and share, rate, and regenerate on the latest reply, as described in Reply actions. Ratings and regenerating go through your own code, such as your backend, so keep the ratings in your view model next to the rest of the state.

Pressing and holding the user's message offers to copy its text, and to edit it when you pass an onEdit, as described in Editing a message. What sending the edited text does, such as updating the message and asking for a new reply, is up to your app.

Generative UI

To show the UI your agent writes in place of a reply's text, pass an AIUIHost as ui. The host's send receives what the user did in a reply's UI. With a backend of your own, send it to the channel as their next message, with the action in its custom data:

val ui: AIUIHost = AIUIHost(
    renderers = listOf(A2UIRenderer(), OpenUIRenderer()),
) { action: AIUIAction ->
    ChatClient.instance().channel(cid)
        .sendMessage(Message(text = action.label, extraData = mapOf(AIUIAction.CUSTOM_DATA_KEY to action.json)))
        .enqueue()
}

AIConversation(messages = messages, ui = ui)

With Stream Agents, ask the session instead, with session.responses.create(action.label, uiAction = action.json). Keep the same host for as long as the conversation shows, such as in your view model, so each reply's UI keeps its state. Everything else stays as it is: the reply actions show under a reply that answers only with UI, once its UI is written, and the footer and emptyContent work the same.