Sessions API

Sessions store a conversation with AI. Every frontier model handles the storage of messages differently. That's why we standardize message persistence and make it easy for you to change the underlying model.

Sessions have a title, description and custom fields. You can do the following with sessions:

Let's get started and create your first session:

JavaScript
import { Client } from "@stream-io/stream-agents";

const client = new Client({ apiKey: "your_api_key" });
await client.setUser("jlahey", () =>
  fetch("/api/agent-token").then((response) => response.text()),
);
const agent = client.agent("my-agent");

const session = await agent.sessions.create({ custom: { ticket: "4721" } });
Kotlin
val agents = StreamAgents(apiKey = "your_api_key")
agents.setUser(User(id = "jlahey")) { yourBackend.agentToken() }
val agent = agents.agent("my-agent")

val session = agent.chat()
Swift
let agents = StreamAgents(apiKey: "your_api_key")
agents.setUser(User(id: "jlahey")) { try await yourBackend.agentToken() }
let agent = agents.agent("my-agent")

let session = try await agent.sessions.create()
await session.start()
Dart
final agents = VisionAgents(
  apiKey: 'your_api_key',
  userId: 'jlahey',
  token: () => http.read(Uri.parse('https://your-app.example.com/api/agent-token')),
);
final agent = agents.agent('my-agent');

final session = await agent.chat();

Your server authenticates with STREAM_API_KEY and STREAM_API_SECRET; a client needs the API key, the user's id and a token for them. Auth explains how this works in detail.

Now that you have a session open, you can use responses.create to interact with the agent.

JavaScript
await session.responses.create("Where is order 1042?");

for await (const event of session.events()) {
  if (event.kind === "response_delta") console.log(event.text);
  if (event.kind === "responded") break;
}
Kotlin
val answer = launch {
    session.events()
        .takeWhile { it.type != "responded" }
        .collect { if (it.type == "response_delta") print(it.text) }
}
session.responses.create("Where is order 1042?")
answer.join()
Swift
let events = session.events()
_ = try await session.responses.create("Where is order 1042?")

for await event in events {
    if event.kind == .responseDelta { print(event.text, terminator: "") }
    if event.kind == .responded { break }
}
Dart
final events = session.events();
await session.responses.create('Where is order 1042?');

await for (final event in events) {
  if (event.kind == AgentEventKind.responseDelta) stdout.write(event.text);
  if (event.kind == AgentEventKind.responded) break;
}

A session keeps its conversation in a Stream Chat channel, which session.chat() opens. The channel is a transcript, for text and voice sessions alike.

Warning:

The agent does not answer a message sent to the channel. Write to the agent with responses.create, which also stores your message in the channel.

Not documented for Flutter yet. Showing JavaScript.

JavaScript

session.chat() returns the StreamChat client and the channel. Import @stream-io/stream-agents/chat once first, or it throws. It opens on the StreamChat you handed over with client.use, or else connects one as the user setUser named.

Kotlin

session.chat(context) returns a ChannelClient for the channel. It opens on the ChatClient you handed over with agents.use(client), or else connects one as the user setUser named.

Swift

session.chat() returns a ChatChannelController for the channel. It opens on the ChatClient you handed over with agents.use(_:), or else connects one as the user setUser named.

If you've used OpenAI before the responses.create API will seem familiar. Creating responses API explains how to interact with the agent.

A few fields on the session are updated automatically:

  • Title
  • Description
  • Usage

Creating or updating a session

sessions.create creates a conversation

JavaScript
const session = await agent.sessions.create({
  title: "Order 1042",
  custom: { ticket: "4721" },
});
Kotlin
val session = agent.chat(
    SessionOptions(
        title = "Order 1042",
        custom = buildJsonObject { put("ticket", "4721") },
    ),
)
Swift
var options = SessionOptions()
options.title = "Order 1042"
options.custom = ["ticket": "4721"]
let session = try await agent.sessions.create(options)
await session.start()
Dart
final session = await agent.chat(const SessionOptions(
  title: 'Order 1042',
  custom: {'ticket': '4721'},
));

Passing your own id gives you control over uniqueness. This is convenient when you want to create a session for an object in your database. For instance opening a session for a specific order in your shop etc.

Not documented for Flutter yet. Showing JavaScript.

JavaScript
const session = await agent.sessions.create({
  id: "order-1042",
  title: "Order 1042",
});
Kotlin
val session = agent.chat(SessionOptions(id = "order-1042", title = "Order 1042"))
Swift
var options = SessionOptions()
options.id = "order-1042"
options.title = "Order 1042"
let session = try await agent.sessions.create(options)

An id is any string of up to 64 letters, digits, - and _. A UUID works too.

Everything is optional:

Option What it does
id The id to hold the session by, such as your own record's id. Left out, the router generates a UUIDv7
title, description What a person finds the conversation by later. Both are searched
project_id Groups conversations, and is recorded as the project cost tag
custom Your own labels, handed back untouched
tags Cost tags for this conversation, merged with the agent's own tags
model_overwrites Change the models for this conversation, such as { llm: "llm-thinking", thinking: "high" }
incognito Keep nothing: no Stream Chat channel, no session row, no turns. It cannot be listed, searched or forked afterwards
start_voice Start voice as the session opens. Left out, the conversation is held in writing until voice is started

Models, cost tags and incognito

It's common to override the thinking level, cost tracking or incognito mode.

model_overwrites changes the models for one conversation, tags adds cost tags to it. incognito mode disables permanent storage.

Not documented for Flutter yet. Showing JavaScript.

JavaScript
const session = await agent.sessions.create({
  modelOverwrites: { llm: "llm-thinking", thinking: "high" },
  tags: { customer_id: "123", plan: "pro" },
  incognito: true,
});
Kotlin
val session = agent.chat(
    SessionOptions(
        modelOverwrites = ModelOverwrites(llm = "llm-thinking", thinking = ModelOverwrites.Thinking.High),
        tags = mapOf("customer_id" to "123", "plan" to "pro"),
        incognito = true,
    ),
)
Swift
var options = SessionOptions()
options.modelOverwrites = ModelOverwrites(llm: "llm-thinking", thinking: .high)
options.tags = ["customer_id": "123", "plan": "pro"]
options.incognito = true
let session = try await agent.sessions.create(options)

model_overwrites takes llm, stt, tts, sts, search, thinking, temperature, max_output_tokens and verbosity. thinking is how hard the model reasons before answering: none, minimal, low, medium or high. A model that does not reason ignores it.

tags are cost labels. This helps you easily see costs by user, use case, or any other tag you provide.

Update a running session

update changes a session's title, description, custom data, models or voice. New models take over from the next turn, and one that doesn't route is refused. A client can only change the title, description and custom data:

JavaScript
const updated = await session.update({ title: "Pricing" });

// From your backend, the models too:
// await session.update({ llm: "llm-thinking", thinking: "high" });
Kotlin
val updated = session.update(title = "Pricing")
Swift
let updated = try await session.update(title: "Pricing")
Dart
final updated = await session.update(title: 'Pricing');

Fields you leave out stay as they are. Setting sts switches the session to speech-to-speech, and an empty sts switches it back. A session you only have the id of is changed with sessions.update(id, changes), from a client too.

Title, description and review

The router writes some fields for you:

Field Written By
title, description After the first answer, then after every third answer A fast model reading the start of the conversation
review_score, review_notes Once, a few seconds after the session ends A quality model reading the whole conversation
usage When the session ends The router, from every model it called

The title is up to eight words, such as "Add push notifications to an Android app", and the description is one sentence, both in the user's language. A title passed to create is replaced after the first answer; once you set one with update, the router stops renaming the session.

An incognito session gets none of these, and a session nobody spoke in gets no review. Read them with Getting a session.

Delete or close a session

close ends the conversation and keeps it. delete removes it.

JavaScript
await session.close();
await session.delete();
// A session you only have the id of: agent.sessions.close(id)
Kotlin
session.close()
session.delete()
// A session you only have the id of: sessions.close(id)
Swift
await session.close()
try await session.delete()
// A session you only have the id of: agents.close(sessionID:)
Dart
await session.close();
await session.delete();
// A session you only have the id of: sessions.stop(id)

close ends the conversation: with voice on, the agent leaves the call. It is safe to call on a session that has already ended, and keeps what the session recorded and remembered. A session you only have the id of is closed by its id, as the JavaScript, Kotlin, Swift and Dart samples show.

Warning:

A text session also ends when the last client watching it disconnects. To continue it, resume it by its session id: watching it again (attach in Kotlin and Swift, Sessions.Resume in Go) or asking it something with agent.sessions.responses(id).create(text) reopens it with everything that was said.

delete stops the session if it is running. A session you only hold the id of is deleted with sessions.delete(id).

Warning:

delete also deletes the session's turns and what it remembered. The user's other memories are kept. To keep the session and forget only what it remembered, see Memory and persistence.

Voice

voice.start() starts the realtime communication and voice.stop() stops it. The agent talks on a call of the agent call type, which is built into every Stream app.

Not documented for Flutter yet. Showing JavaScript.

JavaScript
// npm install @stream-io/video-client (React: @stream-io/video-react-sdk, see below)
import { VoiceSession } from "@stream-io/stream-agents/video";

await session.voice.start();

// Joins the call "agent:" + session.id from this device, with the microphone on,
// as the user setUser named.
const voice = VoiceSession.of(session);
await voice.join();

// Without a UI framework, play the agent with one <audio> element per remote participant.
// In React, render <ParticipantsAudio /> instead: see Voice UI below.
const call = voice.call;
const playing = new Set<string>();
call?.state.remoteParticipants$.subscribe((participants) => {
  for (const { sessionId } of participants) {
    if (playing.has(sessionId)) continue;
    playing.add(sessionId);
    const audio = document.body.appendChild(document.createElement("audio"));
    call.bindAudioElement(audio, sessionId);
  }
});

// When you are done talking: this device leaves the call, and the conversation
// carries on in writing.
await voice.leave();
await session.voice.stop();
Kotlin
session.startVoice()

// Joins the call "agent:" + session.id with Stream Video, as the user setUser named.
val voice = VoiceSession.attach(context, agents, session.id)
voice.join()

// When you are done talking. Leaving a call you attached to keeps the session.
voice.leave()
session.stopVoice()
Swift
try await session.startVoice()

// Joins the call "agent:" + session.id with Stream Video, as the user setUser named.
let voice = try await VoiceSession.attach(agents: agents, sessionID: session.id)
await voice.join()

// When you are done talking. Leaving a call you attached to keeps the session.
await voice.leave()
try await session.stopVoice()

VoiceSession is this device on the session's call. A session created with start_voice: true has voice on from the start.

Not documented for Flutter yet. Showing JavaScript.

JavaScript

Import VoiceSession from @stream-io/stream-agents/video.

  • Joining:
    • VoiceSession.start(agent) opens a session with voice on.
    • VoiceSession.of(session) joins one whose voice is already on.
    • VoiceSession.attach(client, sessionId) does the same from the session's id alone, with a watcher of its own that runs no tools.
    • join() turns the microphone on and joins, and rejects if it can't, for example when the user denies the microphone.
    • join({ camera: true }) also turns the camera on, on the back lens where there is one.
  • During the call: setMuted and setCameraEnabled.
  • Leaving: leave() leaves the call, and closes the session only if start opened it; an attached one keeps running. end() always closes it.
  • State: voice.get() returns the call, the client, muted, cameraEnabled and a status, and voice.subscribe reports changes.

A session whose voice is on can be joined from its id alone, such as one your server started for this user. Turn the camera on to let the agent look through it:

import { VoiceSession } from "@stream-io/stream-agents/video";

// `client` holds your apiKey and a user from setUser.
const voice = await VoiceSession.attach(client, sessionId);
await voice.join({ camera: true });

// Later: stop sharing the camera, and keep talking.
await voice.setCameraEnabled(false);

// Leaving an attached session keeps it running.
await voice.leave();

The attached watcher runs no tools. Attach to a session whose tools run on your server, as tools a server declared always do. A session another page opened with browser tools keeps them in that page: join its call from that page with VoiceSession.of(session) instead, so the tool calls reach the page that runs them.

The agent looks through the camera when it's asked about what it sees, if its config has the vision skill and a vision subagent. The SDK answers its requests for frames from the browser, with nothing else to wire up.

Joining needs a client made with your apiKey and a user from setUser, as in the quickstart. To join the call yourself instead, session.video() returns { client, call }, typed loosely because @stream-io/video-client is an optional peer.

To use the StreamChat and StreamVideoClient your app already holds, hand them over first with client.use({ chat: chatClient, video: videoClient }), so the session does not open second connections. In a React app, install @stream-io/video-react-sdk instead of @stream-io/video-client: it brings its own video client, and installing both risks two copies of it.

Kotlin

VoiceSession joins the call with Stream Video, signed in as the user you passed to setUser.

  • Joining:
    • VoiceSession.start(context, agents, options) opens a session with voice on.
    • VoiceSession.attach(context, agents, sessionId) joins one whose voice is already on.
    • join() turns the microphone on, joins, and plays the agent on the speaker. It doesn't throw: if joining fails, the error is in voice.failure. Ask for RECORD_AUDIO before you call it.
    • join(camera = true) also turns the camera on, on the back lens. It needs CAMERA too.
  • During the call: setMuted(muted) and setCameraEnabled(enabled).
  • Leaving: leave() leaves the call, and closes the session only if start opened it. end() always closes it.
  • State: call, isMuted, isCameraEnabled and failure are StateFlows. session is the conversation.

To use the ChatClient and StreamVideo your app already holds, hand them over first with agents.use(client) and agents.use(video), so the session does not open second connections. Stream Video keeps one client per process: one of your own that is signed in as another user makes joining fail.

Swift

VoiceSession joins the call with Stream Video, signed in as the user you passed to setUser.

  • Joining:
    • VoiceSession.start(agents:agent:) opens a session with voice on.
    • VoiceSession.attach(agents:sessionID:) joins one whose voice is already on.
    • join() turns the microphone on, joins, and plays the agent on the speaker. It doesn't throw: if joining fails, for example when the user denies the microphone, the error is in voice.failure.
    • join(camera: true) also turns the camera on, on the back lens where there is one.
  • During the call: setMuted(_:) and setCameraEnabled(_:).
  • Leaving: leave() leaves the call, and closes the session only if start opened it. end() always closes it.
  • State: VoiceSession is an ObservableObject that publishes call, isMuted, isCameraEnabled and failure. session is the conversation.

To use the ChatClient and StreamVideo your app already holds, hand them over first with agents.use(_:), so the session does not open second connections. An app with a StreamVideo of its own must hand it over: another one would become the instance Stream's call views and CallKit read. The quickstart builds a voice UI on VoiceSession.

A phone call is a session too. Each person who rings one of your numbers gets a new session. See accepting calls and making calls.

Voice UI: transcript, captions and status

A voice UI is three pieces: the agent's audio, the transcript with live captions, and a status that says whether the agent is listening, thinking or speaking.

The agent's status is one of these, the states the glow and wave views take:

State When
connecting Until this device has joined.
speaking While the call hears the agent.
listening While the caller is heard or transcribed.
thinking While the agent answers or its skills work, and until its reply is heard.
idle Otherwise, and after leave().

Speaking comes from the call and not from the events. spoke fires when a piece of the reply has been synthesized and sent to the call, not when its audio starts, and responded means the model finished, not that the reply was spoken.

Without the call, session.state says what the agent is doing on any session, typed or spoken: idle, listening, responding, working (with the skills at work) or ended. The state follows the session's socket beside session.events(), so a loop reading events elsewhere is unaffected.

For latency timings, read session.events(): turn carries the turn's timings and spoke its time_to_first_byte_ms.

Not documented for Flutter yet. Showing React.

React
  • Audio: VoiceSession joins the call. Play the agent with <ParticipantsAudio /> from the Video React SDK, inside StreamVideo and StreamCall with the client and call from voice.get().
  • Transcript and captions: render session.chat()'s channel with MessageList from stream-chat-react. isMessageAIGenerated streams the agent's replies in with markdown, and AIStateIndicator shows while the agent thinks or generates (useAIState(channel) gives the same state to your own indicator).
  • Status: voice.get().status is what the agent is doing, in the states VoiceGlow and VoiceWave from stream-chat-react/ai-components take. Read session.events() for latency timings, not for the status or the transcript.

Create the session from a user action such as a Start button, not at module scope: module-scope setup runs during server rendering and again on every hot reload, and asks for the microphone as soon as the page loads.

AgentVoice.tsx
import "@stream-io/stream-agents/chat";
import type { AgentHandle } from "@stream-io/stream-agents";
import { VoiceSession } from "@stream-io/stream-agents/video";
import {
  ParticipantsAudio,
  StreamCall,
  StreamVideo,
  useCallStateHooks,
} from "@stream-io/video-react-sdk";
import { useEffect, useState, useSyncExternalStore } from "react";
import type { Channel as ChatChannel, StreamChat } from "stream-chat";
import {
  AIStateIndicator,
  Channel,
  Chat,
  MessageList,
} from "stream-chat-react";
import { VoiceGlow } from "stream-chat-react/ai-components";
import "stream-chat-react/css/index.css";

declare module "stream-chat" {
  interface CustomMessageData {
    ai_generated?: boolean;
  }
}

// `agent` comes from a Client with your apiKey and a user from setUser, as in the quickstart.
const start = async (agent: AgentHandle) => {
  // The agent joins the session's call right away.
  const voice = await VoiceSession.start(agent);
  const { client, channel } = await voice.session.chat();
  return {
    voice,
    chatClient: client as StreamChat,
    channel: channel as ChatChannel,
  };
};

// Plays the agent.
const AgentAudio = () => {
  const { useRemoteParticipants } = useCallStateHooks();
  return <ParticipantsAudio participants={useRemoteParticipants()} />;
};

const Conversation = ({
  voice,
  chatClient,
  channel,
}: Awaited<ReturnType<typeof start>>) => {
  const { call, client, status } = useSyncExternalStore(
    voice.subscribe,
    voice.get,
  );

  // Joins from this device, with the microphone on. Run twice, as in Strict Mode, it joins once.
  useEffect(() => {
    voice
      .join()
      .catch((error: unknown) =>
        console.error("Could not join the call", error),
      );
  }, [voice]);

  return (
    <>
      <VoiceGlow state={status} />
      {call && client && (
        <StreamVideo client={client}>
          <StreamCall call={call}>
            <AgentAudio />
          </StreamCall>
        </StreamVideo>
      )}
      <Chat
        client={chatClient}
        isMessageAIGenerated={(message) => Boolean(message.ai_generated)}
      >
        <Channel channel={channel}>
          <MessageList />
          <AIStateIndicator />
          {/* No composer while voice is on: a typed turn on a call is not marked
              finished, and the next responses.create is refused. */}
        </Channel>
      </Chat>
      {/* Leaves the call and ends the session start() opened. */}
      <button onClick={() => void voice.leave()}>Hang up</button>
    </>
  );
};

export const AgentVoice = ({ agent }: { agent: AgentHandle }) => {
  const [started, setStarted] = useState<Awaited<ReturnType<typeof start>>>();

  if (!started) {
    return (
      <button onClick={() => void start(agent).then(setStarted)}>Start</button>
    );
  }
  return <Conversation {...started} />;
};

To style spoken turns, interrupted replies or a reply still being generated, check those custom fields in a custom Message component. The default one draws a cut-off reply whose text was cleared as an empty bubble, so hide messages with no text there; the React AI integration guide shows how to register one. To let the caller cut the agent off, call session.interrupt(). When you leave, call voice.leave(), which ends the session start() opened. To keep the conversation going in writing instead, join with VoiceSession.of(session) and call session.voice.stop() after leave().

session.state has the same get and subscribe as voice. Read latency timings from event.frame, and each loop over session.events() is told everything from when it started. Start one in an effect and stop it in the effect's cleanup, so an effect that runs twice, as in Strict Mode, leaves one loop running.

The call stays up when the agent fails, so say so under the glow. session.conversation has the same get and subscribe, and its failure comes with failureContext, the part that failed (stt for speech recognition, llm, tts), until the agent next hears, answers or speaks. A tool that asks first waits in its approvals: show it over the call with AIToolApprovalCard and answer with session.decide(id, { allowed }). The voice mode of the React assistant sample does both.

Both work on any session, with or without a call:

AgentActivity.tsx
import type { Session, ToolApprovalRequest } from "@stream-io/stream-agents";
import { useSyncExternalStore } from "react";

// The approval's title with each {argument} filled in from the call.
const question = ({ approval, arguments: args }: ToolApprovalRequest) => {
  const values = JSON.parse(args || "{}") as Record<string, unknown>;
  return approval.title.replace(/\{(\w+)\}/g, (_, name: string) =>
    String(values[name] ?? ""),
  );
};

export const AgentActivity = ({ session }: { session: Session }) => {
  // idle, listening, responding, working (with the skills at work) or ended.
  const state = useSyncExternalStore(
    session.state.subscribe,
    session.state.get,
  );
  const { turns, failure, failureContext, approvals } = useSyncExternalStore(
    session.conversation.subscribe,
    session.conversation.get,
  );

  return (
    <>
      <p>
        {state.kind === "working"
          ? `Working: ${state.skills.join(", ")}`
          : state.kind}
      </p>
      {failure && (
        <p role="alert">
          The agent failed ({failureContext}): {failure}
        </p>
      )}
      {approvals.map((request) => (
        <p key={request.id}>
          {question(request)}
          <button onClick={() => session.decide(request.id, { allowed: true })}>
            Allow
          </button>
          <button
            onClick={() => session.decide(request.id, { allowed: false })}
          >
            Not now
          </button>
        </p>
      ))}
      <ol>
        {turns.map((turn) => (
          <li key={turn.id}>
            {turn.speaker === "agent" ? "Agent" : "You"}: {turn.text}
          </li>
        ))}
      </ol>
    </>
  );
};

Each snapshot keeps its identity until something changes, so a component re-renders only when the state does.

Compose
  • Audio: VoiceSession joins the call and plays the agent on the speaker, so it needs no view of yours.
  • Transcript and captions: session.conversation holds the turns, each reply growing as it is generated. To show the channel instead, open it with session.chat(context) and render it with MessageList from the Compose UI components, as in the quickstart's chat UI.
  • Status: AIVoiceGlow and AIVoiceWave from stream-chat-android-ai-compose take an AIVoiceAgentState. Work it out from the call, which says who is speaking, and from the conversation's state. Read session.events() for latency timings, not for the status or the transcript.
VoiceScreen.kt
import io.getstream.chat.android.ai.compose.ui.component.AIVoiceAgentState
import io.getstream.chat.android.ai.compose.ui.component.AIVoiceGlow
import io.getstream.agents.android.core.Conversation
import io.getstream.agents.android.core.SessionOptions
import io.getstream.agents.android.core.StreamAgents
import io.getstream.agents.android.core.VoiceSession

// `agents` has your apiKey and a user from setUser, as in the quickstart.
// Ask for RECORD_AUDIO before the user taps Start.
@Composable
fun VoiceScreen(agents: StreamAgents) {
    val context = LocalContext.current
    val scope = rememberCoroutineScope()
    var voice by remember { mutableStateOf<VoiceSession?>(null) }

    val started = voice
    if (started != null) {
        VoiceConversation(started)
    } else {
        Button(onClick = {
            scope.launch {
                // The agent joins the session's call right away.
                voice = VoiceSession.start(context, agents, SessionOptions(agent = "my-agent"))
            }
        }) { Text("Start") }
    }
}

@Composable
fun VoiceConversation(voice: VoiceSession) {
    val scope = rememberCoroutineScope()
    val conversation by voice.session.conversation.collectAsStateWithLifecycle()

    // Joins from this device, with the microphone on and the agent on the speaker.
    LaunchedEffect(voice) { voice.join() }

    Column(horizontalAlignment = Alignment.CenterHorizontally, verticalArrangement = Arrangement.spacedBy(16.dp)) {
        VoiceStatus(voice, conversation.state)
        // The transcript, with each reply growing as it is generated.
        LazyColumn(Modifier.weight(1f)) {
            items(conversation.turns) { turn -> Text(turn.text) }
        }
        // Leaves the call and ends the session start() opened.
        Button(onClick = { scope.launch { voice.leave() } }) { Text("Hang up") }
    }
}

/** What the agent is doing, in the states the glow takes. */
@Composable
fun VoiceStatus(voice: VoiceSession, session: Conversation.State) {
    val call by voice.call.collectAsStateWithLifecycle()
    // The call's participants that are speaking, this device's user included.
    val speakers by remember(call) { call?.state?.activeSpeakers ?: MutableStateFlow(emptyList()) }
        .collectAsStateWithLifecycle()

    val state = when {
        call == null -> AIVoiceAgentState.Connecting
        // Speaking comes from the call: a reply is written before it is spoken.
        speakers.any { !it.isLocal } -> AIVoiceAgentState.Speaking
        speakers.any { it.isLocal } -> AIVoiceAgentState.Listening
        else -> when (session) {
            Conversation.State.Listening -> AIVoiceAgentState.Listening
            Conversation.State.Responding, is Conversation.State.Working -> AIVoiceAgentState.Thinking
            Conversation.State.Idle, Conversation.State.Ended -> AIVoiceAgentState.Idle
        }
    }
    AIVoiceGlow(state = state, modifier = Modifier.size(240.dp))
}

To let the caller cut the agent off, call session.interrupt(). voice.leave() ends the session start opened. To keep the conversation going in writing instead, join with VoiceSession.attach(context, agents, sessionId) and call session.stopVoice() after leave().

AgentSession exposes conversation as a StateFlow, so a composable collecting it redraws as its state changes. Read latency timings from event.fields.

SwiftUI
  • Audio: VoiceSession joins the call and plays the agent on the speaker, so it needs no view of yours.
  • Transcript and captions: show session.chat()'s channel with AIConversationView from StreamChatAI, through the AgentTransport of the quickstart's chat UI. It streams the agent's replies in with markdown and shows the agent thinking.
  • Status: AIVoiceGlowView and AIVoiceWaveView from StreamChatAI take an AIVoiceAgentState. Work it out from the call, which says who is speaking, and from session.state. Read session.events() for latency timings, not for the status or the transcript.
VoiceScreen.swift
import StreamChatAI
import StreamVideo
import SwiftUI
import StreamAgents

// `agents` has your apiKey and a user from setUser, and `AgentTransport` and `AgentMessage`
// are the chat UI's, as in the quickstart.
struct VoiceScreen: View {
    let agents: StreamAgents
    @State private var started: (voice: VoiceSession, transcript: AgentTransport)?

    var body: some View {
        if let started {
            VoiceConversation(voice: started.voice, transcript: started.transcript)
        } else {
            Button("Start") {
                Task { started = try? await start() }
            }
        }
    }

    private func start() async throws -> (voice: VoiceSession, transcript: AgentTransport) {
        // The agent joins the session's call right away.
        let voice = try await VoiceSession.start(agents: agents, agent: "my-agent")
        // The session's socket, which follows what the agent is doing.
        await voice.session.start()
        return (voice, try await AgentTransport(voice.session))
    }
}

struct VoiceConversation: View {
    @ObservedObject var voice: VoiceSession
    @StateObject private var transcript: AIConversation<AgentMessage>

    init(voice: VoiceSession, transcript: AgentTransport) {
        self.voice = voice
        _transcript = StateObject(
            wrappedValue: AIConversation(transport: transcript, approver: transcript.approver)
        )
    }

    var body: some View {
        VStack(spacing: 16) {
            if let call = voice.call {
                VoiceStatus(voice: voice, call: call.state)
            } else {
                AIVoiceGlowView(state: .connecting)
                    .frame(width: 240, height: 240)
            }
            // The transcript, with each reply streaming in as it is generated.
            // No composer while voice is on: a typed turn on a call is not marked
            // finished, and the next responses.create is refused.
            AIConversationView(conversation: transcript)
            // Leaves the call and ends the session start() opened.
            Button("Hang up", role: .destructive) {
                Task { await voice.leave() }
            }
        }
        // Joins from this device, with the microphone on and the agent on the speaker.
        .task { await voice.join() }
    }
}

/// What the agent is doing, in the states the glow takes.
struct VoiceStatus: View {
    @ObservedObject var voice: VoiceSession
    // The call's participants, and which of them is speaking.
    @ObservedObject var call: CallState

    var body: some View {
        AIVoiceGlowView(state: state)
            .frame(width: 240, height: 240)
    }

    private var state: AIVoiceAgentState {
        // Speaking comes from the call: a reply is written before it is spoken.
        if call.remoteParticipants.contains(where: \.isSpeaking) { return .speaking }
        if call.localParticipant?.isSpeaking == true { return .listening }
        switch voice.session.state {
        case .listening: return .listening
        case .responding, .working: return .thinking
        case .idle, .ended: return .idle
        }
    }
}

AIConversationView leaves out a reply with nothing to show, such as one the caller cut off whose text was cleared. To style spoken turns, interrupted replies or a reply still being generated, read those custom fields from the message's extraData in your AIMessage, as AgentMessage reads generating, and pass AIConversationView a view factory. To let the caller cut the agent off, call session.interrupt(). voice.leave() ends the session start opened. To keep the conversation going in writing instead, join with VoiceSession.attach(agents:sessionID:) and call session.stopVoice() after leave(). Add a microphone usage description to your Info.plist.

AgentSession is an ObservableObject, so a view observing the session redraws as its state changes. Read latency timings from event.fields.

Listing and searching sessions

sessions.query lists an agent's sessions, including ended ones. It works like other Stream queries, with a filter, sort, limit and cursor. sessions.get reads one session by id.

Getting a session

sessions.get reads one session, running or ended. Once it has ended, your server is also told how it went: a review and what it cost.

Not documented for Flutter yet. Showing JavaScript.

JavaScript
const session = await agent.sessions.get(id);
console.log(session.title, session.state, session.closed_at);
console.log(session.review_score, session.usage?.cost_micros);
Kotlin
val session = agent.sessions.get(id)
println("${session.title} ${session.state} ${session.modality}")
// The review and usage are only given to your backend.
Swift
let session = try await agent.sessions.get(id)
print(session.title, session.state, session.modality)
// The review and usage are only given to your backend.
Field What it says
title, description, custom, project_id Your labels, or the ones the router wrote for you
state live while the agent holds it, ended after
modality text, voice, or video once the agent has seen the user's video
llm, stt, tts, voice The models and voice routing picked, while it is running
created_at, last_response_at, closed_at When it began, last answered and ended
review_score, review_notes How well the agent handled it, from 1 to 5, and why
usage input_tokens (with cached_input_tokens), output_tokens, requests and cost_micros, in millionths of a dollar, over every model it called, typed and spoken

usage is filled in once the session has ended, and the review a few seconds later. A session nobody spoke in has no review. These three fields are only given to your server, so a client asks your server for them.

Querying

For example, the ended sessions of the support project in October:

JavaScript
const page = await agent.sessions.query({
  projectId: "support",
  state: "ended",
  createdSince: new Date("2026-10-01T00:00:00Z"),
  createdBefore: new Date("2026-11-01T00:00:00Z"),
  sort: "updated_at",
  limit: 20,
});
const sessions = page.items;
Kotlin
import kotlin.time.Instant

val page = agent.sessions.query(
    SessionQuery(
        projectId = "support",
        state = Session.State.Ended,
        createdSince = Instant.parse("2026-10-01T00:00:00Z"),
        createdBefore = Instant.parse("2026-11-01T00:00:00Z"),
        sort = SessionQuery.Sort.UpdatedAt,
        limit = 20,
    ),
)
val sessions = page.items
Swift
var query = SessionQuery(limit: 20)
query.projectID = "support"
query.state = .ended
query.createdSince = try Date("2026-10-01T00:00:00Z", strategy: .iso8601)
query.createdBefore = try Date("2026-11-01T00:00:00Z", strategy: .iso8601)
let page = try await agent.sessions.query(query)
let sessions = page.items
Dart
final page = await agent.sessions.query(
  filter: {
    'project_id': 'support',
    'state': 'ended',
    'created_at': {r'$gte': '2026-10-01T00:00:00Z', r'$lt': '2026-11-01T00:00:00Z'},
  },
  sort: const [SessionSort(field: 'updated_at')],
  limit: 20,
);
final sessions = page.items;

Every condition in the filter has to match:

Field Matches
project_id The project the session belongs to
agent_id, config_id The agent id or config the session ran under
user_id Whose session it is (server-side only)
modality text, voice or video
state live or ended
custom Sessions with all of these labels, such as { ticket: "4721" }
created_at A time range, with $gte and $lt
text A search, with $q

Sessions come back most recently active first (updated_at). A text search comes back best match first (relevance).

Searching

Search with a text condition. It matches the title, description, project and agent name, not what was said, and can't be combined with project_id:

JavaScript
const found = await agent.sessions.search("order 1042", {
  state: "ended",
  limit: 5,
});
const one = await agent.sessions.get(found.items[0]!.id);
Kotlin
val found = agent.sessions.search(
    "order 1042",
    SessionQuery(state = Session.State.Ended, limit = 5),
)
val one = agent.sessions.get(found.items.first().id)
Swift
var query = SessionQuery(limit: 5)
query.state = .ended
let found = try await agent.sessions.search("order 1042", query)
let one = try await agent.sessions.get(found.items[0].id)
Dart
final found = await agent.sessions.query(
  filter: {'text': {r'$q': 'order 1042'}, 'state': 'ended'},
  limit: 5,
);
final one = await agent.sessions.get(found.items.first.id);

sessions.search("order 1042") is a shortcut for the same query. Incognito sessions are never searchable.

What comes back are records, not live sessions: most of them are over. Read their turns with sessions.responses(id) (see Reading Responses and History), or continue one with a fork or by resuming it by its id.

Pagination

Results come back a page at a time as { items, has_more, next_cursor }: 25 by default, up to 200 with limit. When has_more is true, pass next_cursor back as cursor with the same filter and sort:

Not documented for Flutter yet. Showing JavaScript.

JavaScript
const filter = { projectId: "support" };

const page1 = await agent.sessions.query({ ...filter, limit: 50 });

if (page1.has_more && page1.next_cursor) {
  const page2 = await agent.sessions.query({
    ...filter,
    limit: 50,
    cursor: page1.next_cursor,
  });
}
Kotlin
val query = SessionQuery(projectId = "support", limit = 50)

val page1 = agent.sessions.query(query)

val cursor = page1.nextCursor
if (page1.hasMore && cursor != null) {
    val page2 = agent.sessions.query(query.copy(cursor = cursor))
}
Swift
var query = SessionQuery(limit: 50)
query.projectID = "support"

let page1 = try await agent.sessions.query(query)

if page1.hasMore, let cursor = page1.nextCursor {
    query.cursor = cursor
    let page2 = try await agent.sessions.query(query)
}

A session's turns page the same way, with limit and cursor on responses.list and on the items; items.all() and items.unwind() follow the cursor for you.

Info:

Filter field names are the same in every SDK. The page fields follow each SDK's casing, such as nextCursor in Kotlin and Swift and NextCursor in Go.

Fork, rewind and interrupt

Fork

fork continues a conversation as a new session, with the parent's history carried across. The usual reason is to ask the same question of a different model or a different agent without losing the original answer. The parent is untouched and keeps running if it was.

JavaScript
// Fork options take the API's own snake_case names, except modelOverwrites, as on create.
const harder = await session.fork({
  modelOverwrites: { llm: "llm-thinking" },
  title: "Order 1042, thinking harder",
});

const fromThere = await session.fork({ response_id: turns[0]!.id });
const fresh = await session.fork({ messages: false });
Kotlin
val forked = session.fork(
    ForkOptions(
        modelOverwrites = ModelOverwrites(llm = "llm-thinking"),
        title = "Order 1042, thinking harder",
    ),
)
// A fork comes back as a record: attach opens it as a live session, as it does any session by id.
val harder = agents.attach(forked.id)

val fromThere = session.fork(ForkOptions(responseId = turns.first().id))
val fresh = session.fork(ForkOptions(withoutHistory = true))
Swift
var options = ForkOptions()
options.modelOverwrites = ModelOverwrites(llm: "llm-thinking")
options.title = "Order 1042, thinking harder"
let forked = try await session.fork(options)
// A fork comes back as a record: attach opens it as a live session, as it does any session by id.
let harder = try await agents.attach(sessionID: forked.id)

let fromThere = try await session.fork(ForkOptions(responseID: turns[0].id))

var fromNothing = ForkOptions()
fromNothing.withoutHistory = true
let fresh = try await session.fork(fromNothing)
Dart
final harder = await session.fork(const ForkOptions(
  modelOverwrites: ModelOverwrites(llm: 'llm-thinking'),
  title: 'Order 1042, thinking harder',
));
// A fork is live, but not watched until start().
await harder.start();

final fromThere = await session.fork(ForkOptions(responseId: turns.first.id));
final fresh = await session.fork(const ForkOptions(withoutHistory: true));

Anything you name overrides what the parent was opened with: agent, title, description, project_id, custom, model_overwrites and incognito. response_id carries the history only up to the end of that response, and messages: false starts the same configuration over with no history. A fork writes its own transcript, so the two conversations never interleave.

A fork is always held in writing, even of a session with voice on: talking to two agents at once is not a conversation. Start voice on the fork to talk to it. An incognito session cannot be forked, because nothing was recorded to fork from.

Rewind

responses.rewind goes back to a response and carries on from there, as though nothing after it was said. It takes the response, any item of it, or its id.

JavaScript
await session.responses.rewind(turns[0]!);
Kotlin
session.responses.rewind(turns.first())
Swift
try await session.responses.rewind(to: turns[0].id)
Dart
await session.responses.rewind(turns.first.id);

Rewinding abandons the reply being spoken and marks every later response rewound, so they drop out of the list and the items. It needs a recorded history it can cut, so it is refused for an incognito session, a native speech-to-speech one, and a session that keeps its conversation in Stream Chat, whose channel would bring the later turns back. Fork that session at the response instead.

Interrupt

interrupt() abandons the reply being spoken. It goes over the socket a watched session holds open to the backend, alongside the other live calls: what the conversation does arrives on it as events, and you can send it text.

Method What it does
responses.create(text) Answer the text through the model, as though it had been said
say(text) Speak or write the text as it is, without the model. For a greeting you already know
interrupt() Abandon the reply being spoken
events() Everything the conversation does, until it ends
JavaScript
// say(text, { interrupt: true }) stops the reply in progress first.
session.say("Hi, I can help with your order.");
await session.responses.create("Where is order 1042?");

for await (const event of session.events()) {
  if (event.kind === "response_delta") console.log(event.text);
  if (event.kind === "responded") break;
}

// interrupt({ requestId }) stops only the reply answering that request, the request_id on its events.
session.interrupt();
Kotlin
val answer = launch {
    session.events()
        .dropWhile { it.type != "responding" }
        .takeWhile { it.type != "responded" }
        .collect { if (it.type == "response_delta") print(it.text) }
}
session.say("Hi, I can help with your order.")
session.responses.create("Where is order 1042?")
answer.join()

session.interrupt()
Swift
let events = session.events()
// say(_:interrupt: true) stops the reply in progress first.
try await session.say("Hi, I can help with your order.")
_ = try await session.responses.create("Where is order 1042?")

for await event in events {
    if event.kind == .responseDelta { print(event.text, terminator: "") }
    if event.kind == .responded { break }
}

// interrupt(requestID:) stops only the reply answering that request, the request_id on its events.
try await session.interrupt()
Dart
session.say('Hi, I can help with your order.');
final events = session.events();
await session.responses.create('Where is order 1042?');

await for (final event in events) {
  if (event.kind == AgentEventKind.responseDelta) stdout.write(event.text);
  if (event.kind == AgentEventKind.responded) break;
}

session.interrupt();

By default, what say sends waits for the reply in progress to finish.

The events you will use most are response_delta (a fragment of the answer), responded (the answer is complete; pending_work stays true while tools or delegated work are outstanding), heard (what the caller said, on a call), tool_call and tool_ran, error and left (the session ended). Your SDK answers tool_call for you by running the tool you registered.

responses.create returns the turn's id, so you can follow one particular question.

Client vs server

Each call on this page is made in one place, not on both sides: in the client for its signed-in user's own conversations, or on your server for anyone's. The client reaches only that user's sessions: it can open, watch, ask, search, fork, rewind, rename, close and delete them.

Your server reaches every session in your Stream app, so it can search and query them all, for admin screens for example. To narrow a query to one user, filter on user_id. For a phone line or another channel with no client, your server opens the sessions. See Auth.