Quickstart

The AI components give your Flutter app the screens of an AI assistant: a conversation that streams the agent's replies, a prompt composer with dictation and photos, the agent's reasoning and tool calls, and a face for a voice agent. The agent behind them runs on Stream Agents: you describe it once, with its instructions, knowledge, tools and voice, and your app talks to it with the agents SDK.

This page shows what you can build, how the pieces fit together, and takes you from an empty project to a working assistant.

What you can build

Voice runs through all of them. Every composer can have a microphone button that dictates into it, and the same agent can hold the conversation out loud: VoiceSession puts it on a call with the user, AIVoiceGlowView or AIVoiceWaveView shows what it's doing, and AIConversationView writes out what was said.

How it fits together

Your app never talks to the model, and it doesn't run the agent. It opens a session with your agent and asks it things. The agent runs on Stream's router, which calls the model, runs the agent's tools and writes the conversation into the session's Stream Chat channel as it goes, and the components show that channel:

sequenceDiagram
    participant App as Flutter app
    participant Router as Stream Agents
    participant LLM as Model
    App->>Router: agent('my-agent').chat() with the agents SDK
    App->>Router: responses.create('How do I...?')
    Router->>LLM: The conversation so far
    LLM-->>Router: Reasoning, tool calls, tokens
    Router-->>App: The reply in the session's channel, with its steps
    Note over App: AIConversationView shows it

• The agents SDK (stream_agents_core, with stream_agents_chat and stream_agents_rtc) signs the user in, opens and lists sessions, asks, stops a reply, answers the questions of tools that ask first, and runs the tools your app gives the agent on the phone.
• The AI components (stream_chat_flutter_ai) show it all: AIConversationView and ChatComposer, the reasoning and tool calls of each reply, and the voice visualizations.
• Your agent lives in a folder of plain files that you sync with the Stream CLI: its instructions, knowledge, skills, tools, connectors and models.

For voice, VoiceSession starts the agent on a call and joins it from the phone. The agent hears the user and answers out loud, and what is said is written into the session's channel.

Before you start

• A Stream app, from the Stream Dashboard.
• The beta Stream CLI, which has the getstream agents commands.
• A Flutter app on Dart 3.12 or later and Flutter 3.44 or later, targeting iOS 15 or later and Android minSdk 24 or later.

Step 1: Create the agent

Install the beta CLI, pick your Stream app, and make an agent from the text template:

curl -fsSL https://getstream.io/cli/beta/install.sh | bash
getstream init                                     # pick or create a Stream app
getstream agents init my-agent --template text     # or --template voice

That creates agents/my-agent/, with the agent's settings in agent.yaml and its system prompt in instructions.md. Write what the agent is for in instructions.md, then sync it to Stream and try it in the terminal:

getstream agents sync my-agent
getstream agents test my-agent

The Stream Agents quickstart covers the agent's folder: knowledge, skills, tools, connectors and guardrails.

Step 2: Install the SDKs

Add the agents SDK, the client for your agent, and stream_chat_flutter_ai, the views, to your pubspec.yaml:

# pubspec.yaml: needs Dart 3.12 and Flutter 3.44 or newer
dependencies:
  stream_agents_core: ^0.1.0
  stream_agents_chat: ^0.1.0
  stream_agents_rtc: ^0.1.0
  stream_chat_flutter_ai: ^0.2.0
  # The channel the chat UI reads.
  stream_chat: ^10.6.0

stream_agents_chat brings the Stream Chat client that keeps the conversation, and stream_agents_rtc brings Stream Video, which carries the voice.

Is your agent on a backend of your own? Add only stream_chat_flutter_ai, and connect the views to your agent with a transport, as described in Conversation State. stream_chat_flutter_ai doesn't depend on the other Stream Chat packages, so it works next to stream_chat_flutter or a UI of your own. If your agent answers with generative UI, also add stream_chat_flutter_ai_a2ui.

Then import the views:

import 'package:stream_chat_flutter_ai/stream_chat_flutter_ai.dart';

On its own, stream_chat_flutter_ai needs Dart 3.12 and Flutter 3.44 or later. Syntax highlighting and math formulas need a library you add yourself, as described in Code and charts. The package's showcase app demonstrates the main components and needs no API key to run.

The composer can attach photos and dictate, so add these permissions, each iOS key with a description of why your app needs it:

• NSCameraUsageDescription and NSPhotoLibraryUsageDescription in ios/Runner/Info.plist, and READ_MEDIA_IMAGES and READ_MEDIA_VISUAL_USER_SELECTED in AndroidManifest.xml, for photos.
• NSMicrophoneUsageDescription and NSSpeechRecognitionUsageDescription in Info.plist, and RECORD_AUDIO and a <queries> entry for android.speech.RecognitionService in AndroidManifest.xml, for dictation and voice.

Dictation asks for the microphone the first time the user taps it, and the attachment sheet asks for the photo library when it first opens. Permissions has the exact entries, and the ones macOS needs.

Step 3: Sign the user in

Create one StreamAgents client once the user signs in, with your API key, the user's id and a function that returns their Stream token. The user's Stream token comes from your backend, which mints it with your API secret, as described in authentication. The client asks for it again when it expires:

import 'package:stream_agents_core/stream_agents_core.dart';

/// Call once, when the user signs in.
StreamAgents signIn(String userId) => StreamAgents(
  apiKey: 'your_api_key',
  userId: userId,
  token: () => yourBackend.streamToken(),
);

To try things out before you have a backend, mint a token for a test user with the CLI, getstream token jane --ttl 2h, and return it from token: StreamAgents(apiKey: 'your_api_key', userId: 'jane', token: () async => 'paste_the_token_here').

Step 4: Connect a conversation to the agent

AIConversation holds a conversation's state and drives the views, and a transport connects it to your agent. Connecting them takes two types of your own, AgentMessage and AgentTransport, in lib/agent_transport.dart. AgentMessage is one message, as the views show it:

import 'dart:async';

import 'package:stream_chat/stream_chat.dart' as stream;
import 'package:stream_chat_flutter_ai/stream_chat_flutter_ai.dart';
import 'package:stream_agents_chat/stream_agents_chat.dart';
import 'package:stream_agents_core/stream_agents_core.dart';

/// A message of the channel, as the AI components show it.
class AgentMessage with AIMessage {
  // The reply's reasoning and tool calls, which the router writes as its attachments,
  // decoded once here rather than on every rebuild.
  AgentMessage(this.message)
    : parts = AIMessagePart.parseAll([
        for (final attachment in message.attachments)
          if (attachment.rawType case final type?)
            (type: type, payload: {'id': attachment.id, ...attachment.extraData}),
      ]);

  final stream.Message message;

  @override
  String get id => message.id;

  // The agent's replies carry `ai_generated: true`.
  @override
  AIMessageRole get role =>
      message.extraData['ai_generated'] == true ? AIMessageRole.assistant : AIMessageRole.user;

  @override
  String get text => message.text ?? '';

  @override
  bool get isGenerating => message.extraData['generating'] == true;

  @override
  final List<AIMessagePart> parts;
}

In the same file, AgentTransport is how an AIConversation reaches the agent. It follows the channel's messages and what the agent is doing, asks the agent, and stops it. Its approver() makes the approver that answers the tools that ask before they run:

class AgentTransport with AIConversationTransport<AgentMessage> {
  AgentTransport._(this.session, this._channel);

  /// Opens the transport for a session with your agent.
  static Future<AgentTransport> open(AgentSession session) async {
    // The agent writes the conversation into a channel, which the views read.
    final channel = await session.chat();
    await channel.watch();
    return AgentTransport._(session, channel);
  }

  final AgentSession session;
  final stream.Channel _channel;
  final _subscriptions = <StreamSubscription<Object?>>[];

  // Whether the latest reply failed, until the next one starts: the session goes on saying
  // the agent responds.
  var _failed = false;

  @override
  void connect(AIConversationFeed<AgentMessage> feed) {
    _subscriptions
      // The channel lists its messages oldest first, as the conversation does. A deleted
      // message stays in the channel, so leave it out.
      ..add(
        _channel.state!.messagesStream.listen((messages) {
          feed.updateMessages([
            for (final message in messages)
              if (message.deletedAt == null) AgentMessage(message),
          ]);
        }),
      )
      // What the agent is doing while its reply shows nothing yet.
      ..add(
        session.conversation.stream.listen((conversation) {
          if (!_failed) feed.updateActivity(_activityOf(conversation.state));
        }),
      )
      // A reply the model couldn't write leaves nothing in the channel: only the session says so.
      ..add(
        session.events().listen((event) {
          if (event.kind == AgentEventKind.responding) {
            _failed = false;
            feed.updateActivity(_activityOf(session.state));
          } else if (event.kind == AgentEventKind.error && const {'llm', 'sts'}.contains(event['context'])) {
            _failed = true;
            feed.updateActivity(const AIAgentActivity(state: AIAgentActivityState.error));
          }
        }),
      );
  }

  static AIAgentActivity? _activityOf(ConversationState state) => switch (state) {
    Responding() => const AIAgentActivity(state: AIAgentActivityState.thinking),
    Working(:final skills) => AIAgentActivity(
      state: AIAgentActivityState.checkingExternalSources,
      text: 'Working on ${skills.join(' and ')}',
    ),
    _ => null,
  };

  @override
  Future<void> send(AIConversationRequest<AgentMessage> request) async {
    await session.responses.create(request.text);
  }

  @override
  Future<void> stop() async => session.interrupt();

  /// Answers the tools that ask before they run, as the signed-in user. Dispose it with
  /// the conversation.
  AIToolApprover approver({AIToolAuthorizer? authorizer}) => AIToolApprover(
    identity: AIClientIdentity(userId: session.backend.userId),
    decide: (call, decision) async {
      // The session ignores an answer to a call it isn't waiting on, such as one asked before
      // the app was relaunched, so say so rather than show the answer as sent.
      if (!session.approvals.value.any((request) => request.id == call.id)) {
        throw StateError('This question is no longer waiting for an answer.');
      }
      session.decide(call.id, allowed: decision == AIToolApprovalDecision.allowed);
    },
    authorizer: authorizer,
  );

  /// Stops following the channel and the session, as the conversation is disposed.
  @override
  void disconnect() {
    for (final subscription in _subscriptions) {
      unawaited(subscription.cancel());
    }
    _subscriptions.clear();
  }
}

The agent doesn't answer a message sent to the channel, so send asks the session, and the router writes the question into the channel itself. session.chat() connects Stream Chat as the user from step 3, and watch() keeps the channel's messages up to date. approver() answers the tools that ask before they run. It answers the tools your app gives the session: a tool your server hosts asks only on a voice call, as Approval explains.

Step 5: Show the conversation

Open a session with your agent, and show it with AIConversationView and the composer:

import 'package:flutter/material.dart';
import 'package:stream_chat_flutter_ai/stream_chat_flutter_ai.dart';
import 'package:stream_agents_core/stream_agents_core.dart';

import 'agent_transport.dart';

class AssistantScreen extends StatefulWidget {
  const AssistantScreen({super.key, required this.agents});

  final StreamAgents agents;

  @override
  State<AssistantScreen> createState() => _AssistantScreenState();
}

class _AssistantScreenState extends State<AssistantScreen> {
  // Opens a session with the agent you synced in step 1.
  late final Future<AgentTransport> _transport = widget.agents
      .agent('my-agent')
      .chat()
      .then(AgentTransport.open);

  @override
  Widget build(BuildContext context) => FutureBuilder(
    future: _transport,
    builder: (context, snapshot) {
      if (snapshot.data case final transport?) return AssistantView(transport: transport);
      // The session couldn't be opened, such as with a token that has expired.
      if (snapshot.hasError) return Center(child: Text('${snapshot.error}'));
      return const Center(child: CircularProgressIndicator());
    },
  );
}

class AssistantView extends StatefulWidget {
  const AssistantView({super.key, required this.transport});

  final AgentTransport transport;

  @override
  State<AssistantView> createState() => _AssistantViewState();
}

class _AssistantViewState extends State<AssistantView> {
  late final _approver = widget.transport.approver();
  late final _conversation = AIConversation(
    transport: widget.transport,
    approver: _approver,
    actions: AIConversationActions.rate,
  );

  @override
  void dispose() {
    _conversation.dispose();
    _approver.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) => Column(
    children: [
      Expanded(child: AIConversationView.conversation(conversation: _conversation)),
      ChatComposer.conversation(conversation: _conversation, enableSpeechToText: true),
    ],
  );
}

Show it once the user has signed in, with the StreamAgents that signIn returns, such as from your app's main:

void main() {
  final agents = signIn('jane');
  runApp(
    MaterialApp(
      home: Scaffold(body: SafeArea(child: AssistantScreen(agents: agents))),
    ),
  );
}

chat() throws an AgentsException when the session can't be opened, for example when the agent's name is wrong, or when the test token from step 3 has expired. The FutureBuilder then shows the error in place of the progress indicator.

Run the app and ask something. The agent thinks, then its reply writes itself out, with Markdown, code and charts, and with the reasoning and tool calls that led to it. While it writes, the send button is a stop button, and under each finished reply the user can copy, share and rate it.

AIConversation also offers to edit the user's latest message and regenerate the latest reply. A session keeps its conversation in Stream Chat, which can't be rewound, so AgentTransport would ask those again as new messages below rather than replacing them. actions: AIConversationActions.rate leaves them out, as described in Actions.

The Conversation View guide covers what else the screen can show, such as suggestions while the conversation is empty. send here passes only the text: to send the photos picked in the composer too, see the Personal assistant use case.

Step 6: Add voice

The composer already takes voice: with enableSpeechToText: true, while the field is empty its send button is a microphone that dictates into it with the platform's speech recognizer.

To let the user talk to the agent instead, start a voice session. The agent joins a call right away, and the phone joins it with the microphone on:

import 'package:stream_agents_rtc/stream_agents_rtc.dart';

final voice = await VoiceSession.start(agents, agent: 'my-agent');
await voice.join();

join() doesn't throw when the call can't be joined: the error is in voice.state.value.failure. Declare the permissions a call needs first: RECORD_AUDIO, MODIFY_AUDIO_SETTINGS, BLUETOOTH_CONNECT and ACCESS_NETWORK_STATE in AndroidManifest.xml, and an NSMicrophoneUsageDescription in Info.plist. Without the description, iOS closes the app when it asks for the microphone. On Android, ask for the microphone at runtime before you start the voice session, as the Voice agent use case does.

Give the agent a face with AIVoiceGlowView, which moves differently while the agent is connecting, listening, thinking and speaking. The Voice agent use case builds the whole screen: the agent's state, the visualizations, mute and hang up, and what was said written out as a conversation. An agent made with --template voice also lets you pick its speech-to-text and text-to-speech models, its voice and its greeting.

Theming

The package's defaults come from StreamTheme, the ThemeExtension the Stream Chat widgets read too: its color scheme, text theme, spacing, radii and icons. Register one to restyle the AI components and Stream Chat together. With none, they use Stream's defaults for the app's brightness:

MaterialApp(
  theme: ThemeData(
    extensions: [
      StreamTheme(colorScheme: StreamColorScheme.fromSeed(brand: brandBlue)),
    ],
  ),
);

stream_chat_flutter_ai re-exports StreamTheme and its tokens from stream_core_flutter, so this code needs no other import besides Flutter's material.dart.

To change one component, set its theme on AITheme, also a ThemeExtension on your app's ThemeData, such as composerTheme for ChatComposer:

MaterialApp(
  theme: ThemeData(
    extensions: const [
      AITheme(
        composerTheme: ComposerThemeData(sendButtonColor: Color(0xFF005FFF)),
        suggestionsTheme: SuggestionsThemeData(backgroundColor: Color(0xFFEFF4FF)),
        chartTheme: ChartThemeData(seriesColors: [Color(0xFF005FFF), Color(0xFF00C1FF)]),
      ),
    ],
  ),
);

A field you leave out follows the StreamTheme, in light and dark mode alike. The code block stays dark in both modes unless you set its brightness, as described in Code and charts. To theme a subtree only, wrap it in the component's scope, such as ComposerTheme.

Each component's guide lists its theme's fields under Appearance or Theming, such as Conversation View and Reasoning and Tool Calls.

Replacing components

A component that AIConversationView, ChatComposer or the markdown draws for you, such as the user's message bubble, a code block or the composer's "+" button, is a builder of StreamComponentFactory, the component factory of stream_core_flutter that the Stream Chat widgets read too. streamAIComponentBuilders turns the builders you pass into the factory's extensions. Each builder is optional. Leave one out and its component keeps its default:

Widget codeBlock(BuildContext context, AICodeBlockProps props) =>
    MyCodeBlock(code: props.code, language: props.language);

StreamComponentFactory(
  builders: StreamComponentBuilders(
    extensions: streamAIComponentBuilders(codeBlock: codeBlock),
  ),
  child: MaterialApp(home: home),
);

A nested StreamComponentFactory replaces its parent's builders instead of adding to them, so register every builder in one factory. With Stream Chat, that's the extensions of the StreamComponentBuilders you pass to StreamChat's componentBuilders, as shown in Stream Chat integration.

Localization

Every text the components show, such as the composer's "Ask anything..." placeholder or "Thought for 12s", resolves through an AITranslations instance. Register nothing and the widgets render English. To translate, implement AITranslations for each language you support:

class DutchTranslations implements AITranslations {
  const DutchTranslations();

  @override
  String get composerHint => 'Vraag maar';

  @override
  String get send => 'Verstuur';

  @override
  String clearOption(String option) => '$option wissen';

  // ...and every other string.
}

Then register the translations with AITranslationsDelegate, a regular LocalizationsDelegate. GlobalMaterialLocalizations and GlobalWidgetsLocalizations come from the Flutter SDK's flutter_localizations package, which you add to your pubspec.yaml with sdk: flutter:

MaterialApp(
  localizationsDelegates: const [
    AITranslationsDelegate({
      'nl': DutchTranslations(),
    }),
    GlobalMaterialLocalizations.delegate,
    GlobalWidgetsLocalizations.delegate,
  ],
  supportedLocales: const [Locale('en'), Locale('nl')],
);

The package ships English only. The strings are listed in its ai_translations.dart.

The components

Next steps

• Customer support agent: answers from your help center, order lookups, a refund the user confirms, and a handover to your team.
• Personal assistant: a list of conversations, the calendar and reminders as tools on the phone, and a voice mode.
• Voice agent: a voice conversation with the agent as a glowing globe, and what was said written out.
• Stream Agents: the agent itself, its tools, knowledge, connectors and phone numbers.