# AI Integrations The AI UI components are designed specifically for AI-first applications written in Flutter. When paired with our real-time [Chat API](https://getstream.io/chat/), they make it easier to integrate with and render responses from LLM providers such as ChatGPT, Gemini, Anthropic or any custom backend, by providing out-of-the-box components able to render Markdown, code blocks, tables, charts, thinking indicators and more. The `stream_chat_flutter_ai` package has no dependency on `stream_chat`, `stream_chat_flutter` or any other Stream Chat package. Every widget works with plain strings, callbacks and controllers, so you can drop it into any Flutter app and pair it with any backend. Wiring it up to a Stream Chat channel is covered in the [Stream Chat integration](https://getstream.io/chat/docs/sdk/flutter/guides/ai-integrations/stream-chat-integration/) guide. This library includes the following components: - `StreamingMessageView` - renders text, markdown and code in real time, using a character-by-character animation, similar to ChatGPT. - `AITypingIndicatorView` - displays the different states of the LLM (thinking, checking external sources, etc). - `ChatComposer` - a fully featured prompt composer with attachments, chat options, speech input, and a send button that turns into a stop button while a response streams. - `AISuggestionsView` - conversation starters for your users. - `SpeechToTextButton` - records voice input and streams the recognized transcript into the composer. - `ChartView` - renders line, bar, area, scatter, bubble, pie, histogram and heatmap charts from the JSON an LLM emits. - `CodeBlockView` - a framed code block with a language label, a copy button and opt-in syntax highlighting. - `AIToolRegistry` - client-side tool calling, letting the agent reach into your app. See [Client-side tools](https://getstream.io/chat/docs/sdk/flutter/guides/ai-integrations/client-side-tools/). You can find a complete sample app that uses these components [here](https://github.com/GetStream/chat-ai-samples/tree/main/flutter). The package repository also includes a [showcase app](https://github.com/GetStream/stream-chat-flutter-ai/tree/main/packages/stream_chat_flutter_ai/example) that demonstrates every component and needs no API key to run. ## Installation Add `stream_chat_flutter_ai` to your `pubspec.yaml`. It requires Dart 3.11 or later and Flutter 3.41 or later. ```yaml dependencies: stream_chat_flutter_ai: ^0.1.0 ``` Then import it: ```dart import 'package:stream_chat_flutter_ai/stream_chat_flutter_ai.dart'; ``` ### Optional integrations The package is deliberately small, so two features need a library that you add yourself. Without them the components still work, but code and formulas render as plain text. | Feature | Add to your app | What you provide | | ------------------- | ------------------------------------- | ------------------------------------------------------------------------------ | | Syntax highlighting | `re_highlight` (or any tokenizer) | A `codeHighlighter` callback. See [Syntax highlighting](#syntax-highlighting). | | Math (LaTeX) | `flutter_math_fork` (or any renderer) | A `mathBuilder` callback. See [LaTeX](#latex). | ```yaml dependencies: stream_chat_flutter_ai: ^0.1.0 re_highlight: ^0.0.3 # code highlighting flutter_math_fork: ^0.7.4 # math rendering ``` Neither library is required, and you can swap in a different one, since the package only asks for a callback. Opening links from `onTapLink` also needs a launcher of your choice, such as `url_launcher`. ### Platform requirements The attachment sheet and speech input use the camera, the photo library, the microphone and the speech recognizer, so each platform needs the matching permissions. The values below are the ones the [sample app](https://github.com/GetStream/chat-ai-samples/tree/main/flutter) uses, and you can change the descriptions. Skip the entries for features you turn off. **iOS**, in `ios/Runner/Info.plist`: ```xml NSCameraUsageDescription Camera access is needed to attach a new photo to your message. NSPhotoLibraryUsageDescription Photo library access is needed to attach images to your message. NSMicrophoneUsageDescription Microphone access is needed for voice input. NSSpeechRecognitionUsageDescription Speech recognition is used to convert your voice to text. ``` **macOS**, in `macos/Runner/Info.plist`: ```xml NSPhotoLibraryUsageDescription Photo library access is needed to attach images to your message. NSMicrophoneUsageDescription Microphone access is needed for voice input. NSSpeechRecognitionUsageDescription Speech recognition is used to convert your voice to text. ``` The macOS app sandbox also needs entitlements in `DebugProfile.entitlements` and `Release.entitlements`: ```xml com.apple.security.device.audio-input com.apple.security.device.microphone com.apple.security.personal-information.photos-library com.apple.security.files.user-selected.read-only ``` The sample doesn't request camera access on macOS. If you enable camera capture there, also add `NSCameraUsageDescription` and the `com.apple.security.device.camera` entitlement. **Android**, in `android/app/src/main/AndroidManifest.xml`. The recent-photos strip needs `READ_MEDIA_IMAGES` and `READ_MEDIA_VIDEO` on Android 13 and later, even when you only attach images. `READ_MEDIA_VISUAL_USER_SELECTED` supports the "limited access" selection on Android 14 and later: ```xml ``` Speech recognition on Android 11 and later also needs package visibility for the recognizer. Add this inside the existing `` element, or the microphone button finds no speech recognizer: ```xml ``` ### Streaming message view `StreamingMessageView` renders markdown efficiently while the text is still growing. It supports tables, images, code blocks and charts. Code fences are drawn with a `CodeBlockView`, and JSON chart fences are drawn with a `ChartView`. A fence is styled from its opening line onwards, so a block that is still streaming never shows the raw fence markers. Under the hood, it uses a typewriter with a character queue, similar to ChatGPT. ```dart StreamingMessageView( text: markdownText, // updated in real time as chunks arrive typingSpeed: const Duration(milliseconds: 10), onTypewriterStateChanged: (state) { // TypewriterState.typing | idle | paused | stopped }, ); ``` `typingSpeed` controls the animation speed. Use `onTypewriterStateChanged` to find out whether the view is still revealing text, for example to keep a "Generating" indicator visible until the animation finishes. If you need the typewriter effect on a layout of your own, drive it with a `TypewriterController` and a `TypewriterBuilder`: ```dart final controller = TypewriterController(text: ''); // As new chunks arrive from the LLM: controller.updateText(accumulatedText); // In your widget tree: TypewriterBuilder( controller: controller, builder: (context, value, child) => Text(value.text), ); ``` If you want markdown without the animation, use `AIMarkdownBody`, the renderer that `StreamingMessageView` is built on: ```dart AIMarkdownBody( data: markdownText, selectable: true, // enables text selection on desktop and web onTapLink: (text, href, title) { // Handle the link tap. }, ); ``` #### Syntax highlighting Code blocks render as plain monospace text by default. Highlighting is opt-in, because a grammar set is several times the size of the whole package. Add a highlighting library such as `re_highlight` to your app, and supply the tokenizer through `codeHighlighter`: ```dart StreamingMessageView( text: message, codeHighlighter: highlightCode, ); ``` The callback has a small contract: ```dart TextSpan? highlightCode(String code, String language, TextStyle baseStyle); ``` `language` arrives exactly as the fence wrote it, so case folding and alias resolution are yours to handle. Return `null` for a language you don't cover and the block renders as plain text. A highlighter that throws is reported through `FlutterError.onError`, and the block falls back to plain text as well. The `highlightCode` function used on this page is yours to define. The [sample app](https://github.com/GetStream/chat-ai-samples/blob/main/flutter/lib/src/code_highlighter.dart) contains a complete implementation built on `re_highlight`. Add `re_highlight: ^0.0.3` to your `pubspec.yaml`, copy [`code_highlighter.dart`](https://github.com/GetStream/chat-ai-samples/blob/main/flutter/lib/src/code_highlighter.dart) into your app and trim the language list to taste. Every grammar you import adds to your app size, so keep only the languages your users are likely to see. While a fence is streaming, your highlighter isn't called on every frame. It runs each time the code gains 64 characters, plus one final pass once the text stops changing. Blocks over 20,000 characters skip highlighting. Use `codeBackgroundColor` and `codeForegroundColor` to change the colors of the code block. It stays dark regardless of the ambient `Theme`. #### LaTeX The package recognizes `\(…\)` and `\[…\]`, but typesetting is left to you, so it doesn't pull in a TeX renderer. Add a renderer such as `flutter_math_fork: ^0.7.4` to your `pubspec.yaml` and provide a `mathBuilder`, and formulas render. Without one, the TeX source is shown as text. ```dart // With `flutter_math_fork` in your pubspec. StreamingMessageView( text: markdownText, mathBuilder: (context, tex, style, {required inline}) => Math.tex( tex, textStyle: style, mathStyle: inline ? MathStyle.text : MathStyle.display, ), ); ``` Pass `useDollarDelimitersForMath: true` to also accept `$…$` and `$$…$$`. It is off by default, because `$` collides with currency in ordinary prose. ### AI typing indicator view `AITypingIndicatorView` presents the current state of the LLM, such as "Thinking" or "Checking sources". You can specify any text you need. The dots pulse while the indicator is shown. ```dart AITypingIndicatorView( text: 'Thinking', dotColor: Colors.blue, dotCount: 3, dotSize: 8, ); ``` ### Chat composer `ChatComposer` gives users a modern text-entry surface. It has an attachment sheet behind a leading "+" button (camera, recent photos, the photo library and your chat options), a badge for the selected chat option, and a send button that becomes a stop button while a response streams. ```dart ChatComposer( controller: controller, enableSpeechToText: true, onSendPressed: (text, selectedOption, attachments) async { await myBackend.sendMessage(text, attachments); }, onStopPressed: () => myBackend.stopGenerating(), ); ``` The composer clears the field as soon as `onSendPressed` is invoked, without waiting for the returned `Future`. If your send can fail, you own surfacing that error and restoring the text. `ChatComposerController` holds the composer state. Create it with the chat options you want to offer, and flip `isGenerating` to swap between the send and stop buttons: ```dart final controller = ChatComposerController( chatOptions: [ ChatOption(id: 'summarize', text: 'Summarize this', icon: Icons.summarize), ChatOption(id: 'email', text: 'Write an email', icon: Icons.email), ], ); // While the AI is generating: controller.isGenerating = true; // When the response finishes: controller.isGenerating = false; ``` By default the composer refuses to send while `isGenerating` is `true`. If your backend accepts follow-up messages mid-stream, pass `allowSendWhileGenerating: true`. #### Customizing the composer To replace part of the composer, subclass `ChatComposerFactory` and override the slot you want to change. Each slot receives a props object that carries the composer's wiring (`controller`, `focusNode`, `onSend` and `onStop`). | Slot | Props | Default | Returns | | ---------------------- | ---------------------------------- | ---------------------------- | -------------------------- | | `buildLeading` | `ChatComposerLeadingProps` | outlined circular "+" button | `Widget?`, `null` hides it | | `buildTrailing` | `ChatComposerTrailingProps` | nothing | `Widget?`, `null` hides it | | `buildInput` | `ChatComposerInputProps` | `ChatComposerInput` | `Widget` | | `buildAttachmentSheet` | `ChatComposerAttachmentSheetProps` | `ComposerAttachmentSheet` | `Widget` | ```dart class MyComposerFactory extends ChatComposerFactory { // Replace the leading "+" button. @override Widget? buildLeading(BuildContext context, ChatComposerLeadingProps props) { return IconButton(icon: const Icon(Icons.attach_file), onPressed: () {}); } // Keep the default input, but decorate around it. @override Widget buildInput(BuildContext context, ChatComposerInputProps props) { return Padding( padding: const EdgeInsets.only(bottom: 4), child: ChatComposerInput(props: props), ); } } ChatComposer( factory: MyComposerFactory(), controller: controller, onSendPressed: send, ); ``` A few things to keep in mind: - `buildInput` is placed in an `Expanded` by the composer, so don't return an `Expanded` yourself. - `props.onSend` clears the controller and returns focus to `props.focusNode`, so call it rather than reimplementing the send path. `props.onStop` is `null` when you passed no `onStopPressed`. - `props.canSend` combines "there is something to send" and "a response isn't already streaming". Use it to enable or disable a custom send button. - To change one value and keep the rest, use `props.copyWith(...)`. ### Suggestions view `AISuggestionsView` is a horizontally scrolling row of quick-reply chips, typically docked above the composer on a "new chat" screen. Tapping a chip calls back with its text, and you decide what happens next. ```dart Column( children: [ const Spacer(), AISuggestionsView( suggestions: const [ 'Create a painting in Renaissance style', 'Help me study vocabulary for an exam', ], onSuggestionSelected: (text) => sendMessage(text), ), ], ); ``` ### Speech to text Set `enableSpeechToText: true` on `ChatComposer` and the send button shows a microphone while the field is empty, then swaps to send as soon as the user types. Dictation streams partial and final results straight into the composer. ```dart ChatComposer( controller: controller, enableSpeechToText: true, onSendPressed: send, ); ``` You can also place `SpeechToTextButton` yourself, for example in the composer's trailing slot through `ChatComposerFactory`, which is empty by default. Placed that way it is always visible. Pass options through a `SpeechToTextConfig`: ```dart SpeechToTextButton( controller: controller, config: SpeechToTextConfig( localeId: 'en-US', // optional, defaults to the device locale listenFor: const Duration(seconds: 30), pauseFor: const Duration(seconds: 3), onError: (error) => debugPrint(error.errorMsg), ), ); ``` Speech input needs platform permissions. See [Platform requirements](#platform-requirements). ### Charts `ChartView` renders a `USpec` as a line, bar, area, scatter, bubble, pie, histogram or heatmap chart. `StreamingMessageView` already renders JSON chart fences for you. Use `ChartView` directly when you parse the model output yourself. `USpecParser` reads USpec, Chart.js, Plotly, ECharts, Highcharts and Vega-Lite payloads: ```dart final spec = USpecParser.tryParse(jsonString); if (spec != null) ChartView(spec: spec); ``` Each chart describes itself to screen readers, for example "Bar chart, Messages per day, 5 categories, values 8 to 24". Pass `semanticsLabel` to replace that sentence, or an empty string to describe the chart yourself. ## Theming Colors come from `AITheme`, a `ThemeExtension` that you register on your app's `ThemeData`. It carries one data class per component: `ChartThemeData`, `ComposerThemeData` and `SuggestionsThemeData`. ```dart MaterialApp( theme: ThemeData( colorSchemeSeed: brandBlue, extensions: const [ AITheme( composerTheme: ComposerThemeData(sendButtonColor: Color(0xFF005FFF)), suggestionsTheme: SuggestionsThemeData(backgroundColor: Color(0xFFEFF4FF)), chartTheme: ChartThemeData(seriesColors: [Color(0xFF005FFF), Color(0xFF00C1FF)]), ), ], ), ); ``` Every field is nullable, and an unset field is derived from the ambient `ThemeData`. Overriding the send button leaves the rest of the composer following your app, in light and dark mode alike. To theme a subtree only, wrap it in `ComposerTheme`, `SuggestionsTheme` or `ChartTheme`: ```dart ComposerTheme( data: const ComposerThemeData(sendButtonColor: Colors.teal), child: ChatComposer(onSendPressed: send), ) ``` A few things to keep in mind: - `ThemeData.copyWith(extensions:)` replaces the whole extension set. If your app already registers other extensions, re-list them: `theme.copyWith(extensions: [...theme.extensions.values, const AITheme()])`. - Nesting a scope replaces the enclosing one instead of layering on top of it, the way `IconTheme` does. Use `ComposerTheme.merge`, `SuggestionsTheme.merge` or `ChartTheme.merge` to add to the enclosing scope. - The suggestion chips default to the same tokens as the composer's input, since they are normally docked right above it. ## Localization Every string the package renders resolves through an `AITranslations` instance. Register nothing and the widgets render English. To translate, subclass `DefaultAITranslations` and override what you need: ```dart class DutchTranslations extends DefaultAITranslations { const DutchTranslations(); @override String get composerHint => 'Vraag maar'; @override String get send => 'Verstuur'; @override String clearOption(String option) => '$option wissen'; } ``` Then register the translations with `AITranslationsDelegate`, a regular `LocalizationsDelegate`: ```dart MaterialApp( localizationsDelegates: const [ AITranslationsDelegate({ 'nl': DutchTranslations(), }), GlobalMaterialLocalizations.delegate, GlobalWidgetsLocalizations.delegate, ], supportedLocales: const [Locale('en'), Locale('nl')], ); ``` Keys use the `Locale.toString()` form, such as `'nl'` or `'pt_BR'`. To pin one language over part of the tree, wrap it in an `AITranslationsScope`: ```dart AITranslationsScope( translations: const DutchTranslations(), child: ChatComposer(onSendPressed: send), ) ``` Most strings are tooltips on icon-only buttons, which makes them the only accessible labels those buttons have. Translating them is what makes the composer usable with a screen reader in another language. ## Best practices ### Mark assistant messages with `ai_generated` The Chat SDK treats a reply from your AI bot like any other message. To render it with `StreamingMessageView` and its typewriter animation, the app needs a way to tell AI replies apart from the rest. The convention used by the AI components, the sample apps and the Stream backend SDKs is a custom field, `ai_generated: true`, set by the backend when it creates the assistant's placeholder message: ```js label="Node.js" const { message } = await channel.sendMessage({ text: "", user_id: botId, ai_generated: true, }); ``` ```py label="Python" placeholder = channel.send_message( MessageRequest(text="", user_id=bot_id, custom={"ai_generated": True}), ).data.message ``` In Flutter the field is available as `message.extraData['ai_generated']`. Use it in two places: - **Rendering.** Build AI messages with `StreamingMessageView` instead of the default bubble, as shown in the [Stream Chat integration](https://getstream.io/chat/docs/sdk/flutter/guides/ai-integrations/stream-chat-integration/) guide. A message without the flag renders as a regular text bubble that jumps to the new text on every update. - **Your backend.** The bot usually listens for new messages in the channel. Checking `ai_generated` lets it skip its own replies instead of answering itself. ### Drive the AI indicator from events Before the first token arrives, the LLM can spend a while thinking or calling tools, and the message is still empty. The backend reports what it's doing with `ai_indicator.*` custom events on the channel, and the Flutter SDK decodes them for you: | Event type | `EventType` constant | Sent by | Use it to | | --------------------- | ----------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------- | | `ai_indicator.update` | `EventType.aiIndicatorUpdate` | Backend | Show the current state of the LLM. `event.aiState` carries the state and `event.messageId` the reply it belongs to. | | `ai_indicator.clear` | `EventType.aiIndicatorClear` | Backend | Hide the indicator. Sent after the final text is stored. | | `ai_indicator.stop` | `EventType.aiIndicatorStop` | App | Ask the backend to stop the reply. See [Let users stop the response](#let-users-stop-the-response). | The state of an update is an `AITypingState`: | `ai_state` | `AITypingState` | Suggested UI | | --------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------- | | `AI_STATE_THINKING` | `thinking` | `AITypingIndicatorView(text: 'Thinking')` | | `AI_STATE_EXTERNAL_SOURCES` | `checkingSources` | `AITypingIndicatorView(text: 'Checking sources')` | | `AI_STATE_GENERATING` | `generating` | Show a "Generating" caption while the text streams into the message, and a stop button. | | `AI_STATE_ERROR` | `error` | Hide the indicator. The backend writes the error into the message itself. You can also show an error caption. | A typical reply sends `AI_STATE_THINKING` right after the placeholder, `AI_STATE_EXTERNAL_SOURCES` while a tool call runs, `AI_STATE_GENERATING` with the first token, and `ai_indicator.clear` once the final text is stored. On failure the backend sends an `ai_indicator.update` with `AI_STATE_ERROR` instead of a clear, so treat `error` as the end of a reply too. Indicator events only reach clients that are connected when they're sent. They aren't stored or replayed, so a user who opens the channel halfway through a reply never gets the earlier `AI_STATE_THINKING` or `AI_STATE_GENERATING`. Use them for the indicator only, and rely on the message's `text` field, which arrives with every message update, for the reply itself. The [Stream Chat integration](https://getstream.io/chat/docs/sdk/flutter/guides/ai-integrations/stream-chat-integration/) guide shows a handler that listens for these events, and how the sample app turns the states into a caption. ### Let users stop the response A long reply can take a while, and the user may already have what they need or see the model heading the wrong way. `ChatComposer` swaps its send button for a stop button while `controller.isGenerating` is `true`, and calls `onStopPressed` when it's tapped. Call `channel.stopAIResponse()` from there, which sends an `ai_indicator.stop` event to the channel: ```dart ChatComposer( controller: controller, onSendPressed: (text, selectedOption, attachments) => channel.sendMessage(Message(text: text)), onStopPressed: () => channel.stopAIResponse(), ); ``` The stop event carries only the channel, not a message id, so on the backend treat it as "stop whatever reply is in flight in this channel". Abort the LLM stream, then finish the same way a normal reply finishes: store the text generated so far with a regular partial update, then send `ai_indicator.clear`. Skipping that final update leaves the message stuck in its last stored state, as described in the next section. The [Stream Chat AI SDK](https://getstream.io/chat/docs/sdk/flutter/guides/ai-integrations/stream-chat-ai-sdk/) and the [Stream Chat LangChain SDK](https://getstream.io/chat/docs/sdk/flutter/guides/ai-integrations/stream-chat-langchain-sdk/) already listen for `ai_indicator.stop` and do this for you. ### Use ephemeral updates while the response is streaming The components above render whatever your backend writes into the assistant's message. A typical backend posts an empty placeholder message as the AI bot, then keeps updating it as the LLM streams tokens back. Each update reaches the app as a `message.updated` event, and `StreamingMessageView` animates the new text in. How the backend sends those updates matters. A regular partial update (`partialUpdateMessage`) writes the message to the database on every call. A long reply flushed every few tokens turns into dozens or hundreds of writes, and with several conversations running at once you can quickly hit the [rate limits](https://getstream.io/chat/docs/node/rate-limits/) for message updates. For the updates sent while the response is still streaming, use `ephemeralUpdateMessage` instead: - It accepts the same payload as a partial update (`set`, `unset` and the acting user). - It sends the same `message.updated` event to everyone watching the channel, so the live typing in the app looks exactly the same. - It doesn't store anything in the database, it only sends the event. Skipping the database write is what keeps high-frequency streaming from getting rate limited. **The last update must not be ephemeral.** When the LLM finishes, send the complete text with a regular partial update. Ephemeral updates are never stored, so if the final one is ephemeral too, anyone who opens the channel later or relaunches the app loads the last stored version of the message: the empty placeholder. The same applies to whatever else ends the stream, such as an error message or the user stopping the generation. `ephemeralUpdateMessage` is available only on the server side. Call it from your backend, not from the Flutter app. In Node.js it ships in `stream-chat` 9.27.0 and later, and every server-side SDK exposes an equivalent: ```js label="Node.js" // While the LLM is streaming: send the latest text without storing it. await client.ephemeralUpdateMessage( messageId, { set: { text, generating: true } }, botId, ); // Once the LLM has finished: store the final text. await client.partialUpdateMessage( messageId, { set: { text, generating: false } }, botId, ); ``` ```python label="Python" # While the LLM is streaming: send the latest text without storing it. client.chat.ephemeral_message_update( id=message_id, set={"text": text, "generating": True}, user_id=bot_id, ) # Once the LLM has finished: store the final text. client.chat.update_message_partial( id=message_id, set={"text": text, "generating": False}, user_id=bot_id, ) ``` ```ruby label="Ruby" # While the LLM is streaming: send the latest text without storing it. client.chat.ephemeral_message_update(message_id, Models::UpdateMessagePartialRequest.new( set: { 'text' => text, 'generating' => true }, user_id: bot_id, )) # Once the LLM has finished: store the final text. client.chat.update_message_partial(message_id, Models::UpdateMessagePartialRequest.new( set: { 'text' => text, 'generating' => false }, user_id: bot_id, )) ``` ```php label="PHP" // While the LLM is streaming: send the latest text without storing it. $client->ephemeralMessageUpdate($messageId, new Models\UpdateMessagePartialRequest( set: (object)["text" => $text, "generating" => true], userID: $botId, )); // Once the LLM has finished: store the final text. $client->updateMessagePartial($messageId, new Models\UpdateMessagePartialRequest( set: (object)["text" => $text, "generating" => false], userID: $botId, )); ``` ```go label="Go" // While the LLM is streaming: send the latest text without storing it. _, err := client.Chat().EphemeralMessageUpdate(ctx, messageID, &getstream.EphemeralMessageUpdateRequest{ Set: getstream.PtrTo(map[string]any{ "text": text, "generating": true, }), UserID: getstream.PtrTo(botID), }) // Once the LLM has finished: store the final text. _, err = client.Chat().UpdateMessagePartial(ctx, messageID, &getstream.UpdateMessagePartialRequest{ Set: getstream.PtrTo(map[string]any{ "text": text, "generating": false, }), UserID: getstream.PtrTo(botID), }) ``` ```csharp label="C#" // While the LLM is streaming: send the latest text without storing it. await chat.EphemeralMessageUpdateAsync(messageId, new UpdateMessagePartialRequest { Set = new Dictionary { { "text", text }, { "generating", true }, }, UserID = botId, }); // Once the LLM has finished: store the final text. await chat.UpdateMessagePartialAsync(messageId, new UpdateMessagePartialRequest { Set = new Dictionary { { "text", text }, { "generating", false }, }, UserID = botId, }); ``` ```java label="Java" // While the LLM is streaming: send the latest text without storing it. chat.ephemeralMessageUpdate(messageId, EphemeralMessageUpdateRequest.builder() .set(Map.of("text", text, "generating", true)) .userID(botId) .build()).execute(); // Once the LLM has finished: store the final text. chat.updateMessagePartial(messageId, UpdateMessagePartialRequest.builder() .set(Map.of("text", text, "generating", false)) .userID(botId) .build()).execute(); ``` Even ephemeral updates are delivered to every device watching the channel, so don't send one per token. Flushing every 20 or so chunks, or at most every 50 to 100 ms, keeps the typing smooth without wasting bandwidth. The [AI message streaming](https://getstream.io/chat/docs/node/ai-message-streaming/) guide covers the full backend flow, including throttling and the AI indicator events that drive `AITypingIndicatorView`. If you use the [Stream Chat AI SDK](https://getstream.io/chat/docs/sdk/flutter/guides/ai-integrations/stream-chat-ai-sdk/) or the [Stream Chat LangChain SDK](https://getstream.io/chat/docs/sdk/flutter/guides/ai-integrations/stream-chat-langchain-sdk/) on your backend, this is already handled for you: both send ephemeral updates while streaming and store the final text with a regular partial update. --- For the most recent version of this documentation, visit [https://getstream.io/chat/docs/sdk/flutter/guides/ai-integrations/](https://getstream.io/chat/docs/sdk/flutter/guides/ai-integrations/).