# AI integrations overview > **Info:** This is release candidate documentation for **Stream Chat React SDK v15**. For the latest stable version, see the [latest version (v14)](https://getstream.io/chat/docs/sdk/react/). The AI UI components are built for AI-first React apps. Paired with the real-time [Chat API](https://getstream.io/chat/), they make it easier to integrate and render responses from LLM providers like ChatGPT, Gemini, Anthropic, or a custom backend. You get rich, out-of-the-box components for markdown, syntax-highlighted code, tables, thinking indicators, charts, and more. The components ship with `stream-chat-react` behind a dedicated entry point, `stream-chat-react/ai-components`: - `StreamingMessage` - renders text/markdown/code in real time with a typewriter animation. - `AIMarkdown` - renders AI-generated markdown with syntax highlighting, tables and Chart.js charts. - `AIMessageComposer` - a full prompt composer with attachments, speech input and model selection. - `AIStateIndicator` - keeps users informed while the AI generates a response. The entry point ships with `stream-chat-react` from 15.0.0-rc.2, and needs no other package. A complete ChatGPT-style app built with these components is in the [`examples/ai-chatbot`](https://github.com/GetStream/stream-chat-react/tree/master/examples/ai-chatbot) folder of the `stream-chat-react` repository. [SDK integration](https://getstream.io/chat/docs/sdk/react/v15/features/ai/sdk-integration/) covers the AI support in the main `stream-chat-react` entry point: `StreamedMessageText`, `AIStateIndicator`, and the `StopAIGenerationButton` the default composer shows. ## Usage Import the components from the entry point, and load the stylesheet next to the default SDK styles: ```tsx import { AIMarkdown, AIMessageComposer, AIStateIndicator, StreamingMessage, } from "stream-chat-react/ai-components"; import "stream-chat-react/dist/css/index.css"; import "stream-chat-react/dist/css/ai-components.css"; ``` > > **Note:** Coming from the standalone `@stream-io/chat-react-ai` package? See [Migrating from `@stream-io/chat-react-ai`](#migrating-from-stream-iochat-react-ai) below. > ## Components All components are designed to work with the existing [React Chat SDK](https://getstream.io/chat/sdk/react/tutorial/). The [developer guide](https://getstream.io/chat/solutions/ai-integration/) walks through AI integrations with Stream. The components render `stream-chat-react` icons, which you can replace through `WithComponents` like any other SDK icon. Their labels go through `Streami18n` like the rest of the SDK: inside `Chat` (or a `TranslationProvider`) they follow the active language, and outside it they render their built-in English copy. English is the only bundled language, so register a dictionary to translate them, as described in the [localization guide](https://getstream.io/chat/docs/sdk/react/v15/ui/customizing/localization/). ### `StreamingMessage` `StreamingMessage` renders markdown efficiently, with syntax highlighting for major languages. It supports common markdown elements like tables, inline code, headings, and lists. Under the hood, it types the text out a few letters at a time, on a timer. | Name | Type | Required | Description | | ---------------------- | -------- | -------- | ---------------------------------------------------- | | `text` | `string` | yes | Markdown text to render. | | `letterIntervalMs` | `number` | no | Interval between typewriter ticks. Defaults to `30`. | | `renderingLetterCount` | `number` | no | Letters rendered per tick. Defaults to `2`. | The component accepts a `ref` (`StreamingMessageRef`) that exposes `skipAnimation()`, which shows the full text immediately. If you need a custom version, combine `AIMarkdown` with the `useMessageTextStreaming` hook from `stream-chat-react`. The hook takes `streamingLetterIntervalMs` instead of `letterIntervalMs`. #### Example ```tsx import { StreamingMessage } from "stream-chat-react/ai-components"; const markdownText = ` # Heading some text ## Another heading `; ; ``` ### `AIMarkdown` `AIMarkdown` is the markdown renderer that `StreamingMessage` uses. Render it directly when the text is already complete and doesn't need the typewriter animation. | Name | Type | Required | Description | | -------------------- | --------------------------------------------------- | -------- | ------------------------------------------------------------- | | `children` | `string` | yes | Markdown to render. | | `toolComponents` | `Record>` | no | Components that render fenced code blocks by language name. | | `markdownComponents` | `Components` from `react-markdown` | no | Overrides for the default markdown elements (`code`, `pre`…). | Fenced code blocks tagged `chartjs`, or `json` that matches the Chart.js data schema, render as a chart. The chart renderer is loaded lazily and shows a "Loading chart..." fallback until it's ready. If it fails to load or the data is invalid, the block falls back to the raw code instead of breaking the message. To render your own tool output, register a component for its language. It receives the raw block content as `data`, plus a `fallback` element to return when the data can't be rendered: ```tsx import { AIMarkdown } from "stream-chat-react/ai-components"; import type { ToolComponentProps } from "stream-chat-react/ai-components"; // Renders a fenced block tagged `weather`, for example {"city":"Oslo","temperature":4}. const WeatherTool = ({ data, fallback }: ToolComponentProps) => { try { const { city, temperature } = JSON.parse(data) as { city: string; temperature: number; }; return (
{city}: {temperature}°C
); } catch { return fallback; } }; // Declared outside the component, so AIMarkdown gets the same object on every render. const toolComponents = { weather: WeatherTool }; export const MessageContent = ({ text }: { text: string }) => ( {text} ); ``` ### `AIStateIndicator` `AIStateIndicator` shows what the LLM is doing, for example "Thinking" or "Checking external sources", based on the states your backend sends. Pass the text, or let it pick a built-in phrase, such as "Brewing up an answer". It picks the phrase when it mounts, so change its `key` to pick a new one. It renders with `role="status"`, so screen readers read the text. | Name | Type | Required | Description | | ------ | -------- | -------- | ------------------------------------ | | `text` | `string` | no | Text displayed inside the component. | #### Example ```tsx import { AIStates } from "stream-chat"; import { useAIState, useChannel } from "stream-chat-react"; import { AIStateIndicator } from "stream-chat-react/ai-components"; // Render inside Channel. Shows a built-in phrase while the LLM thinks. export const ThinkingIndicator = () => { const channel = useChannel(); const { aiState } = useAIState(channel); return aiState === AIStates.Thinking ? : null; }; ``` Pass `text` to show your own words instead: ``. `stream-chat-react` also exports an `AIStateIndicator` from its main entry point. That one follows the channel's AI state by itself, while this one only draws the indicator. [Drive the AI indicator from events](#drive-the-ai-indicator-from-events) shows how to use them together. ### `AIMessageComposer` `AIMessageComposer` is a full message composer with text input, file attachments, speech-to-text, and model selection. | Name | Type | Required | Description | | -------------------------- | -------------------------------------------- | -------- | -------------------------------------------------------------------------------------- | | `resetAttachmentsOnSelect` | `boolean` | no | Resets file input after selection. Defaults to `true`. | | `nameMapping` | `{ message?: string; attachments?: string }` | no | Maps custom input names to internal state. Default names: `message` and `attachments`. | | `onSubmit` | `(e: FormEvent) => void` | no | Form submission handler. | | `onChange` | `(e: FormEvent) => void` | no | Form change handler. | | `disabled` | `boolean` | no | Disables every input in the composer. | | | `...HTMLFormElement props` | no | Supports all standard HTML form props. | #### Sub-components - **`AIMessageComposer.FileInput`** - File input button. Supports multiple file selection. The native file input stays focusable for keyboard users, and the round button is the pointer target. - **`AIMessageComposer.TextInput`** - Text input field, synced with composer state. - **`AIMessageComposer.SpeechToTextButton`** - Toggles speech-to-text input via the Web Speech API. See [`AIMessageComposer.SpeechToTextButton`](#aimessagecomposerspeechtotextbutton). - **`AIMessageComposer.SubmitButton`** - Submits the prompt. - **`AIMessageComposer.ModelSelect`** - Model selector dropdown. Customizable via `options`. - **`AIMessageComposer.AttachmentPreview`** - Preview container for attachments. - **`AIMessageComposer.AttachmentPreview.Item`** - Preview item; renders images differently for `image/*` types. The sub-components are static properties of `AIMessageComposer`, not separate exports. #### Example ```tsx import type { FormEvent } from "react"; import { AIMessageComposer, useAttachments, } from "stream-chat-react/ai-components"; // Reads the composer's attachments, so it renders inside AIMessageComposer. const AttachmentPreviews = () => { const { attachments, removeAttachment } = useAttachments(); return ( {attachments.map((attachment) => ( removeAttachment(attachment.id)} /> ))} ); }; export const ChatComposer = ({ onPrompt, }: { onPrompt: (data: FormData) => void; }) => { const handleSubmit = (event: FormEvent) => { event.preventDefault(); onPrompt(new FormData(event.currentTarget)); event.currentTarget.reset(); }; return ( ); }; ``` Resetting the form clears the composer's text and attachments. > > **Note:** `TextInput` is named `"message"` by default. `FileInput` has no default `name`, so give it `name="attachments"` for the composer to pick up the selected files. You can use other names and map them back with `nameMapping`. > The composer state is available inside `AIMessageComposer` through the `useText`, `useAttachments`, `useIsDisabled` and `useAIMessageComposerContext` hooks, all exported from `stream-chat-react/ai-components`. ### `AIMessageComposer.SpeechToTextButton` `AIMessageComposer.SpeechToTextButton` handles voice input via the Web Speech API, with a built-in mic icon and listening state UI. Render it inside `AIMessageComposer`: it writes the transcript into the composer's text input. | Name | Type | Required | Description | | --------- | ---------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | `options` | `UseSpeechToTextOptions` | no | Speech recognition options (`lang`, `continuous`, `interimResults`, `maxAlternatives`), plus `onTranscript` and `onError` callbacks of your own. | | | `...HTMLButtonElement props` | no | Supports standard HTML button props. | #### Example ```tsx import { AIMessageComposer } from "stream-chat-react/ai-components"; function VoiceComposer() { return ( ); } ``` > > **Note:** The button renders nothing in browsers without the Web Speech API (for example Firefox). Support can only be detected in the browser, so with server rendering the button appears after hydration. For voice input outside `AIMessageComposer`, use the `useSpeechToText` hook, which returns `isSupported`, `isListening`, `startListening` and `stopListening`. > ### AI States A channel's AI state says how far the reply has come. `stream-chat` exports the states as `AIStates`: - `AIStates.Thinking`, `AI_STATE_THINKING`: the LLM is working out an answer. - `AIStates.Generating`, `AI_STATE_GENERATING`: the answer is being written into the message. - `AIStates.ExternalSources`, `AI_STATE_EXTERNAL_SOURCES`: the LLM is checking external sources, such as a tool call. - `AIStates.Error`, `AI_STATE_ERROR`: the reply failed. - `AIStates.Stop`, `AI_STATE_STOP`: the user asked to stop the reply. - `AIStates.Idle`, `AI_STATE_IDLE`: nothing is happening. The client keeps the channel's state in `channel.state`, from the backend's `ai_indicator.*` events, and [`useAIState(channel)`](https://getstream.io/chat/docs/sdk/react/v15/reference/hooks/ai-hooks/#useaistate) reads it. Pass it the channel, for example from `useChannel()`: it doesn't read one from context. [Drive the AI indicator from events](#drive-the-ai-indicator-from-events) shows which event sets which state. If your backend uses states of its own, read them the same way and replace the default components. The [React assistant tutorial](https://getstream.io/blog/react-assistant/) walks through a complete integration. ### Theming The AI components follow the `stream-chat-react` theme. Inside `.str-chat` they pick up the SDK's light and dark theme tokens, and outside of it they fall back to built-in light colors. To adjust them, set the `--str-chat__ai-*` CSS variables. The stylesheet never declares these variables itself, so your values win at any scope, including `:root`: ```css :root { --str-chat__ai-bg-primary: #0f172a; --str-chat__ai-border: #334155; --str-chat__ai-syntax-keyword: #c084fc; } ``` | Variable (`--str-chat__ai-…`) | Used for | Falls back to SDK token (`--str-chat__…`) | | -------------------------------------------------------------------- | --------------------------------------------- | ----------------------------------------------------------------------------------------- | | `bg-primary` | Code block background | `background-core-elevation-0` | | `text-secondary` | AI state indicator text and dots | `text-secondary` | | `border` | Composer, model select and attachment borders | `border-core-default` | | `surface-hover` / `surface-pressed` | Composer button hover and listening states | `background-utility-hover` / `background-utility-selected` | | `button-bg` | Attachment delete and retry buttons | `background-core-surface-strong` | | `inline-code-bg` | Inline code | `background-core-surface-default` | | `table-border`, `table-header-bg`, `table-header-text`, `table-text` | Markdown tables | `border-core-default`, `background-core-surface-subtle`, `text-primary`, `text-secondary` | | `syntax-*` | Code highlighting palette | None, built-in palette | The code palette variables are `--str-chat__ai-syntax-text`, `-selection-bg`, `-comment`, `-punctuation`, `-number`, `-keyword`, `-tag`, `-string`, `-function`, `-cyan`, `-error` and `-whitespace`. Focus rings use the SDK's `--str-chat__border-utility-focused` token. The rules use the `str-chat__ai-` class prefix (for example `str-chat__ai-message-composer__form` and `str-chat__ai-streaming-message`), so you can target them with your own CSS, in a [CSS layer](https://getstream.io/chat/docs/sdk/react/v15/ui/customizing/theming/) like the rest of the SDK styles. ## Migrating from `@stream-io/chat-react-ai` The components used to ship as the standalone `@stream-io/chat-react-ai` package, which they replace. To move to `stream-chat-react/ai-components`: 1. **Dependencies.** Remove `@stream-io/chat-react-ai`, and upgrade `stream-chat-react` to v15. Remove `material-symbols` as well if you only installed it for these components: they render `stream-chat-react` icons now. 2. **Imports.** Replace the package name in imports, and the stylesheet path: | Before | After | | ------------------------------------------- | ---------------------------------------------- | | `@stream-io/chat-react-ai` | `stream-chat-react/ai-components` | | `@stream-io/chat-react-ai/styles/index.css` | `stream-chat-react/dist/css/ai-components.css` | 3. **Class names and CSS variables.** The `aicr__` prefix became `str-chat__ai-`. `aicr__ai-message-composer…` becomes `str-chat__ai-message-composer…`, every other `aicr__…` class becomes `str-chat__ai-…`, and `--aicr__…` variables become `--str-chat__ai-…`. The [full mapping table](https://github.com/GetStream/stream-chat-react/blob/master/src/plugins/AIComponents/README.md#3-class-names-and-css-custom-properties) lists every renamed class and variable. 4. **`useMessageTextStreaming`.** The hook is no longer exported next to the AI components. Import it from `stream-chat-react` and rename its `letterIntervalMs` option to `streamingLetterIntervalMs`. The `StreamingMessage` props haven't changed. 5. **`SpeechToTextButton`.** It's only available as `AIMessageComposer.SpeechToTextButton`, and its `options` prop is now passed to the speech recognizer. The component APIs are otherwise unchanged. ## Best practices - Start with the default AI components before building custom UX. - Keep AI state messaging clear and aligned with backend state names. - Rate-limit streaming updates to avoid UI jank. - Provide accessible labels for AI controls and speech input. - Ensure model selection options are validated server-side. ### Mark assistant messages with `ai_generated` The Chat SDK treats a reply from your AI bot like any other message. To render it with `StreamedMessageText` and its typewriter animation, the app needs a way to tell AI replies apart from the rest. The convention used throughout these guides, the sample apps and the Stream backend SDKs is a custom field, `ai_generated: true`, set by the backend when it creates the assistant's placeholder message: ```js label="Node.js" const { message } = await channel.sendMessage({ text: "", user_id: botId, ai_generated: true, }); ``` ```py label="Python" placeholder = channel.send_message( MessageRequest(text="", user_id=bot_id, custom={"ai_generated": True}), ).data.message ``` `ai_generated` is custom message data, so the client puts it under `message.custom`: read it as `message.custom.ai_generated`. Declare it for TypeScript, together with the `generating` field described below, as optional fields, so messages without them still type-check. [Custom data types](https://getstream.io/chat/docs/sdk/react/v15/state/state-and-entities/data-conventions/#custom-data-types) explains the declaration file: ```ts title="stream-custom-data.d.ts" import "stream-chat"; declare module "stream-chat" { interface CustomMessageData { ai_generated?: boolean; generating?: boolean; } } ``` Use the field in three places: - **Rendering.** Pass `isMessageAIGenerated={(message) => !!message.custom?.ai_generated}` to `Chat`. For every message it returns `true` for, `MessageUI` renders `StreamedMessageText` instead of `MessageText`, which is what gives the reply its typewriter animation. A message without the flag renders as regular text that jumps to the new text on every update. The [SDK integration](https://getstream.io/chat/docs/sdk/react/v15/features/ai/sdk-integration/#streamedmessagetext) shows the setup, and how to swap in `StreamingMessage`. - **The rest of the message UI.** Every update the backend makes to the text marks the message as edited, so `MessageUI` hides the "Edited" label for messages that `isMessageAIGenerated` marks. The channel list also shows the preview of an AI reply as raw text instead of rendering its markdown. There is nothing to configure for either. - **Your backend.** The bot usually listens for new messages in the channel. Checking `ai_generated` lets it skip its own replies instead of answering itself. The second field the backend sets is `generating`, also under `message.custom` on the client. The React SDK doesn't read it: `StreamedMessageText` shows the text it mounts with at once, and then animates whatever the text grows by. That is why stored replies don't replay the animation when the user scrolls back or reopens the channel. `generating` is still the only field that tells a client a reply is in flight, so set it to `true` on interim updates and `false` on the final one, and read it in custom UI, for example to show a cursor while the reply streams. **The animation only moves forward.** `StreamedMessageText` keeps a cursor into the text and only types out what comes after it. If an update makes the text shorter than what is already on screen, the component keeps showing the old text until it mounts again. That happens when the backend replaces a half-written reply with an error message, which is what the Stream Chat AI SDK and the Stream Chat LangChain SDK do when the LLM fails mid-stream. If you render the text with your own component built on `useMessageTextStreaming`, call the `skipAnimation` it returns when the new text doesn't start with the text on screen. ### Drive the AI indicator from events Before the first token arrives, the LLM can spend a while thinking or calling tools, and the message is still empty. The backend reports what it's doing with `ai_indicator.*` custom events on the channel. The client records the latest state on the channel, and the `useAIState` hook from `stream-chat-react` returns it as `aiState`: | Event type | Sent by | `aiState` becomes | Use it to | | --------------------- | ------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | `ai_indicator.update` | Backend | The event's `ai_state` | Show the current state of the LLM. The event also carries the `cid`, the `message_id` of the reply and an optional `ai_message`. | | `ai_indicator.clear` | Backend | `AI_STATE_IDLE` | Hide the indicator. Sent after the final text is stored. | | `ai_indicator.stop` | App | `AI_STATE_STOP` | Ask the backend to stop the reply. See [Let users stop the response](#let-users-stop-the-response). | Compare `aiState` with the `AIStates` constants. The [AI states](#ai-states) section above describes what each state means: | `ai_state` | `AIStates` | Built-in `stream-chat-react` UI | Suggested UI | | --------------------------- | -------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | | `AI_STATE_THINKING` | `AIStates.Thinking` | `AIStateIndicator` shows "Thinking...", and the stop button replaces the send button. | A "Thinking" indicator and a stop button. | | `AI_STATE_EXTERNAL_SOURCES` | `AIStates.ExternalSources` | Nothing. The indicator hides and the send button comes back. | A "Checking external sources" indicator and a stop button. | | `AI_STATE_GENERATING` | `AIStates.Generating` | `AIStateIndicator` shows "Generating...", and the stop button stays. | Hide the indicator, the text is now streaming into the message. Keep the stop button. | | `AI_STATE_ERROR` | `AIStates.Error` | Nothing. | Hide the indicator. The backend writes the error into the message itself. | A typical reply sends `AI_STATE_THINKING` right after the placeholder, `AI_STATE_EXTERNAL_SOURCES` while a tool call runs, `AI_STATE_GENERATING` with the first token, and `ai_indicator.clear` once the final text is stored. To show the suggested UI, combine `useAIState` with the `AIStateIndicator` from this package, and render it inside `Channel`: ```tsx import { AIStates, type Channel as StreamChannel } from "stream-chat"; import { AIStateIndicator as AIStateIndicatorView } from "stream-chat-react/ai-components"; import { Channel, MessageComposer, MessageList, useAIState, useChannel, } from "stream-chat-react"; const indicatorText: Record = { [AIStates.Thinking]: "Thinking", [AIStates.ExternalSources]: "Checking external sources", }; const AIIndicator = () => { const channel = useChannel(); const { aiState } = useAIState(channel); const text = indicatorText[aiState]; return text ? : null; }; const ChannelView = ({ channel }: { channel: StreamChannel }) => ( ); ``` `stream-chat-react` exports an `AIStateIndicator` of its own, the one described in the [SDK integration](https://getstream.io/chat/docs/sdk/react/v15/features/ai/sdk-integration/#aistateindicator). That one follows the channel's AI state itself, but only shows the thinking and generating states. The one from `stream-chat-react/ai-components` only draws the indicator, which is why the example renames it on import. The [sample app](https://github.com/GetStream/stream-chat-react/blob/master/examples/ai-chatbot/src/components/AIStateIndicator/AIStateIndicator.tsx) uses the same pattern. A few things to know about `useAIState`: - **Pass it the channel.** It takes the channel as its argument and has no default. Called without one, it never subscribes and always returns `AI_STATE_IDLE`. Inside `Channel`, get it from `useChannel()`. - **It reads the channel's own state.** The AI state lives on the channel, and the client keeps it up to date from the events. The hook reads whatever the client has collected for the channel you pass, so when you switch channels it shows the new channel's current state rather than starting from `AI_STATE_IDLE`. `Channel` keeps its children mounted when the channel changes, so local state in your own components carries over to the next channel: reset it yourself if it belongs to one channel. - **`AI_STATE_ERROR` stays.** The Stream backend SDKs don't send `ai_indicator.clear` after an error, so `aiState` stays `AI_STATE_ERROR` until the next reply starts, or until the connection drops, which resets it to `AI_STATE_IDLE`. Anything you show for it stays up until then. Indicator events are only delivered live. They aren't stored or replayed, so a client that wasn't receiving the channel's events when they were sent sees no indicator and no stop button until the next event arrives. Use the events for the indicator only, and rely on the message's `text` and `generating` fields, which arrive with every message update, for the reply itself. ### Let users stop the response A long reply can take a while, and the user may already have what they need or see the model heading the wrong way. The default composer swaps its send button for `StopAIGenerationButton` while `aiState` is `AI_STATE_THINKING` or `AI_STATE_GENERATING`, and calls `Channel.stopAIResponse()` when it's clicked. That sends an `ai_indicator.stop` event to the channel. To restyle or remove the button, override it as described in the [SDK integration](https://getstream.io/chat/docs/sdk/react/v15/features/ai/sdk-integration/#stopaigenerationbutton). Your component receives the `onClick` that sends the event. The default button doesn't show in these cases: - **During `AI_STATE_EXTERNAL_SOURCES`.** The composer only checks for thinking and generating, so the send button comes back while a tool call runs. - **When another control takes its place.** In a channel with slow mode, the cooldown timer replaces it until the cooldown ends. With `audioRecordingEnabled` set on `MessageComposer`, the voice recording button replaces it whenever the input is empty, which it usually is while a reply streams. - **With `AIMessageComposer`.** Replacing `MessageComposerUI` with `AIMessageComposer` also replaces the default composer actions, and `AIMessageComposer` has no stop button of its own. For those cases, render your own button, for example inside your `AIMessageComposer`: ```tsx import { AIStates, type AIState } from "stream-chat"; import { useAIState, useChannel } from "stream-chat-react"; const activeStates: AIState[] = [ AIStates.Thinking, AIStates.ExternalSources, AIStates.Generating, ]; const StopButton = () => { const channel = useChannel(); const { aiState } = useAIState(channel); if (!activeStates.includes(aiState)) return null; return ( ); }; ``` `type="button"` keeps the click from submitting the surrounding form. Neither button hides itself on click. They go away once an `ai_indicator.stop` or the backend's `ai_indicator.clear` reaches the channel and moves `aiState` out of the active states. When an `ai_indicator.stop` event reaches the channel, the default `StreamedMessageText` also jumps straight to the full text received so far instead of typing out the rest. `StreamingMessage` and custom components built on `useMessageTextStreaming` don't listen for it. If you want the same behavior, call their `skipAnimation` on the event: `useMessageTextStreaming` returns it, and `StreamingMessage` exposes it through its `ref`. The stop event carries only the channel's `cid`, not a message id, so on the backend treat it as "stop whatever reply is in flight in this channel". Abort the LLM stream, then finish the same way a normal reply finishes: store the text generated so far with a regular partial update and `generating: false`, then send `ai_indicator.clear`. Skipping that final update leaves the message stuck in its last stored state, as described in the next section. The [Stream Chat AI SDK](https://getstream.io/chat/docs/sdk/react/v15/features/ai/stream-chat-ai-sdk/) and the [Stream Chat LangChain SDK](https://getstream.io/chat/docs/sdk/react/v15/features/ai/stream-chat-langchain-sdk/) already listen for `ai_indicator.stop` and do this for you. ### Use ephemeral updates while the response is streaming The components above render whatever your backend writes into the assistant's message. A typical backend posts an empty placeholder message as the AI bot, then keeps updating it as the LLM streams tokens back. Each update reaches the app as a `message.updated` event, and `StreamingMessage` animates the new text in. How the backend sends those updates matters. A regular partial update (`partialUpdateMessage`) writes the message to the database on every call. A long reply flushed every few tokens turns into dozens or hundreds of writes, and with several conversations running at once you can quickly hit the [rate limits](https://getstream.io/chat/docs/node/rate-limits/) for message updates. For the updates sent while the response is still streaming, use `ephemeralUpdateMessage` instead: - It accepts the same payload as a partial update (`set`, `unset` and the acting user). - It sends the same `message.updated` event to everyone watching the channel, so the live typing in the app looks exactly the same. - It doesn't store anything in the database, it only sends the event. Skipping the database write is what keeps high-frequency streaming from getting rate limited. **The last update must not be ephemeral.** When the LLM finishes, send the complete text with a regular partial update and set `generating` to `false`. Ephemeral updates are never stored, so if the final one is ephemeral too, anyone who opens the channel later or reloads the page loads the last stored version of the message: the empty placeholder, still marked as generating. The same applies to whatever else ends the stream, such as an error message or the user stopping the generation. `ephemeralUpdateMessage` is available only on the server side. Call it from your backend, not from the browser. In Node.js it ships in `stream-chat` 9.27.0 and later, and every server-side SDK exposes an equivalent: ```js label="Node.js" // While the LLM is streaming: send the latest text without storing it. await client.ephemeralUpdateMessage( messageId, { set: { text, generating: true } }, botId, ); // Once the LLM has finished: store the final text. await client.partialUpdateMessage( messageId, { set: { text, generating: false } }, botId, ); ``` ```python label="Python" # While the LLM is streaming: send the latest text without storing it. client.chat.ephemeral_message_update( id=message_id, set={"text": text, "generating": True}, user_id=bot_id, ) # Once the LLM has finished: store the final text. client.chat.update_message_partial( id=message_id, set={"text": text, "generating": False}, user_id=bot_id, ) ``` ```ruby label="Ruby" # While the LLM is streaming: send the latest text without storing it. client.chat.ephemeral_message_update(message_id, Models::UpdateMessagePartialRequest.new( set: { 'text' => text, 'generating' => true }, user_id: bot_id, )) # Once the LLM has finished: store the final text. client.chat.update_message_partial(message_id, Models::UpdateMessagePartialRequest.new( set: { 'text' => text, 'generating' => false }, user_id: bot_id, )) ``` ```php label="PHP" // While the LLM is streaming: send the latest text without storing it. $client->ephemeralMessageUpdate($messageId, new Models\UpdateMessagePartialRequest( set: (object)["text" => $text, "generating" => true], userID: $botId, )); // Once the LLM has finished: store the final text. $client->updateMessagePartial($messageId, new Models\UpdateMessagePartialRequest( set: (object)["text" => $text, "generating" => false], userID: $botId, )); ``` ```go label="Go" // While the LLM is streaming: send the latest text without storing it. _, err := client.Chat().EphemeralMessageUpdate(ctx, messageID, &getstream.EphemeralMessageUpdateRequest{ Set: getstream.PtrTo(map[string]any{ "text": text, "generating": true, }), UserID: getstream.PtrTo(botID), }) // Once the LLM has finished: store the final text. _, err = client.Chat().UpdateMessagePartial(ctx, messageID, &getstream.UpdateMessagePartialRequest{ Set: getstream.PtrTo(map[string]any{ "text": text, "generating": false, }), UserID: getstream.PtrTo(botID), }) ``` ```csharp label="C#" // While the LLM is streaming: send the latest text without storing it. await chat.EphemeralMessageUpdateAsync(messageId, new UpdateMessagePartialRequest { Set = new Dictionary { { "text", text }, { "generating", true }, }, UserID = botId, }); // Once the LLM has finished: store the final text. await chat.UpdateMessagePartialAsync(messageId, new UpdateMessagePartialRequest { Set = new Dictionary { { "text", text }, { "generating", false }, }, UserID = botId, }); ``` ```java label="Java" // While the LLM is streaming: send the latest text without storing it. chat.ephemeralMessageUpdate(messageId, EphemeralMessageUpdateRequest.builder() .set(Map.of("text", text, "generating", true)) .userID(botId) .build()).execute(); // Once the LLM has finished: store the final text. chat.updateMessagePartial(messageId, UpdateMessagePartialRequest.builder() .set(Map.of("text", text, "generating", false)) .userID(botId) .build()).execute(); ``` Even ephemeral updates are delivered to every device watching the channel, so don't send one per token. Flushing every 20 or so chunks, or at most every 50 to 100 ms, keeps the typing smooth without wasting bandwidth. The [AI message streaming](https://getstream.io/chat/docs/node/ai-message-streaming/) guide covers the full backend flow, including throttling and the AI indicator events that drive `AIStateIndicator`. If you use the [Stream Chat AI SDK](https://getstream.io/chat/docs/sdk/react/v15/features/ai/stream-chat-ai-sdk/) or the [Stream Chat LangChain SDK](https://getstream.io/chat/docs/sdk/react/v15/features/ai/stream-chat-langchain-sdk/) on your backend, this is already handled for you: both send ephemeral updates while streaming and store the final text with a regular partial update. --- For the most recent version of this documentation, visit [https://getstream.io/chat/docs/sdk/react/v15/features/ai/ai-integrations-overview/](https://getstream.io/chat/docs/sdk/react/v15/features/ai/ai-integrations-overview/).