AI Integrations

The AI UI components are built for AI-first React apps. Paired with the real-time Chat API, they make it easier to integrate and render responses from LLM providers like ChatGPT, Gemini, Anthropic, or a custom backend. You get rich, out-of-the-box components for markdown, syntax-highlighted code, tables, thinking indicators, charts, and more.

The components ship with stream-chat-react behind a dedicated entry point, stream-chat-react/ai-components:

• StreamingMessage - renders text/markdown/code in real time with a typewriter animation.
• AIMarkdown - renders AI-generated markdown with syntax highlighting, tables and Chart.js charts.
• AIMessageComposer - a full prompt composer with attachments, speech input and model selection.
• AIStateIndicator - keeps users informed while the AI generates a response.
• VoiceWave and VoiceGlow - animated visualizations of what a voice agent is doing, with the useAudioLevel hook to move them with a live audio level.

We’ll keep iterating and adding components. If there’s something you use every day and want added, open an issue.

You can find a complete ChatGPT-style sample app built with these components in the examples/ai-chatbot folder of the stream-chat-react repository.

Usage

Import the components from the entry point, and load the stylesheet next to the default SDK styles:

import {
  AIMarkdown,
  AIMessageComposer,
  AIStateIndicator,
  StreamingMessage,
} from "stream-chat-react/ai-components";

import "stream-chat-react/css/index.css";
import "stream-chat-react/css/ai-components.css";

[!NOTE] Coming from the standalone @stream-io/chat-react-ai package? See Migrating from @stream-io/chat-react-ai below.

Components

All components are designed to work with the existing React Chat SDK. The developer guide walks through AI integrations with Stream.

The components render stream-chat-react icons, which you can replace through WithComponents like any other SDK icon. Their labels are translated through Streami18n, so render them inside Chat (or a TranslationProvider) to get translated, human-readable accessible labels.

StreamingMessage

StreamingMessage renders markdown efficiently, with syntax highlighting for major languages. It supports common markdown elements like tables, inline code, headings, and lists.

Under the hood, it uses a letter-by-letter typewriter animation with a character queue.

The component accepts a ref (StreamingMessageRef) that exposes skipAnimation(), which shows the full text immediately.

If you need a custom version, combine AIMarkdown with the useMessageTextStreaming hook from stream-chat-react. The hook takes streamingLetterIntervalMs instead of letterIntervalMs.

Example

import { StreamingMessage } from "stream-chat-react/ai-components";

const markdownText = `
# Heading

some text

## Another heading
`;

<StreamingMessage
  text={markdownText}
  letterIntervalMs={30} // every 30ms
  renderingLetterCount={3} // render 3 letters at a time
/>;

AIMarkdown

AIMarkdown is the markdown renderer that StreamingMessage uses. Render it directly when the text is already complete and doesn't need the typewriter animation.

Fenced code blocks tagged chartjs, or json that matches the Chart.js data schema, render as a chart. The chart renderer is loaded lazily and shows a "Loading chart..." fallback until it's ready. If it fails to load or the data is invalid, the block falls back to the raw code instead of breaking the message.

To render your own tool output, register a component for its language. It receives the raw block content as data, plus a fallback element to return when the data can't be rendered:

import { AIMarkdown } from "stream-chat-react/ai-components";

const toolComponents = {
  weather: ({ data, fallback }) => {
    try {
      const { city, temperature } = JSON.parse(data);
      return (
        <div className="weather-tool">
          {city}: {temperature}°C
        </div>
      );
    } catch {
      return fallback;
    }
  },
};

const MessageContent = ({ text }) => (
  <AIMarkdown toolComponents={toolComponents}>{text}</AIMarkdown>
);

AIStateIndicator

AIStateIndicator represents LLM states (for example, “Thinking” or “Checking External Sources”) based on the states you define on your backend. You can provide custom text, or let it pick a built-in phrase. To force a new phrase, change the key prop to reset its internal state. It renders with role="status", so screen readers announce the text.

Example

{
  (aiState === "thinking" || aiState === "generating") && (
    <AIStateIndicator key={resetKey} />
  );
}

// or

{
  (aiState === "thinking" || aiState === "generating") && (
    <AIStateIndicator text="Thinking of a proper response..." />
  );
}

stream-chat-react also exports an AIStateIndicator from its main entry point. That one listens to the channel's AI state events by itself, while this one only draws the indicator. Drive the AI indicator from events shows how to use them together.

AIMessageComposer

AIMessageComposer is a full message composer with text input, file attachments, speech-to-text, and model selection.

Sub-components

• AIMessageComposer.FileInput - File input button. Supports multiple file selection. The native file input stays focusable for keyboard users, and the round button is the pointer target.
• AIMessageComposer.TextInput - Text input field, synced with composer state.
• AIMessageComposer.SpeechToTextButton - Toggles speech-to-text input via the Web Speech API. See AIMessageComposer.SpeechToTextButton.
• AIMessageComposer.SubmitButton - Submits the prompt.
• AIMessageComposer.ModelSelect - Model selector dropdown. Customizable via options.
• AIMessageComposer.AttachmentPreview - Preview container for attachments.
• AIMessageComposer.AttachmentPreview.Item - Preview item; renders images differently for image/* types.

The sub-components are static properties of AIMessageComposer, not separate exports.

Example

import { AIMessageComposer } from "stream-chat-react/ai-components";

function ChatComposer({ attachments }: ChatComposerProps) {
  const handleSubmit = (e) => {
    e.preventDefault();
    const formData = new FormData(e.target);
    // Handle submission
  };

  return (
    <AIMessageComposer onSubmit={handleSubmit}>
      <AIMessageComposer.FileInput name="attachments" />
      <AIMessageComposer.TextInput name="message" />
      <AIMessageComposer.SpeechToTextButton />
      <AIMessageComposer.ModelSelect name="model" />
      <AIMessageComposer.SubmitButton />
      <AIMessageComposer.AttachmentPreview>
        {attachments.map((attachment) => (
          <AIMessageComposer.AttachmentPreview.Item
            key={attachment.id}
            {...attachment}
          />
        ))}
      </AIMessageComposer.AttachmentPreview>
    </AIMessageComposer>
  );
}

[!NOTE] TextInput is named "message" by default. FileInput has no default name, so give it name="attachments" for the composer to pick up the selected files. You can use other names and map them back with nameMapping.

The composer state is available inside AIMessageComposer through the useText, useAttachments, useIsDisabled and useAIMessageComposerContext hooks, all exported from stream-chat-react/ai-components.

AIMessageComposer.SpeechToTextButton

AIMessageComposer.SpeechToTextButton handles voice input via the Web Speech API, with a built-in mic icon and listening state UI. Render it inside AIMessageComposer: it writes the transcript into the composer's text input.

Example

import { AIMessageComposer } from "stream-chat-react/ai-components";

function VoiceComposer() {
  return (
    <AIMessageComposer>
      <AIMessageComposer.TextInput />
      <AIMessageComposer.SpeechToTextButton
        options={{
          lang: "en-US",
          continuous: false,
          interimResults: true,
        }}
      />
    </AIMessageComposer>
  );
}

[!NOTE] The button renders nothing in browsers without the Web Speech API (for example Firefox). Support can only be detected in the browser, so with server rendering the button appears after hydration. For voice input outside AIMessageComposer, use the useSpeechToText hook, which returns isSupported, isListening, startListening and stopListening.

VoiceWave

VoiceWave shows what a voice agent is doing as a single string carrying a wave packet, which erupts while the agent speaks. It's the web counterpart of AIVoiceWave in the Android and iOS AI components, with the same motion. Everything in this section also applies to VoiceGlow, which takes the same props.

Each state has its own motion:

• idle - waiting to be spoken to: a calm, barely-there shimmer.
• connecting - not ready yet: a loader sweep with a pulsing breath.
• listening - hearing somebody: a gentle expand and contract.
• thinking - working on an answer: a slow turn or a travelling wave.
• speaking - talking: strong motion that ebbs and pauses like speech.

To move with real speech, pass a getter that returns the live level, such as the one useAudioLevel returns for a MediaStream.

Size

Neither component has a size of its own. They draw in the middle of the box you give them, so size them with CSS: VoiceWave is about 80% as wide as the box's shorter side, and VoiceGlow about half. The stylesheet's defaults are width: 100%; height: 8rem for the wave and a 12rem square for the glow.

.voice-panel .str-chat__ai-voice-wave {
  height: 6rem;
}

Behavior

• It runs only while visible. The animation pauses while the element is off screen or the tab is hidden.
• Reduced motion. With prefers-reduced-motion: reduce, the motion slows to a crawl, taps and hover are ignored, and the fade-in is instant.
• Only the shape takes the pointer. The pointer reaches the component only over the string or, for VoiceGlow, the globe. Elsewhere in the box, clicks and hovers go to whatever is behind it. Over the shape, a tap bounces it and a hovering mouse or pen ripples it. Pointer events are never stopped, so your own onClick on the root or a parent still fires.
• It follows the theme. The default color and the light or dark look are read from --str-chat__ai-voice-color and --str-chat__ai-voice-backdrop about once a second, so switching themes needs no prop. See Theming.
• It fades in. The first frame fades in over 0.3 seconds, so the component never flashes empty while it starts.
• Server rendering. Under SSR the markup is rendered, and drawing starts after hydration.

Accessibility

The root renders with role="img" and an accessible name made of "Voice agent" and the state, for example "Voice agent: Speaking". The name is translated through Streami18n (the aria/Voice agent: {{ state }} key, with the state words Ready, Connecting, Listening, Thinking and Speaking), and an aria-label you pass replaces it. The drawing itself is hidden from assistive technology.

The component isn't a live region, so screen readers don't announce state changes. If users need to hear them, add a role="status" text next to it.

Example

import { VoiceWave } from "stream-chat-react/ai-components";
import "stream-chat-react/css/ai-components.css";

<VoiceWave
  colors={["#005fff", "#7c3aed"]}
  expressivity={1.4}
  state="listening"
  style={{ height: "6rem" }}
/>;

VoiceGlow

VoiceGlow shows the same states as a soft globe whose color masses orbit inside it and swell with speech. It's the web counterpart of AIVoiceGlow in the Android and iOS AI components. It takes the VoiceWave props, plus rendering:

Either way the glow moves the same; only the look and the cost differ:

• auto is the richest look: a shader computes every pixel, with noise-marbled color masses, and a tap or a hovering pointer sends out ripples. Every WebGL glow on a page draws through one shared WebGL context, so the shader compiles once and many glows stay under the browser's limit on live contexts.
• css composes the glow from CSS gradients that are painted once and animated only through transform and opacity, so the browser's compositor runs the motion. It keeps moving while the main thread is busy, for example while a long answer streams in, needs no canvas or WebGL, and runs no script per frame unless audioLevel is given. It lacks the shader's fine marbling: a tap sends out a ring instead of a ripple, and a hovering pointer does nothing. It needs @property, container query units and color-mix() (Chrome 111, Safari 16.4, Firefox 128 and later).

Example

import { VoiceGlow } from "stream-chat-react/ai-components";

<VoiceGlow
  aria-label="Assistant is thinking"
  rendering="css"
  state="thinking"
  style={{ width: "8rem", height: "8rem" }}
/>;

useAudioLevel

useAudioLevel reads the live level of a MediaStream's audio for the audioLevel prop of VoiceWave and VoiceGlow. The stream can be a microphone from getUserMedia({ audio: true }) or a remote participant's audio.

const useAudioLevel: (
  stream: MediaStream | null | undefined,
) => () => number | null;

It returns a stable getter that measures the level, from 0 to 1, each time it's called, so nothing re-renders as the level changes. The getter returns null while there is no stream or the browser has no Web Audio support, and the visualization then animates speech on its own. The hook tears its audio graph down when the stream changes and on unmount. It leaves the stream's tracks running, since they belong to you.

Example

Show the agent listening to the user's microphone:

import { useEffect, useState } from "react";
import {
  useAudioLevel,
  type VoiceAgentState,
  VoiceGlow,
  VoiceWave,
} from "stream-chat-react/ai-components";

const VoiceAgent = ({ state }: { state: VoiceAgentState }) => {
  const [mic, setMic] = useState<MediaStream | null>(null);
  const micLevel = useAudioLevel(mic);

  useEffect(() => {
    let stream: MediaStream | undefined;
    navigator.mediaDevices.getUserMedia({ audio: true }).then((granted) => {
      stream = granted;
      setMic(granted);
    });
    // The hook leaves the tracks running, so stop them yourself.
    return () => stream?.getTracks().forEach((track) => track.stop());
  }, []);

  return (
    <>
      <VoiceGlow state={state} />
      <VoiceWave
        audioLevel={state === "listening" ? micLevel : undefined}
        state={state}
      />
    </>
  );
};

For an agent on a Stream Video call, pass the agent participant's audioStream instead.

AI States

The SDK supports a set of AI states that represent progress during response generation. The current list of states is here: https://github.com/GetStream/stream-chat-react/blob/0577ffdbd2abf11b6b99a2e70caa938ea19635e9/src/components/AIStateIndicator/hooks/useAIState.ts#L7.

Provided below is a brief explanation of what each state means:

• AI_STATE_THINKING - the AI is thinking and trying to internally craft an answer to your query
• AI_STATE_GENERATING - the actual response to your query is being generated
• AI_STATE_EXTERNAL_SOURCES - the AI is checking external resources for information
• AI_STATE_ERROR - the AI has reached an error state while trying to answer your query
• AI_STATE_IDLE - the AI is in an idle state and is not doing anything

If you are using your own implementation and have different states than these, you can feel free to override our default components as well as their behaviour.

Please, refer to our React Assistant tutorial for more in-depth guidance. It is highly encouraged to read the tutorial before integrating AI in your application.

Theming

The AI components follow the stream-chat-react theme. Inside .str-chat they pick up the SDK's light and dark theme tokens, and outside of it they fall back to built-in light colors.

To adjust them, set the --str-chat__ai-* CSS variables. The stylesheet never declares these variables itself, so your values win at any scope, including :root:

:root {
  --str-chat__ai-bg-primary: #0f172a;
  --str-chat__ai-border: #334155;
  --str-chat__ai-syntax-keyword: #c084fc;
}

The code palette variables are --str-chat__ai-syntax-text, -selection-bg, -comment, -punctuation, -number, -keyword, -tag, -string, -function, -cyan, -error and -whitespace. Focus rings use the SDK's --str-chat__border-utility-focused token.

Set --str-chat__ai-voice-backdrop when a voice visualization sits on a surface whose color differs from the page's, so it picks the right light or dark look:

.voice-panel {
  --str-chat__ai-voice-color: #7c3aed;
  --str-chat__ai-voice-backdrop: #0b1020;
}

The rules use the str-chat__ai- class prefix (for example str-chat__ai-message-composer__form and str-chat__ai-streaming-message), so you can target them with your own CSS, in a CSS layer like the rest of the SDK styles.

Migrating from @stream-io/chat-react-ai

The components used to ship as the standalone @stream-io/chat-react-ai package, which they replace. To move to stream-chat-react/ai-components:

1. Dependencies. Remove @stream-io/chat-react-ai, and upgrade stream-chat-react to 14.13.0 or later. Remove material-symbols as well if you only installed it for these components: they render stream-chat-react icons now.

2. Imports. Replace the package name in imports, and the stylesheet path:

   Before	After
   @stream-io/chat-react-ai	stream-chat-react/ai-components
   @stream-io/chat-react-ai/styles/index.css	stream-chat-react/css/ai-components.css

3. Class names and CSS variables. The aicr__ prefix became str-chat__ai-. aicr__ai-message-composer… becomes str-chat__ai-message-composer…, every other aicr__… class becomes str-chat__ai-…, and --aicr__… variables become --str-chat__ai-…. The full mapping table lists every renamed class and variable.

4. useMessageTextStreaming. The hook is no longer exported next to the AI components. Import it from stream-chat-react and rename its letterIntervalMs option to streamingLetterIntervalMs. The StreamingMessage props haven't changed.

5. SpeechToTextButton. It's only available as AIMessageComposer.SpeechToTextButton, and its options prop is now passed to the speech recognizer.

The component APIs are otherwise unchanged.

Best practices

• Start with the default AI components before building custom UX.
• Keep AI state messaging clear and aligned with backend state names.
• Rate-limit streaming updates to avoid UI jank.
• Provide accessible labels for AI controls and speech input.
• Ensure model selection options are validated server-side.

Mark assistant messages with ai_generated

The Chat SDK treats a reply from your AI bot like any other message. To render it with StreamedMessageText and its typewriter animation, the app needs a way to tell AI replies apart from the rest.

The convention used throughout these guides, the sample apps and the Stream backend SDKs is a custom field, ai_generated: true, set by the backend when it creates the assistant's placeholder message:

const { message } = await channel.sendMessage({
  text: "",
  user_id: botId,
  ai_generated: true,
});

placeholder = channel.send_message(
    MessageRequest(text="", user_id=bot_id, custom={"ai_generated": True}),
).data.message

ai_generated is custom message data, so declare it for TypeScript, together with the generating field described below. See TypeScript and custom data types for how the declaration file is picked up:

import "stream-chat";

declare module "stream-chat" {
  interface CustomMessageData {
    ai_generated?: boolean;
    generating?: boolean;
  }
}

Use the field in three places:

• Rendering. Pass isMessageAIGenerated={(message) => !!message.ai_generated} to Chat. For every message it returns true for, MessageUI renders StreamedMessageText instead of MessageText, which is what gives the reply its typewriter animation. A message without the flag renders as regular text that jumps to the new text on every update. The SDK integration shows the setup, and how to swap in StreamingMessage.
• The rest of the message UI. Every update the backend makes to the text marks the message as edited, so MessageUI hides the "Edited" label for messages that isMessageAIGenerated marks. The channel list also shows the preview of an AI reply as raw text instead of rendering its markdown. There is nothing to configure for either.
• Your backend. The bot usually listens for new messages in the channel. Checking ai_generated lets it skip its own replies instead of answering itself.

The second field the backend sets is generating. The React SDK doesn't read it: StreamedMessageText shows the text it mounts with at once, and then animates whatever the text grows by. That is why stored replies don't replay the animation when the user scrolls back or reopens the channel. generating is still the only field that tells a client a reply is in flight, so set it to true on interim updates and false on the final one, and read it in custom UI, for example to show a cursor while the reply streams.

The animation only moves forward. StreamedMessageText keeps a cursor into the text and only types out what comes after it. If an update makes the text shorter than what is already on screen, the component keeps showing the old text until it mounts again. That happens when the backend replaces a half-written reply with an error message, which is what the Stream Chat AI SDK and the Stream Chat LangChain SDK do when the LLM fails mid-stream. If you render the text with your own component built on useMessageTextStreaming, call the skipAnimation it returns when the new text doesn't start with the text on screen.

Drive the AI indicator from events

Before the first token arrives, the LLM can spend a while thinking or calling tools, and the message is still empty. The backend reports what it's doing with ai_indicator.* custom events on the channel. The useAIState hook from stream-chat-react listens for them and returns the latest state as aiState:

Compare aiState with the AIStates constants. The AI states section above describes what each state means:

A typical reply sends AI_STATE_THINKING right after the placeholder, AI_STATE_EXTERNAL_SOURCES while a tool call runs, AI_STATE_GENERATING with the first token, and ai_indicator.clear once the final text is stored.

To show the suggested UI, combine useAIState with the AIStateIndicator from this package, and render it inside Channel:

import { AIStateIndicator as AIStateIndicatorView } from "stream-chat-react/ai-components";
import {
  AIStates,
  Channel,
  MessageComposer,
  MessageList,
  useAIState,
  useChannelStateContext,
} from "stream-chat-react";

const indicatorText: Record<string, string> = {
  [AIStates.Thinking]: "Thinking",
  [AIStates.ExternalSources]: "Checking external sources",
};

const AIIndicator = () => {
  const { channel } = useChannelStateContext();
  const { aiState } = useAIState(channel);
  const text = indicatorText[aiState];

  return text ? <AIStateIndicatorView text={text} /> : null;
};

const App = () => (
  <Channel>
    <MessageList />
    <AIIndicator />
    <MessageComposer />
  </Channel>
);

stream-chat-react exports an AIStateIndicator of its own, the one described in the SDK integration. That one listens for the events itself, but only shows the thinking and generating states. The one from stream-chat-react/ai-components only draws the indicator, which is why the example renames it on import. The sample app uses the same pattern.

A few things to know about useAIState:

• Pass it the channel. It takes the channel as its argument and has no default. Called without one, it never subscribes and always returns AI_STATE_IDLE. Inside Channel, get the channel from useChannelStateContext().
• Call it inside Channel. Channel mounts its children again when the active channel changes, so the state starts from AI_STATE_IDLE for each channel. If you call the hook outside Channel and switch the channel yourself, the previous channel's state stays until the next event arrives.
• Compare with AIStates.ExternalSources. The AIState type in stream-chat suggests AI_STATE_CHECKING_SOURCES, but the backends send AI_STATE_EXTERNAL_SOURCES, which is the value of AIStates.ExternalSources.
• AI_STATE_ERROR stays. The Stream backend SDKs don't send ai_indicator.clear after an error, so aiState stays AI_STATE_ERROR until the next reply starts. Anything you show for it stays up until then.

Indicator events only reach clients that are connected when they're sent. They aren't stored or replayed, and useAIState starts from AI_STATE_IDLE when it mounts, so a user who opens the channel halfway through a reply sees no indicator and no stop button until the next event arrives. Use the events for the indicator only, and rely on the message's text and generating fields, which arrive with every message update, for the reply itself.

Let users stop the response

A long reply can take a while, and the user may already have what they need or see the model heading the wrong way. The default composer swaps its send button for StopAIGenerationButton while aiState is AI_STATE_THINKING or AI_STATE_GENERATING, and calls channel.stopAIResponse() when it's clicked. That sends an ai_indicator.stop event to the channel. To restyle or remove the button, override it as described in the SDK integration. Your component receives the onClick that sends the event.

The default button doesn't show in these cases:

• During AI_STATE_EXTERNAL_SOURCES. The composer only checks for thinking and generating, so the send button comes back while a tool call runs.
• When another control takes its place. In a channel with slow mode, the cooldown timer replaces it until the cooldown ends. With audioRecordingEnabled set on MessageComposer, the voice recording button replaces it whenever the input is empty, which it usually is while a reply streams.
• With AIMessageComposer. Replacing MessageComposerUI with AIMessageComposer also replaces the default composer actions, and AIMessageComposer has no stop button of its own.

For those cases, render your own button, for example inside your AIMessageComposer:

import {
  AIStates,
  useAIState,
  useChannelStateContext,
} from "stream-chat-react";

const activeStates = [
  AIStates.Thinking,
  AIStates.ExternalSources,
  AIStates.Generating,
];

const StopButton = () => {
  const { channel } = useChannelStateContext();
  const { aiState } = useAIState(channel);

  if (!activeStates.includes(aiState)) return null;

  return (
    <button onClick={() => channel.stopAIResponse()} type="button">
      Stop
    </button>
  );
};

type="button" keeps the click from submitting the surrounding form. Neither button hides itself on click. They go away once an ai_indicator.stop or the backend's ai_indicator.clear reaches the channel and moves aiState out of the active states.

When an ai_indicator.stop event reaches the channel, the default StreamedMessageText also jumps straight to the full text received so far instead of typing out the rest. StreamingMessage and custom components built on useMessageTextStreaming don't listen for it. If you want the same behavior, call their skipAnimation on the event: useMessageTextStreaming returns it, and StreamingMessage exposes it through its ref.

The stop event carries only the channel's cid, not a message id, so on the backend treat it as "stop whatever reply is in flight in this channel". Abort the LLM stream, then finish the same way a normal reply finishes: store the text generated so far with a regular partial update and generating: false, then send ai_indicator.clear. Skipping that final update leaves the message stuck in its last stored state, as described in the next section. The Stream Chat AI SDK and the Stream Chat LangChain SDK already listen for ai_indicator.stop and do this for you.

Use ephemeral updates while the response is streaming

The components above render whatever your backend writes into the assistant's message. A typical backend posts an empty placeholder message as the AI bot, then keeps updating it as the LLM streams tokens back. Each update reaches the app as a message.updated event, and StreamingMessage animates the new text in.

How the backend sends those updates matters. A regular partial update (partialUpdateMessage) writes the message to the database on every call. A long reply flushed every few tokens turns into dozens or hundreds of writes, and with several conversations running at once you can quickly hit the rate limits for message updates.

For the updates sent while the response is still streaming, use ephemeralUpdateMessage instead:

• It accepts the same payload as a partial update (set, unset and the acting user).
• It sends the same message.updated event to everyone watching the channel, so the live typing in the app looks exactly the same.
• It doesn't store anything in the database, it only sends the event. Skipping the database write is what keeps high-frequency streaming from getting rate limited.

The last update must not be ephemeral. When the LLM finishes, send the complete text with a regular partial update and set generating to false. Ephemeral updates are never stored, so if the final one is ephemeral too, anyone who opens the channel later or reloads the page loads the last stored version of the message: the empty placeholder, still marked as generating. The same applies to whatever else ends the stream, such as an error message or the user stopping the generation.

ephemeralUpdateMessage is available only on the server side. Call it from your backend, not from the browser. In Node.js it ships in stream-chat 9.27.0 and later, and every server-side SDK exposes an equivalent:

// While the LLM is streaming: send the latest text without storing it.
await client.ephemeralUpdateMessage(
  messageId,
  { set: { text, generating: true } },
  botId,
);

// Once the LLM has finished: store the final text.
await client.partialUpdateMessage(
  messageId,
  { set: { text, generating: false } },
  botId,
);

# While the LLM is streaming: send the latest text without storing it.
client.chat.ephemeral_message_update(
    id=message_id,
    set={"text": text, "generating": True},
    user_id=bot_id,
)

# Once the LLM has finished: store the final text.
client.chat.update_message_partial(
    id=message_id,
    set={"text": text, "generating": False},
    user_id=bot_id,
)

# While the LLM is streaming: send the latest text without storing it.
client.chat.ephemeral_message_update(message_id, Models::UpdateMessagePartialRequest.new(
  set: { 'text' => text, 'generating' => true },
  user_id: bot_id,
))

# Once the LLM has finished: store the final text.
client.chat.update_message_partial(message_id, Models::UpdateMessagePartialRequest.new(
  set: { 'text' => text, 'generating' => false },
  user_id: bot_id,
))

// While the LLM is streaming: send the latest text without storing it.
$client->ephemeralMessageUpdate($messageId, new Models\UpdateMessagePartialRequest(
    set: (object)["text" => $text, "generating" => true],
    userID: $botId,
));

// Once the LLM has finished: store the final text.
$client->updateMessagePartial($messageId, new Models\UpdateMessagePartialRequest(
    set: (object)["text" => $text, "generating" => false],
    userID: $botId,
));

// While the LLM is streaming: send the latest text without storing it.
_, err := client.Chat().EphemeralMessageUpdate(ctx, messageID, &getstream.EphemeralMessageUpdateRequest{
    Set: getstream.PtrTo(map[string]any{
        "text":       text,
        "generating": true,
    }),
    UserID: getstream.PtrTo(botID),
})

// Once the LLM has finished: store the final text.
_, err = client.Chat().UpdateMessagePartial(ctx, messageID, &getstream.UpdateMessagePartialRequest{
    Set: getstream.PtrTo(map[string]any{
        "text":       text,
        "generating": false,
    }),
    UserID: getstream.PtrTo(botID),
})

// While the LLM is streaming: send the latest text without storing it.
await chat.EphemeralMessageUpdateAsync(messageId, new UpdateMessagePartialRequest
{
    Set = new Dictionary<string, object>
    {
        { "text", text },
        { "generating", true },
    },
    UserID = botId,
});

// Once the LLM has finished: store the final text.
await chat.UpdateMessagePartialAsync(messageId, new UpdateMessagePartialRequest
{
    Set = new Dictionary<string, object>
    {
        { "text", text },
        { "generating", false },
    },
    UserID = botId,
});

Even ephemeral updates are delivered to every device watching the channel, so don't send one per token. Flushing every 20 or so chunks, or at most every 50 to 100 ms, keeps the typing smooth without wasting bandwidth. The AI message streaming guide covers the full backend flow, including throttling and the AI indicator events that drive AIStateIndicator.

If you use the Stream Chat AI SDK or the Stream Chat LangChain SDK on your backend, this is already handled for you: both send ephemeral updates while streaming and store the final text with a regular partial update.