Streamed answers with a stop button

An assistant's answer can take a while to write.

Note:

What you'll build

  • The answer typing itself out as it's written
  • A status line for what the assistant is doing
  • A Stop button in place of the send button

Preview

A channel with the assistant: the user asks which vegetables are easy for a first season, the assistant's answer is half written, with two of its list items so far, a status line under it reads Writing…, and the composer shows a round blue Stop button where the send button was

Approach: Replace one part. Choosing where to start explains the approaches.

Starts from: the base app. The page shows only what the recipe adds.

Note:

The answers come from an AI agent on your backend, which watches the channel and writes as the assistant user. For each answer it:

  1. Posts an empty answer marked ai_generated.
  2. Sends ai_indicator.update with ai_state set to AI_STATE_THINKING and the answer's message_id, for example with Channel.updateAIState(). While a tool call runs, it sends AI_STATE_EXTERNAL_SOURCES, and with the first words, AI_STATE_GENERATING.
  3. Writes the answer into the message a few words at a time. Each update reaches the app as message.updated.
  4. Stores the final text and sends ai_indicator.clear, for example with Channel.clearAIIndicator(). If the answer fails, it sends AI_STATE_ERROR instead.
  5. Stops writing when the app sends ai_indicator.stop, stores the text it has, and sends ai_indicator.clear.

Drive the AI indicator from events and Let users stop the response explain each step. The cookbook's demo app has no agent, so its screenshots play these events in the page.

Best practices

  • Let the SDK type out the answer. MessageUI renders an AI-generated message with StreamedMessageText, which types out whatever the text grows by, and shows a stored answer in full.
  • Say what the assistant is doing, in words, where the answer appears. Dots alone don't tell thinking from checking sources.
  • Keep Stop where the send button was, so the user finds it without looking, and give it an accessible name, since it shows only an icon.
  • Don't make the growing text a live region: a screen reader would read every update. Announce the state changes instead, and that the answer is finished, through the SDK's announcer.
  • When Stop goes away, move the focus to the text field, so a keyboard user isn't left on nothing.

Recognise the assistant's replies

The backend marks each of the assistant's replies with the custom field ai_generated. isMessageAIGenerated() reads it. Passed to Chat, it makes MessageUI render the text of those messages with StreamedMessageText, and leave out the "Edited" label every streamed update would add. SDK integration explains StreamedMessageText, and Mark assistant messages with ai_generated how the backend sets the field.

src/isMessageAIGenerated.ts
import type { LocalMessage } from "stream-chat";

declare module "stream-chat" {
  // The field the backend sets on the assistant's replies. Optional, because every message in the
  // app shares this interface.
  interface CustomMessageData {
    ai_generated?: boolean;
  }
}

/** The assistant wrote the message: the backend marks its replies with `ai_generated`. */
export const isMessageAIGenerated = (message: LocalMessage) =>
  !!message.custom?.ai_generated;

Show what the assistant is doing

AssistantStatus fills the TypingIndicator slot, which the message list renders after its last message, where the answer appears. It reads the channel's AI state with useAIState(), which follows channel.state.aiState, and shows a line for each state the assistant works in: "Thinking…", "Checking sources…", "Writing…", and "Couldn't finish the answer" after an error. While the assistant is idle, it renders the SDK's TypingIndicator. The slot fills the thread's message list too, and the assistant answers in the channel, so a thread keeps the default.

The AIStateIndicator from stream-chat-react shows only the thinking and generating states, and no SDK component renders it, so the recipe draws its own line.

src/AssistantStatus.tsx
import { useEffect } from "react";
import { AIStates } from "stream-chat";
import type { AIState } from "stream-chat";
import {
  TypingIndicator,
  useAIState,
  useChannel,
  useThreadContext,
} from "stream-chat-react";
import type { TypingIndicatorProps } from "stream-chat-react";
import { useAnswerAnnouncements } from "./useAnswerAnnouncements";
import "./AssistantStatus.css";

/** What the status line says in each state that shows it. */
const labels: Partial<Record<AIState, string>> = {
  [AIStates.Error]: "Couldn’t finish the answer",
  [AIStates.ExternalSources]: "Checking sources…",
  [AIStates.Generating]: "Writing…",
  [AIStates.Thinking]: "Thinking…",
};

/**
 * The assistant's state at the end of the channel's message list, where the answer appears, and
 * the default typing indicator while the assistant is idle. Screen reader announcements come from
 * `useAnswerAnnouncements()`, so the line itself isn't a live region.
 */
const ChannelAssistantStatus = (props: TypingIndicatorProps) => {
  const { isMessageListScrolledToBottom = true, scrollToBottom } = props;
  const channel = useChannel();
  const { aiState } = useAIState(channel);
  const label = labels[aiState];
  useAnswerAnnouncements(aiState);

  // Keeps the line in view when it appears, as the typing indicator does.
  useEffect(() => {
    if (label && isMessageListScrolledToBottom) scrollToBottom?.();
  }, [isMessageListScrolledToBottom, label, scrollToBottom]);

  if (!label) return <TypingIndicator {...props} />;

  return (
    <div className="cookbook-assistant-status" data-ai-state={aiState}>
      {aiState !== AIStates.Error && (
        <span aria-hidden="true" className="cookbook-assistant-status__dots">
          <span />
          <span />
          <span />
        </span>
      )}
      {label}
    </div>
  );
};

/**
 * Fills the `TypingIndicator` slot. The assistant answers in the channel, so a thread keeps the
 * default typing indicator.
 */
export const AssistantStatus = (props: TypingIndicatorProps) => {
  const thread = useThreadContext();
  return thread ? (
    <TypingIndicator {...props} />
  ) : (
    <ChannelAssistantStatus {...props} />
  );
};

While the assistant is thinking

The assistant's channel right after the user's question: the assistant's answer is still an empty bubble, a status line with three blue dots under it reads Thinking…, and the composer already shows the Stop button

The message list isn't a live region, so a screen reader doesn't read the answer as it grows; the SDK only announces "New message from Assistant" when the empty answer is posted. useAnswerAnnouncements() adds the rest through useAriaLiveAnnouncer(), the SDK's polite announcer: that the assistant started answering, and that the answer is finished, stopped or failed. It announces each change once, and an ending only after an answer, so a status line that mounts again on an earlier error says nothing. The user then reads the finished answer like any other message.

src/useAnswerAnnouncements.ts
import { useEffect, useRef } from "react";
import { AIStates } from "stream-chat";
import type { AIState } from "stream-chat";
import { useAriaLiveAnnouncer } from "stream-chat-react";

/** The assistant is working on an answer: thinking, checking sources or writing it. */
const isAnswering = (aiState: AIState) =>
  aiState === AIStates.Thinking ||
  aiState === AIStates.ExternalSources ||
  aiState === AIStates.Generating;

/** What to say when an answer ends in each state that isn't answering. */
const endings: Partial<Record<AIState, string>> = {
  [AIStates.Error]: "The assistant couldn’t finish the answer",
  [AIStates.Idle]: "The assistant finished answering",
  [AIStates.Stop]: "Answer stopped",
};

/**
 * Tells a screen reader how the assistant's answer goes, without reading every streamed word:
 * that it started, and that it finished, stopped or failed. The finished answer is then one
 * message the user reads like any other. Each change is announced once: a repeated state, such as
 * StrictMode running the effect twice, says nothing, and an ending is announced only after an
 * answer, so mounting on an earlier error stays quiet.
 */
export const useAnswerAnnouncements = (aiState: AIState) => {
  const announce = useAriaLiveAnnouncer();
  // Starts idle, so opening the channel mid-answer announces it too.
  const previousState = useRef<AIState>(AIStates.Idle);

  useEffect(() => {
    if (previousState.current === aiState) return;
    const wasAnswering = isAnswering(previousState.current);
    previousState.current = aiState;

    if (isAnswering(aiState)) {
      if (!wasAnswering) announce("The assistant is answering");
    } else if (wasAnswering && endings[aiState]) {
      announce(endings[aiState]);
    }
  }, [aiState, announce]);
};

Replace the send button with Stop

While channel.state.aiState is AI_STATE_THINKING or AI_STATE_GENERATING, the default composer renders its StopAIGenerationButton slot in place of the send button, and passes it an onClick that calls Channel.stopAIResponse(). That sends ai_indicator.stop to the channel, which sets the state to AI_STATE_STOP, and StreamedMessageText shows the rest of the text it has at once.

The slot also takes the place of the record button and the cooldown timer: Message composer UI → Actions lists the order.

The default button draws IconStopFill and is named "Stop AI Generation". StopButton replaces it to name the button "Stop the answer" and to move the focus: the button goes away when the state changes, so StopButton focuses the composer's text field, from useMessageComposerContext().textareaRef. It fills the slot with the SDK's Button, the size of the send button, with a square icon.

src/StopButton.tsx
import type { ComponentProps, MouseEvent } from "react";
import { Button, useMessageComposerContext } from "stream-chat-react";
import "./StopButton.css";

/**
 * Fills the composer's `StopAIGenerationButton` slot, which shows in place of the send button
 * while the assistant answers. It gets a button's props, and its `onClick` calls
 * `Channel.stopAIResponse()`. The button goes away once the answer stops, so the focus moves to
 * the text field, where the user writes the next question.
 */
export const StopButton = ({ onClick }: ComponentProps<"button">) => {
  const { textareaRef } = useMessageComposerContext();

  const stop = (event: MouseEvent<HTMLButtonElement>) => {
    onClick?.(event);
    textareaRef.current?.focus();
  };

  return (
    <Button
      appearance="solid"
      aria-label="Stop the answer"
      circular
      className="cookbook-stop-button"
      onClick={stop}
      size="sm"
      type="button"
      variant="primary"
    >
      <span aria-hidden="true" className="cookbook-stop-button__icon" />
    </Button>
  );
};

The composer shows the slot only for thinking and generating. While the state is AI_STATE_EXTERNAL_SOURCES, it shows the send button, and the status line reads "Checking sources…" without a Stop button.

Add it to the base app

WithStreamedAnswers puts both components in their slots with WithComponents:

src/WithStreamedAnswers.tsx
import type { PropsWithChildren } from "react";
import { WithComponents } from "stream-chat-react";
import { AssistantStatus } from "./AssistantStatus";
import { StopButton } from "./StopButton";

// Declared once, so WithComponents gets the same object on every render.
const overrides = {
  StopAIGenerationButton: StopButton,
  TypingIndicator: AssistantStatus,
};

/**
 * The assistant's state at the end of the message list, and a Stop button in place of the send
 * button while it answers. `StreamedMessageText` needs no override: `MessageUI` renders it for
 * every message `Chat`'s `isMessageAIGenerated` returns `true` for.
 */
export const WithStreamedAnswers = ({ children }: PropsWithChildren) => (
  <WithComponents overrides={overrides}>{children}</WithComponents>
);

Wrap ChatView in it, and pass isMessageAIGenerated to Chat. The rest of the base app stays as it is:

src/App.tsx
// The base app's App, with these changes.
import { isMessageAIGenerated } from "./isMessageAIGenerated";
import { WithStreamedAnswers } from "./WithStreamedAnswers";

// In App:
return (
  <Chat client={client} isMessageAIGenerated={isMessageAIGenerated}>
    <SidePanelProvider>
      <WithStreamedAnswers>
        <ChatView layouts={layouts} views={{ channels: <ChannelsView /> }} />
      </WithStreamedAnswers>
    </SidePanelProvider>
  </Chat>
);

The finished answer

The assistant's finished answer: four vegetables in a list and a tip on watering, no status line under it, and the send button back in the composer

How it works

Info:

Example

The user asks the assistant which vegetables are easy for a first season:

  1. The backend posts an empty answer, marked ai_generated, and the message list renders it with StreamedMessageText.
  2. The backend sends ai_indicator.update with AI_STATE_THINKING. The client sets channel.state.aiState, AssistantStatus shows "Thinking…", useAnswerAnnouncements() announces "The assistant is answering", and the composer shows StopButton in place of the send button. With the first words, the backend sends AI_STATE_GENERATING, and the status line reads "Writing…".
  3. Each message.updated grows the answer's text, and StreamedMessageText types out the new part.
  4. The backend stores the final text and sends ai_indicator.clear. channel.state.aiState goes back to AI_STATE_IDLE, the status line gives way to the typing indicator, the send button comes back, and the user hears "The assistant finished answering".
  5. Had the user pressed Stop in step 3, Channel.stopAIResponse() would have sent ai_indicator.stop. The state becomes AI_STATE_STOP, StreamedMessageText shows the text it has at once, the focus moves to the text field, and the user hears "Answer stopped". The backend stops writing, stores what it has, and sends ai_indicator.clear.
  • ai_indicator.* events are only delivered live. A user who opens the channel mid-answer sees the status line from the next event on, and the answer's text from the next message.updated.
  • AI_STATE_ERROR stays until the next answer starts, so "Couldn't finish the answer" stays under the last message until then.
  • AI_STATE_STOP lasts until the backend's ai_indicator.clear, and shows no status line.
  • The status line sits in the scrolling message list. While the user reads older messages, the list doesn't scroll down to it, as with the typing indicator.

Related