Generative UI

An agent can answer with UI rather than text: a weekend plan with a map of the places, a card for each day and a form to change the pace. AIConversation shows a reply's UI in place of its text, brings it in as the agent writes it, and sends what the user does in it back to the agent as their next message.

The AI components know no UI format of their own. You give a conversation a renderer for each format your agent writes, and the components of your app the agent can place, such as a map, a calendar or a card over your own data. The SDK ships renderers of A2UI and OpenUI, and you can write one for any other format. A component works whichever format the agent writes.

Installation

Install stream-chat-react 14.16.0 or later, and load the AI components' stylesheet next to the SDK's, as in Usage:

npm install stream-chat-react@^14.16.0 stream-chat

import "stream-chat-react/css/index.css";
import "stream-chat-react/css/ai-components.css";

Each renderer is built on its format's official packages, which are optional peers of stream-chat-react. Install the ones for the formats your agent writes. stream-chat-react takes the patch releases of the versions it's tested with, since their 0.x releases change their API between minor versions.

For A2UI:

npm install @a2ui/web_core@~0.15.0 @a2ui/react@~0.15.0

For OpenUI, and import OpenUI's styles once, which ai-components.css themes:

npm install @openuidev/react-lang@~1.0.2 @openuidev/react-ui@~0.18.1 @openuidev/react-headless@~0.18.1 zustand@^4.5.5

import "@openuidev/react-ui/components.css";

An app that installs neither still shows every reply: a reply in a format it has no renderer for shows its text.

How a reply carries its UI

The agent's backend writes a reply's UI in the message's ai_ui custom field. The reply's text stays readable on its own, for wherever the UI can't show, such as a push notification or an app without a renderer for the format:

{
  "text": "Here's a weekend in Ohrid, with a map and a plan for each day.",
  "ai_ui": {
    "v": 1,
    "format": "openui",
    "status": "streaming",
    "content": "root = Card([title, map])\ntitle = CardHeader(\"A weekend in Ohrid\")"
  }
}

A model usually writes the UI into its reply, for example in a fenced code block. Have your backend move it out of the text into ai_ui as the reply streams, so the text keeps only the words around it. Stream Agents does this for you. A message's custom data holds at most 50 KB, so leave the UI out when it doesn't fit beside the rest of it.

fromStreamMessage reads the field into the uiDocument of the AIMessage, for the agent's replies only. A field of a newer v than the SDK reads is left out, so the reply shows its text. To read the field yourself, use parseAIUIDocument(message.ai_ui), and type the fields on your messages:

import type {
  AIUIActionJSON,
  AIUIDocumentJSON,
} from "stream-chat-react/ai-components";

declare module "stream-chat" {
  interface CustomMessageData {
    ai_ui?: AIUIDocumentJSON;
    ai_ui_action?: AIUIActionJSON;
  }
}

Show it in a conversation

Make an AIUIHost with the renderers of the formats your agent writes and your components, and pass it to AIConversation as ui. Keep one host for as long as its conversation shows, for example with useMemo, so each reply's UI keeps its state, such as what the user typed into a form.

With Stream Agents, the host's send asks the agent with the user's words, from @stream-io/calypso, and carries the action as uiAction. The agents router keeps it on the user's message as ai_ui_action:

import { useMemo } from "react";
import {
  AIConversation,
  AIUIHost,
  aiUIActionJSON,
  fromStreamMessage,
} from "stream-chat-react/ai-components";
import { createA2UIRenderer } from "stream-chat-react/a2ui";
import { createOpenUIRenderer } from "stream-chat-react/openui";

export const Assistant = ({ session, messages }) => {
  const ui = useMemo(
    () =>
      new AIUIHost({
        components: [placeCard],
        renderers: [createA2UIRenderer(), createOpenUIRenderer()],
        send: (action) =>
          session.responses.create(action.label, {
            uiAction: aiUIActionJSON(action),
          }),
      }),
    [session],
  );

  return (
    <AIConversation
      messages={messages.map((message) => fromStreamMessage(message))}
      ui={ui}
    />
  );
};

With a backend of your own, send the action as a message on the channel, with the action in its ai_ui_action field:

import {
  AI_UI_ACTION_CUSTOM_DATA_KEY,
  aiUIActionJSON,
} from "stream-chat-react/ai-components";

const send = (action) =>
  channel.sendMessage({
    text: action.label,
    [AI_UI_ACTION_CUSTOM_DATA_KEY]: aiUIActionJSON(action),
  });

A reply shows its UI in place of its text when one of the host's renderers reads its format and version, the first one listed asked first. Otherwise, it shows its text. The UI comes in as the agent writes it: the agent's thinking gives way to it as soon as it starts, and each part fades in, rising into place, after the parts before it. A reply written before it showed, such as one loaded from history, shows at once.

Reply actions show under a reply once its UI is written, as they do under text.

A reply's UI never takes the conversation down with it: when it can't show, the reply shows its text. See Handle errors.

Renderers

A renderer reads one format, and makes the views of each reply's UI written in it. Each one is its own entry point, built on the format's official packages, which your app installs. They load in the browser the first time a reply shows the format, so an app whose agent never writes it never loads them, and server rendering shows a placeholder.

Both renderers place your app's components beside their own, run your app's tools, and follow the AI components' light and dark themes. Both take an onError(error, messageId?) option for the problems with a reply's UI, such as a component the format doesn't know. See Handle errors.

The renderers treat what the agent writes as untrusted: links open through the host, which opens only web and email links by default, and Markdown never renders raw HTML.

A2UI

createA2UIRenderer reads A2UI 1.0 with the basic catalog of the official @a2ui/web_core and @a2ui/react packages. The entry point is an ES module only.

import { createA2UIRenderer } from "stream-chat-react/a2ui";

const ui = new AIUIHost({
  components: [placeCard, tripMap],
  renderers: [createA2UIRenderer()],
  send,
});

A reply's content is A2UI messages, one JSON object per line, and each one shows as it completes, while the agent writes it. The agent writes with the basic catalog, which your components join under their A2UI type (place_card is PlaceCard), in the getstream.io:app-catalog catalog. A parameter takes the value its schema describes, or a binding (@path) or function (@call), which your component gets worked out. Each reply's surfaces are its own, kept for as long as the host is, so what the user entered survives scrolling.

The parts a user sees most, Button, Text, Card, TextField, ChoicePicker and Icon, draw with this SDK's components and follow its theme. The rest of the basic catalog, its layouts and media, draws with A2UI's own elements, themed with the AI components' tokens. A Modal is a native dialog, which keeps focus while open and closes on Escape or its backdrop. What the agent says about a part with its accessibility reaches assistive technology. Text renders with AIMarkdown.

A button reaches the host as an AIUIAction named after its event, labeled with its userMessage, else the button's text, with { surfaceId, sourceComponentId, context } as its payload.

OpenUI

createOpenUIRenderer reads OpenUI Lang 1.0 with the official OpenUI packages. Import OpenUI's styles once, and ai-components.css themes them:

import "@openuidev/react-ui/components.css";
import { createOpenUIRenderer } from "stream-chat-react/openui";

const ui = new AIUIHost({
  components: [placeCard, tripMap],
  renderers: [createOpenUIRenderer()],
  send,
});

A reply's content is an OpenUI Lang program, which shows as the agent writes it. A finished reply with no program, or one that draws nothing, such as words that aren't OpenUI, shows its text. The agent writes with OpenUI's chat library by default, where every reply is one card. Your components join it under their OpenUI name (place_card is PlaceCard), wherever the library's own content goes, such as a Card's children. What the user chose in a reply's forms is kept by the host, so it survives scrolling. A follow-up or a form's submit button reaches the host as an AIUIAction named continue_conversation, labeled with the control's words, with { context, params, form, values } as its payload. values are the values of its form, without a field OpenUI marks secret.

@openuidev/lang-core, which these packages install, reports the install to OpenUI's analytics anonymously, unless OPENUI_TELEMETRY_DISABLED=1 or DO_NOT_TRACK=1 is set. It sends nothing from a browser, and nothing from a server unless OPENUI_RUNTIME_TELEMETRY_ENABLED=1 is set.

Write a renderer

For any other format, write a renderer of your own. A renderer makes an engine for each conversation, which makes the view of each reply's UI:

import {
  AIUIEntrance,
  type AIUIRenderer,
} from "stream-chat-react/ai-components";

const cardsRenderer: AIUIRenderer = {
  format: "cards",
  createEngine: () => ({
    view: (document) => {
      const cards = JSON.parse(document.content);
      if (!Array.isArray(cards)) return null;
      return (
        <div>
          {cards.map((card, index) => (
            <AIUIEntrance key={index}>
              <p>{card.title}</p>
            </AIUIEntrance>
          ))}
        </div>
      );
    },
  }),
};

List it in the host like any other renderer.

• view(document, messageId) is asked again each time the conversation renders. Return null to show the reply's text instead. Keep what it makes for a reply and return it again.
• The host keeps its engines for as long as it's kept, so an engine can hold each reply's state by its messageId, or with host.state(messageId) and host.setState(messageId, state).
• receive(document, messageId) gets each reply's document as it arrives, oldest first, including replies that aren't on screen. Use it for a format whose later replies change an earlier one's UI. remove(messageId) tells the engine a reply, or its UI, left the conversation.
• When what a reply shows changes outside a render, such as once a package loads, call host.refresh(), and the conversation shows its replies again. AIUIRuntimeLoader and AIUIRuntimeView load a format's packages the first time a reply shows it, as the SDK's renderers do.
• supports(version) says which versions of the format it reads. It reads every version by default.
• capabilities(components) says what the agent's backend needs to write the format, such as the instructions for its model. It may return a promise. See Tell the agent what the app shows.

To place the app's components, find one with host.component(name) and show it with AIUIComponentView, giving it the arguments the agent wrote and an emit that reports its events the way your format does.

Components

A component is a view of your app the agent can place in a reply's UI. You describe it once, with the parameters the agent fills in and the events it reports, and say how it looks for the arguments the agent wrote:

import {
  AIUIParameterType,
  defineAIUIComponent,
  optionalAIUIParameter,
  requiredAIUIParameter,
} from "stream-chat-react/ai-components";

type PlaceCardArgs = { title: string; visitDate?: Date };

export const placeCard = defineAIUIComponent<PlaceCardArgs>(
  {
    name: "place_card",
    description: "A photo and a short summary of a place.",
    parameters: [
      requiredAIUIParameter(
        "title",
        AIUIParameterType.string,
        "The place's Wikipedia title.",
      ),
      optionalAIUIParameter("visit_date", AIUIParameterType.date),
    ],
    events: [
      {
        name: "ask_about",
        description: "The user asks about the place.",
        parameters: [],
      },
    ],
  },
  ({ args, context }) => (
    <PlaceCard
      title={args.title}
      date={args.visitDate}
      onClick={() =>
        context.emit("ask_about", `Tell me more about ${args.title}`, {
          title: args.title,
        })
      }
    />
  ),
);

context.emit(event, label, payload?) reports one of its events: the label is what the user did, in their words, which becomes their message. Every renderer places it by its name, so it works whichever format the agent writes. A component named like one of the format's own is left out and logged. The arguments arrive with camelCase keys, and dates as Dates, given as a day, such as 2026-11-14, or a moment in RFC 3339. While the agent is still writing them and they don't fit the parameters yet, the component waits to show. context.isStreaming tells whether they may still change, and context.messageId which reply it's in.

The parameter types are string, number, integer, boolean, date, dateTime, url, enumeration(values), array(items) and object(fields) on AIUIParameterType. A parameter's type can be any other JSON Schema.

A component can load what it shows itself, such as photos, which then never reaches the agent.

One description for every platform

A component's definition is JSON, so your web, iOS and Android apps, and the agent's backend, can share one description of it. Its parameters are a JSON Schema object, as a tool's are:

{
  "name": "place_card",
  "description": "A photo and a short summary of a place.",
  "parameters": {
    "type": "object",
    "properties": {
      "title": {
        "type": "string",
        "description": "The place's Wikipedia title."
      },
      "visit_date": { "type": "string", "format": "date" }
    },
    "required": ["title"],
    "x-order": ["title", "visit_date"]
  },
  "events": [
    { "name": "ask_about", "description": "The user asks about the place." }
  ]
}

Read it with parseAIUIComponentDefinition to use the same file in your app, and write one back with aiUIComponentDefinitionJSON:

import definitions from "./UIComponents.json";

const placeCard = defineAIUIComponent<PlaceCardArgs>(
  parseAIUIComponentDefinition(definitions[0])!,
  PlaceCardView,
);

Run your app's tools

A reply's UI can run your app's tools in the browser, such as to load the user's saved places for the days of a plan. Pass them to the host as tools. They are apart from the tools the agent calls, and they run whenever the UI asks, without asking the user, so give the host only tools that are safe to run that way:

const ui = new AIUIHost({
  renderers,
  send,
  tools: [
    {
      name: "saved_places",
      run: async ({ city }) => loadSavedPlaces(String(city)),
    },
  ],
});

An OpenUI program's Query() and Mutation() run them, and so does an A2UI function the catalog doesn't define. A component of yours runs one through its context's host, and gets the tool's output:

const places = await context.host.runTool("saved_places", { city: "Ohrid" });

The host's open option opens the links of a reply's UI. By default, web and email links open, in a new tab, and others don't, since the agent writes the links, and another kind could call someone or act in another app.

What the user does

When the user does something in a reply's UI that's for the agent, such as tapping a place, choosing a follow-up or submitting a form, the host calls your send with an AIUIAction. Send it as the user's next message. The host sends a reply's actions one after another, in the order they happened.

aiUIActionJSON(action) is the action as the message's ai_ui_action field, or as Stream Agents' uiAction. Have your backend tell the model what the user did after their words, and which reply it was in, since the same UI can show in several replies.

An action the agents router would refuse, one without a reply, format or name, or over 8 KB as JSON, isn't sent. It goes to the host's onError(error, action), as does one that fails to send, and neither holds up the actions after it. See Handle errors.

Tell the agent what the app shows

The model has to know which formats and components your app shows. The host's capabilities() describes them: each format, with what its renderer adds, and your components' definitions. It resolves once the renderers have written their instructions, which loads their packages:

const capabilities = await ui.capabilities();
// { v: 1, renderers: [{ format: "a2ui", fence, catalog_id, instructions }, ...], components: [...] }

Each renderer adds the fence the model writes its UI in and the instructions for the model, written from the format's components and yours. A2UI also adds the catalog_id of the catalog with your components.

Send the capabilities to your backend, for example when a conversation starts, and add the instructions to what the model is told. You can also write the instructions on your backend or at build time, from the same component definitions:

• a2uiInstructions(config), from stream-chat-react/a2ui, writes A2UI's instructions with your components, examples and rules. It rejects with an A2UIInstructionsError when an example wouldn't render, since a broken example teaches the model broken UI. a2uiProblems(text, config) lists what's wrong with A2UI messages for the catalog, for checking what a model writes.
• openUIInstructions(components), from stream-chat-react/openui, writes OpenUI's instructions with your components, with OpenUI's own prompt generator.

import { a2uiInstructions } from "stream-chat-react/a2ui";
import { openUIInstructions } from "stream-chat-react/openui";

const a2ui = await a2uiInstructions({ components: [placeCard, tripMap] });
const openui = await openUIInstructions([placeCard, tripMap]);

Both load their format's packages, so ask for them where that's fine, such as once at build time or when a conversation starts.

Handle errors

A reply's UI never takes the conversation down with it. The host and the renderers each take a callback for what goes wrong, and log it by default:

const ui = new AIUIHost({
  components: [placeCard],
  renderers: [
    createA2UIRenderer({
      onError: (error, messageId) => report("a2ui", error, messageId),
    }),
    createOpenUIRenderer({
      onError: (error, messageId) => report("openui", error, messageId),
    }),
  ],
  send,
  onError: (error, action) => setUnsent(`"${action.label}" wasn't sent.`),
  onRenderError: (error, messageId) => report("render", error, messageId),
});

A reply shows its text in place of its UI when:

• No renderer reads its format or version, or its ai_ui field is of a newer v than the SDK reads.
• The renderer's packages can't load, such as after a deploy replaced the file they are in. The next reply the agent writes in that format tries again.
• A finished reply draws nothing, such as an A2UI reply that made no surface or a program with no UI in it.
• Its UI throws while it renders, outside your components. A reply the agent is still writing tries again once it's written.

One of your components that throws, such as when given what it didn't expect, leaves only itself out of the reply's UI, and is tried again when what it's given changes.

A chat has no channel for these problems, so forward them to your backend if the agent should hear about them. To tell the user that what they did wasn't sent, throw from send, as when the agent is still answering, and show the error from onError where the composer shows a failed question.

Theming

A reply's UI follows the AI components' theme, and with it the stream-chat-react light and dark themes: inside .str-chat, the AI components read the SDK's theme tokens. To change its colors, set the --str-chat__ai-* variables described in Theming. The ones a reply's UI reads most are:

:root {
  --str-chat__ai-accent: #6d28d9;
  --str-chat__ai-button-primary-bg: #6d28d9;
}

Each format's own variables are set from these on the element its renderer draws in: --a2ui-* on .str-chat__a2ui, and --openui-* on .str-chat__openui. To change one format only, set its variables there:

.str-chat__openui {
  --openui-font-body: "Inter", sans-serif;
}

The rest of OpenUI's look, such as its chart palette, stays its own. A2UI's Icon draws the SDK's icons, which you can replace like any SDK icon with the icons of WithComponents, and the renderer's icons and iconFont options draw the names the SDK doesn't.

Show it in your own views

To show replies in views of your own rather than with AIConversation, show a reply's UI with AIUIView, and its text when the host shows nothing for it:

import {
  AIUIView,
  StreamingMessage,
  useAIUIHostRefresh,
} from "stream-chat-react/ai-components";

const Reply = ({ message, ui }) => {
  useAIUIHostRefresh(ui);
  const document = message.uiDocument;
  return document && ui.content(document, message.id) !== null ? (
    <AIUIView document={document} host={ui} messageId={message.id} />
  ) : (
    <StreamingMessage text={message.text} />
  );
};

Each time your messages change, pass them to ui.receive(messages) before the replies render, so the renderers get the UI of every reply, also of those that aren't on screen. useAIUIHostRefresh renders again when the host's answer changes, such as once a renderer's packages load.

Motion

AIUIEntrance brings a part of a reply's UI in when it shows: the part fades in, rising into place and coming into focus, after the parts that came in just before it, 0.1 seconds apart and at most 0.4 seconds later. Parts that show together come in together, so a reply written before it showed shows at once. With reduced motion, parts fade in without moving.

The SDK's renderers give it to each part they place, and every component has it. Use it on the parts of your own renderer's views, as the cards renderer does above.