Generative UI

An agent can answer with UI rather than text: a weekend plan with a map of the places, a card for each day and a form to change the pace. AIConversation shows a reply's UI in place of its text, brings it in as the agent writes it, and sends what the user does in it back to the agent as their next message.

The AI components know no UI format of their own. You give a conversation a renderer for each format your agent writes, and the components of your app the agent can place, such as a map, a calendar or a card over your own data. The SDK ships a renderer of A2UI, and you can write one for any other format. OpenUI, which the React and iOS SDKs also read, has no renderer on React Native. A component works whichever format the agent writes.

Installation

Generative UI needs nothing beyond the AI components. The A2UI renderer is built on the official @a2ui/web_core package, which is installed with the SDK. It has an entry point of its own, so an app that never imports it doesn't bundle the A2UI core.

// React Native CLI
import { createA2UIRenderer } from "stream-chat-react-native/a2ui";
import { AIUIHost } from "stream-chat-react-native/ai-components";

// Expo
import { createA2UIRenderer } from "stream-chat-expo/a2ui";
import { AIUIHost } from "stream-chat-expo/ai-components";

An app without a renderer still shows every reply: a reply in a format it has no renderer for shows its text.

How a reply carries its UI

The agent's backend writes a reply's UI in the message's ai_ui custom field. The reply's text stays readable on its own, for wherever the UI can't show, such as a push notification or an app without a renderer for the format:

{
  "text": "Here's a weekend in Ohrid, with a map and a plan for each day.",
  "ai_ui": {
    "v": 1,
    "format": "a2ui",
    "version": "1.0",
    "status": "streaming",
    "content": "{\"version\":\"1.0\",\"createSurface\":{\"surfaceId\":\"trip\",\"catalogId\":\"getstream.io:app-catalog\"}}\n{\"version\":\"1.0\",\"updateComponents\":{\"surfaceId\":\"trip\",\"components\":[...]}}"
  }
}

A model usually writes the UI into its reply, for example in a fenced code block. Have your backend move it out of the text into ai_ui as the reply streams, so the text keeps only the words around it. Stream Agents does this for you. A message's custom data holds at most 50 KB, so leave the UI out when it doesn't fit beside the rest of it.

fromStreamMessage reads the field into the uiDocument of the AIMessage, for the agent's replies only. A field of a newer v than the SDK reads is left out, so the reply shows its text. To read the field yourself, use parseAIUIDocument(message.ai_ui), and type the fields on your messages:

import type { DefaultMessageData } from "stream-chat-react-native";
import type {
  AIUIActionJSON,
  AIUIDocumentJSON,
} from "stream-chat-react-native/ai-components";

declare module "stream-chat" {
  interface CustomMessageData extends DefaultMessageData {
    ai_ui?: AIUIDocumentJSON;
    ai_ui_action?: AIUIActionJSON;
  }
}

Show it in a conversation

Make an AIUIHost with the renderers of the formats your agent writes and your components, and pass it to AIConversation as ui. Keep one host for as long as its conversation shows, for example with useState, so each reply's UI keeps its state, such as what the user typed into a form, while it scrolls out of sight and back.

With Stream Agents, the host's send asks the agent with the user's words, from @stream-io/stream-agents, and carries the action as uiAction. The agents router keeps it on the user's message as ai_ui_action:

import { useMemo, useState } from "react";
import { createA2UIRenderer } from "stream-chat-react-native/a2ui";
import { AIConversation } from "stream-chat-react-native/ai-components/conversation";
import {
  AIUIHost,
  aiUIActionJSON,
  createFromStreamMessage,
} from "stream-chat-react-native/ai-components";

export const Assistant = ({ session, messages }) => {
  const [ui] = useState(
    () =>
      new AIUIHost({
        components: [placeCard],
        renderers: [createA2UIRenderer()],
        send: (action) =>
          session.responses.create(action.label, {
            uiAction: aiUIActionJSON(action),
          }),
      }),
  );
  const [fromStreamMessage] = useState(() => createFromStreamMessage());
  const conversation = useMemo(
    () => messages.map(fromStreamMessage),
    [fromStreamMessage, messages],
  );

  return <AIConversation messages={conversation} ui={ui} />;
};

With a backend of your own, send the action as a message on the channel, with the action in its ai_ui_action field:

import {
  AI_UI_ACTION_CUSTOM_DATA_KEY,
  aiUIActionJSON,
  type AIUIAction,
} from "stream-chat-react-native/ai-components";

const send = (action: AIUIAction) =>
  channel.sendMessage({
    text: action.label,
    [AI_UI_ACTION_CUSTOM_DATA_KEY]: aiUIActionJSON(action),
  });

With a conversation controller, send the action through it, so it shows as the user's message straight away and reaches your transport like one. Its custom data arrives in the transport's send as request.custom:

const send = (action: AIUIAction) =>
  conversation.send({
    text: action.label,
    custom: { uiAction: aiUIActionJSON(action) },
  });

A reply shows its UI in place of its text, after its steps, when one of the host's renderers reads its format and version, the first one listed asked first. Otherwise, it shows its text. The UI comes in as the agent writes it: the agent's thinking gives way to it as soon as it starts, and each part fades in, rising into place, after the parts before it. A reply written before it showed, such as one loaded from history, shows at once.

Reply actions show under a reply once its UI is written, as they do under text.

A reply's UI never takes the conversation down with it: when it can't show, the reply shows its text. See Handle errors.

The list reuses its rows for the UI too, so one of your components that keeps state of its own should start afresh for a new context.messageId.

The A2UI renderer

createA2UIRenderer reads A2UI 1.0 and draws all 18 components of its basic catalog natively, with React Native views.

import { createA2UIRenderer } from "stream-chat-react-native/a2ui";

const ui = new AIUIHost({
  components: [placeCard, tripMap],
  renderers: [createA2UIRenderer()],
  send,
});

A reply's content is A2UI messages, one JSON object per line, and each one shows as it completes, while the agent writes it. Until the reply's first surface arrives, its UI shows a placeholder, and a finished reply that made no surface shows its text instead. The agent writes with the basic catalog, which your components join under their A2UI type (place_card is PlaceCard), in the getstream.io:app-catalog catalog. A parameter takes the value its schema describes, or a binding (@path) or function (@call), which your component gets worked out. Each reply's surfaces are its own, kept for as long as the host is, so what the user entered survives scrolling.

A button reaches the host as an AIUIAction named after its event and labeled with the button's text, so the user's message says what they tapped. Its payload is { surfaceId, sourceComponentId, context }, with the action's userMessage, what the action asks of the agent, when the agent wrote one. What the user typed into an obscured TextField, such as a password, never leaves the device: it's cut out of what an action sends and of what a tool gets.

A view of your own for a basic component gets node and props, what the agent wrote with its bindings and functions worked out. A property the user can change, such as a TextField's value, comes with a setter named after it (setValue). Draw a child with A2UIChild, and a list of them with A2UIChildren.

Image, Video and AudioPlayer load only public web links by default, never addresses on the user's network, such as 192.168.1.1 or localhost, since the agent writes the links. Video and AudioPlayer play in place with the player you set up for the Chat SDK's video and voice messages. Without one, they show a link that opens where the device plays it.

Write a renderer

For any other format, write a renderer of your own. A renderer makes an engine for each conversation, which makes the view of each reply's UI:

import { Text, View } from "react-native";
import {
  AIUIEntrance,
  type AIUIRenderer,
} from "stream-chat-react-native/ai-components";

const cardsRenderer: AIUIRenderer = {
  format: "cards",
  createEngine: () => ({
    view: (document) => {
      const cards =
        typeof document.content === "string"
          ? JSON.parse(document.content)
          : document.content;
      if (!Array.isArray(cards)) return null;
      return (
        <View>
          {cards.map((card, index) => (
            <AIUIEntrance key={index}>
              <Text>{card.title}</Text>
            </AIUIEntrance>
          ))}
        </View>
      );
    },
  }),
};

List it in the host like any other renderer.

• view(document, messageId) is asked again each time the conversation renders. Return null to show the reply's text instead. Keep what it makes for a reply and return it again.
• The host keeps its engines for as long as it's kept, so an engine can hold each reply's state by its messageId, or with host.state(messageId) and host.setState(messageId, state).
• receive(document, messageId) gets each reply's document as it arrives, oldest first, including replies that aren't on screen. Use it for a format whose later replies change an earlier one's UI. remove(messageId) tells the engine a reply, or its UI, left the conversation.
• When what a reply shows changes outside a render, call host.refresh(), and the conversation shows its replies again.
• supports(version) says which versions of the format it reads. It reads every version by default.
• capabilities(components, tools) says what the agent's backend needs to write the format, such as the instructions for its model. It may return a promise. See Tell the agent what the app shows.

To place the app's components, find one with host.component(name) and show it with AIUIComponentView, giving it the arguments the agent wrote and a context whose emit reports its events the way your format does.

Components

A component is a view of your app the agent can place in a reply's UI. You describe it once, with the parameters the agent fills in and the events it reports, and say how it looks for the arguments the agent wrote:

import { Pressable, Text } from "react-native";
import {
  AIUIParameterType,
  defineAIUIComponent,
  optionalAIUIParameter,
  requiredAIUIParameter,
} from "stream-chat-react-native/ai-components";

type PlaceCardArgs = { title: string; visitDate?: Date };

export const placeCard = defineAIUIComponent<PlaceCardArgs>(
  {
    name: "place_card",
    description: "A photo and a short summary of a place.",
    parameters: [
      requiredAIUIParameter(
        "title",
        AIUIParameterType.string,
        "The place's Wikipedia title.",
      ),
      optionalAIUIParameter("visit_date", AIUIParameterType.date),
    ],
    events: [
      {
        name: "ask_about",
        description: "The user asks about the place.",
        parameters: [],
      },
    ],
  },
  ({ args, context }) => (
    <Pressable
      onPress={() =>
        context.emit("ask_about", `Tell me more about ${args.title}`, {
          title: args.title,
        })
      }
    >
      <Text>{args.title}</Text>
    </Pressable>
  ),
);

context.emit(event, label, payload?) reports one of its events: the label is what the user did, in their words, which becomes their message. Every renderer places it by its name, so it works whichever format the agent writes. A component named like one of the format's own is left out and reported. The arguments arrive with camelCase keys, and dates as Dates, given as a day, such as 2026-11-14, or a moment in RFC 3339. While the agent is still writing them and they don't fit the parameters yet, the component waits to show. context.isStreaming tells whether they may still change, and context.messageId which reply it's in.

The parameter types are string, number, integer, boolean, date, dateTime, url, enumeration(values), array(items) and object(fields) on AIUIParameterType. A parameter's type can be any other JSON Schema.

A component can load what it shows itself, such as photos, which then never reaches the agent.

One description for every platform

A component's definition is JSON, so your React Native, iOS, Android and web apps, and the agent's backend, can share one description of it. Its parameters are a JSON Schema object, as a tool's are:

{
  "name": "place_card",
  "description": "A photo and a short summary of a place.",
  "parameters": {
    "type": "object",
    "properties": {
      "title": {
        "type": "string",
        "description": "The place's Wikipedia title."
      },
      "visit_date": { "type": "string", "format": "date" }
    },
    "required": ["title"],
    "x-order": ["title", "visit_date"]
  },
  "events": [
    { "name": "ask_about", "description": "The user asks about the place." }
  ]
}

Read it with parseAIUIComponentDefinition to use the same file in your app, and write one back with aiUIComponentDefinitionJSON:

import definitions from "./UIComponents.json";

const placeCard = defineAIUIComponent<PlaceCardArgs>(
  parseAIUIComponentDefinition(definitions[0])!,
  PlaceCardView,
);

Run your app's tools

A reply's UI can run your app's tools on the device, such as to load the user's saved places for the days of a plan. Pass them to the host as tools. They are the same AIClientTools an AIClientToolRunner takes, so a tool is defined once:

const savedPlaces: AIClientTool = {
  definition: {
    name: "saved_places",
    description: "The user's saved places in a city.",
    inputSchema: {
      type: "object",
      properties: { city: { type: "string" } },
      required: ["city"],
    },
    annotations: { readOnlyHint: true },
  },
  run: async (call) => {
    const { city } = call.arguments as { city: string };
    return completedToolResult(await loadSavedPlaces(city));
  },
};

const ui = new AIUIHost({ renderers, send, tools: [savedPlaces] });

A tool runs by itself, such as to load data while a reply shows, only when its definition says it's read-only (annotations: { readOnlyHint: true }). Any other tool runs only from something the user did, such as a tap. Output that isn't JSON is refused.

An A2UI reply calls a tool as a function of the catalog, named after the tool, and gives up on one that hasn't answered within 30 seconds. The A2UI instructions list your tools, mark the read-only ones, and tell the model that the others run only from a button the user taps. A component of yours runs one through its context's host, and gets the tool's output:

const places = await context.host.runTool(
  "saved_places",
  { city: "Ohrid" },
  { userInitiated: true },
);

Pass userInitiated: true only when the user asked for it, such as from a tap. runTool rejects with an AIUIToolError when the tool can't run, fails or gives output that isn't JSON.

The host's open option opens the links of a reply's UI. By default, web and email links open, and others don't, since the agent writes the links, and another kind could call someone or act in another app.

What the user does

When the user does something in a reply's UI that's for the agent, such as tapping a place or submitting a form, the host calls your send with an AIUIAction. Send it as the user's next message. The host sends a reply's actions one after another, in the order they happened.

aiUIActionJSON(action) is the action as the message's ai_ui_action field, or as Stream Agents' uiAction. Have your backend tell the model what the user did after their words, and which reply it was in, since the same UI can show in several replies.

An action the agents router would refuse, one without a reply, format or name, or over 8 KB as JSON (AI_UI_ACTION_MAX_BYTES), isn't sent. It goes to the host's onError(error, action), as does one that fails to send, and neither holds up the actions after it. aiUIActionProblem(action) says why one would be refused. See Handle errors.

Tell the agent what the app shows

The model has to know which formats and components your app shows. The host's capabilities() describes them: each format, with what its renderer adds, and your components' definitions and tools:

const capabilities = await ui.capabilities();
// { v: 1, renderers: [{ format: "a2ui", fence, catalog_id, instructions }], components: [...], tools: [...] }

tools are your tools as Model Context Protocol tools: each one's name, description, inputSchema ({ type: "object" } when it takes nothing) and annotations.readOnlyHint. Without tools, capabilities() leaves tools out.

The A2UI renderer adds the fence the model writes its UI in (ui:a2ui@1.0), the catalog_id of the catalog with your components, and the instructions for the model, written from the basic catalog, your components and your tools.

Send the capabilities to your backend, for example when a conversation starts, and add the instructions to what the model is told. You can also write the instructions on your backend or at build time, from the same component definitions, with the functions of the a2ui entry point:

• a2uiInstructions(config) writes A2UI's instructions with your components, examples, rules, notes, tools and the components to leave out (removing). It rejects with an A2UIInstructionsError when an example wouldn't render, since a broken example teaches the model broken UI.
• a2uiProblems(text, config) lists what's wrong with A2UI messages for the catalog, for checking what a model writes.

import { a2uiInstructions } from "stream-chat-react-native/a2ui";

const instructions = await a2uiInstructions({
  components: [placeCard, tripMap],
});

Both take your components or their JSON definitions, such as the entries of a UIComponents.json.

Handle errors

A reply's UI never takes the conversation down with it. The host and the renderer each take a callback for what goes wrong, and log it by default:

const ui = new AIUIHost({
  components: [placeCard],
  renderers: [
    createA2UIRenderer({
      onError: (error, messageId) => report("a2ui", error, messageId),
    }),
  ],
  send,
  onError: (error, action) => setUnsent(`"${action.label}" wasn't sent.`),
  onRenderError: (error, messageId) => report("render", error, messageId),
});

A reply shows its text in place of its UI when:

• No renderer reads its format or version, or its ai_ui field is of a newer v than the SDK reads.
• A finished reply draws nothing, such as an A2UI reply that made no surface.
• Its UI throws while it renders, outside your components. A reply the agent is still writing tries again once it's written. ui.hasFailed(messageId) says whether a reply's UI failed.

One of your components that throws, such as when given what it didn't expect, leaves only itself out of the reply's UI, and is tried again when what it's given changes.

A chat has no channel for these problems, so forward them to your backend if the agent should hear about them. To tell the user that what they did wasn't sent, throw from send, as when the agent is still answering, and show the error from onError where your app shows a failed question.

Theming

A reply's UI follows the AI components' theme, which StreamTheme provides. The theme's generatedUI.container styles what a reply's UI sits in, and its a2ui key styles A2UI's basic components: colors, over ones from the theme's own colors, and one style per part.

The styles of the parts are surfaces, card, button, buttonText, input, label, chip, chipText, tab, tabText, divider, caption and image:

<StreamTheme
  style={{
    a2ui: {
      colors: { accent: "#6d28d9", onAccent: "#ffffff" },
      card: { borderRadius: 20 },
    },
  }}
>
  {children}
</StreamTheme>

Show it in your own views

To show replies in views of your own rather than with AIConversation, show a reply's UI with AIUIView, and its text when the host shows nothing for it:

import {
  AIUIView,
  StreamingMessageView,
  useAIUIHostRefresh,
} from "stream-chat-react-native/ai-components";

const Reply = ({ message, ui }) => {
  useAIUIHostRefresh(ui);
  const document = message.uiDocument;
  return document && ui.content(document, message.id) !== null ? (
    <AIUIView document={document} host={ui} messageId={message.id} />
  ) : (
    <StreamingMessageView text={message.text} />
  );
};

Each time your messages change, pass them to ui.receive(messages) before the replies render, so the renderers get the UI of every reply, also of those that aren't on screen. useAIUIHostRefresh renders again when the host's answer changes, such as once a finished reply turns out to draw nothing.

Motion

AIUIEntrance brings a part of a reply's UI in when it shows: the part fades in, rising into place and, on Android 12 and later, coming into focus, after the parts that came in just before it. Parts that show together come in together, so a reply written before it showed shows at once. With reduced motion, parts only fade in.

The A2UI renderer gives it to each part it places, and every component has it. Use it on the parts of your own renderer's views, as the cards renderer does above.