Generative UI

An agent can answer with UI rather than text: a weekend plan with a map of the places, the user's own calendar for those days and a form to change the pace. StreamChatAI shows a reply's UI in place of its text, brings it in as the agent writes it, and sends what the user does in it back to the agent as their next message.

StreamChatAI knows no UI format of its own. You give a conversation a renderer for each format your agent writes, and the views of your app the agent can place, such as a map, a calendar or a card over your own data. The SDK ships renderers of A2UI and OpenUI, and you can write one for any other format. A component works whichever format the agent writes.

How a reply carries its UI

The agent's backend writes a reply's UI in the message's ai_ui custom field. The reply's text stays readable on its own, for wherever the UI can't show, such as a push notification or an app without a renderer for the format:

{
  "text": "Here's a weekend in Ohrid, with a map and a plan for each day.",
  "ai_ui": {
    "v": 1,
    "format": "openui",
    "status": "streaming",
    "content": "root = Card([title, map])\ntitle = CardHeader(\"A weekend in Ohrid\")"
  }
}

A model usually writes the UI into its reply, for example in a fenced code block. Have your backend move it out of the text into ai_ui as the reply streams, so the text keeps only the words around it. A message's custom data holds at most 50 KB, so leave the UI out when it doesn't fit beside the rest of it.

Read the field on your messages with AIUIDocument(customData:), as the uiDocument of your AIMessage:

struct ChatAIMessage: AIMessage {
    let message: ChatMessage

    // The id, role, text and the rest, as in the Conversation View guide.

    var uiDocument: AIUIDocument? { AIUIDocument(customData: message.extraData) }
}

Show it in a conversation

Make an AIUIHost with the renderers of the formats your agent writes and your components, and pass it to AIConversationView as ui. Keep one host for as long as its conversation shows, for example in the model behind your screen, so each reply's UI keeps its state, such as what the user typed into a form:

import StreamChatA2UI
import StreamChatAI
import StreamChatOpenUI

let ui = AIUIHost(components: [PlaceCardComponent()]) {
    A2UIRenderer()
    OpenUIRenderer()
} send: { action in
    try await chat.sendMessage(with: action.label, extraData: action.customData)
}

AIConversationView(messages: messages, ui: ui)

Here chat is the Stream Chat Chat of the conversation's channel. A reply shows its UI in place of its text when one of the host's renderers reads its format and version, the first one listed asked first. Otherwise, it shows its text. The UI comes in as the agent writes it: each part fades in, rising into place, after the parts before it. A reply written before it showed, such as one loaded from history, shows at once.

Reply actions show under a reply once its UI is written, as they do under text. Their text is the reply's words, which a reply that only shows UI doesn't have, so copying and sharing are left out for it.

With an AIConversation, pass the host the same way:

AIConversationView(conversation: conversation, ui: ui)

OpenUIRenderer requires iOS 17. If your app supports earlier versions, make a host with it only on iOS 17 and later, and one with the other renderers before that.

Renderers

A renderer reads one format, and makes the views of each reply's UI written in it. Add the library of each format your agent writes to your app target, next to StreamChatAI:

Both renderers place your app's components beside their own, run your app's tools, and style their views with the conversation's AIAppearance, so a reply's UI looks like the rest of the conversation. They take the same options:

The renderers treat what the agent writes as untrusted: a reply's UI can't crash or freeze the app, its links open through the host, which opens only web and email links by default, and images download at most once, capped at 20 MB.

A2UI

A2UIRenderer reads A2UI 1.0, with its basic catalog of components and functions drawn in SwiftUI. The agent writes with the basic catalog, which your components join under the getstream.io:app-catalog id (catalogId changes it). A button reaches the host as an AIUIAction named after its event, labeled with the button's text, with the surface's data model when the agent asks for it with sendDataModel. A function the catalog doesn't define runs as one of the host's tools.

By default, only http and https media load (A2UIRenderer.webURL(_:)). To change how one of the basic catalog's components looks, give the renderer a catalog in which your view shows it. The view reads the component's properties from its context, and shows its children with childView:

let catalog = BasicCatalog.catalog.replacingView(of: "Card") { context in
    AnyView(BrandCard { context.childView(context.childId()) })
}

A2UIRenderer(catalog: catalog)

catalog.removing(components:) leaves out components you don't want the agent to use. To see every component of a catalog with its examples, such as while you restyle them, show DebugCatalogView(catalog: catalog).

OpenUI

OpenUIRenderer reads OpenUI Lang 1.0. The agent writes with the chat library by default, ComponentLibrary.openUIChat, where every reply is one card. Pass library: .openUI for the standard library, with flexible layouts and modals, for surfaces such as dashboards. Each reply keeps its own state, so its forms and $variables survive scrolling. A follow-up or a form's submit button reaches the host as an AIUIAction named after OpenUI's action, labeled with the control the user tapped, with the form's values in its payload. A Query runs one of the host's read-only tools while the reply shows, and a Mutation runs any of them when the user taps.

By default, every image link the model wrote, or a tool returned, loads. To change how one of the library's components looks, give the renderer a library in which your view shows it. The view's context holds the component's props, and the library's own view of it, defaultView, to wrap:

let library = ComponentLibrary.openUIChat.replacingView(of: .button) { context in
    context.defaultView.tint(.orange)
}

OpenUIRenderer(library: library)

library.removing(_:) leaves out components you don't want the agent to use. The library's texts, such as its form labels, go through OpenUITheme.localizationProvider, which works like StreamChatAI's.

Write a renderer

For any other format, write a renderer of your own. A renderer makes an engine for each conversation, which makes the view of each reply's UI:

struct CardsRenderer: AIUIRenderer {
    let format: AIUIFormat = "cards"

    func makeEngine(host: any AIUIHosting) -> CardsEngine {
        CardsEngine()
    }
}

final class CardsEngine: AIUIEngine {
    func view(for document: AIUIDocument, messageID: String) -> CardsView? {
        guard let cards = document.content.arrayValue else { return nil }
        return CardsView(cards: cards)
    }
}

struct CardsView: View {
    let cards: [RawJSON]

    var body: some View {
        VStack(alignment: .leading) {
            ForEach(cards.indices, id: \.self) { index in
                Text(cards[index].dictionaryValue?["title"]?.stringValue ?? "")
                    .aiUIEntrance()
            }
        }
    }
}

List it in the host like any other renderer. The views keep their types, so SwiftUI updates the UI of a reply as it does any other view.

• view(for:messageID:) is asked again each time the reply's document changes, such as while the agent writes it. Return nil to show the reply's text instead.
• The host keeps its engines for as long as it's kept, so an engine can hold each reply's state by its messageID. An engine that needs the host keeps it weakly, since the host keeps the engine.
• receive(_:messageID:) gets each reply's document as it arrives, oldest first, including replies that aren't on screen. Use it for a format whose later replies change an earlier one's UI. remove(messageID:) tells the engine a reply, or its UI, left the conversation, so it can let go of what it kept for it.
• supports(version:) says which versions of the format it reads. It reads every version by default.
• capabilities(components:tools:) says what the agent's backend needs to write the format, such as the instructions for its model. See Tell the agent what the app shows.

A renderer places the app's components by name, from the host's components. A component's view(arguments:context:) makes its view for the arguments the agent wrote, or returns nil while they don't fit its parameters yet. Give it an AIUIComponentContext that sends its events the way your format reports what the user did.

Components

A component is a view of your app the agent can place in a reply's UI. You describe it once, with the parameters the agent fills in and the events it reports, and say how it looks for the arguments the agent wrote:

struct PlaceCardComponent: AIUIComponent {
    struct Arguments: Decodable {
        let title: String
        let visitDate: Date?
    }

    let definition = AIUIComponentDefinition(
        name: "place_card",
        description: "A photo and a short summary of a place.",
        parameters: [
            .required("title", .string, description: "The place's Wikipedia title."),
            .optional("visit_date", .date)
        ],
        events: [.init(name: "ask_about", description: "The user asks about the place.")]
    )

    func body(arguments: Arguments, context: AIUIComponentContext) -> some View {
        PlaceCard(title: arguments.title, date: arguments.visitDate) {
            context.emit("ask_about", label: "Tell me more about \(arguments.title)")
        }
    }
}

Every renderer places it by its name, so it works whichever format the agent writes. A component named like one of the format's own, such as card in A2UI, would replace it, so the renderer leaves it out and logs it. The arguments decode into your Arguments type, with snake_case keys, and dates as a day, such as 2026-11-14, or a moment in RFC 3339. While the agent is still writing them and they don't decode yet, the component waits to show. context.isStreaming tells whether they may still change.

The parameter types are .string, .number, .integer, .boolean, .date, .dateTime, .url, .enumeration(_:), .array(_:) and .object(_:). AIUIParameterType(schema:) takes any other JSON Schema.

A component can load what it shows on the device, such as photos or the user's calendar, which then never reaches the agent.

One description for every platform

A component's definition reads and writes JSON, so your iOS, Android and web apps, and the agent's backend, can share one description of it. Its parameters are a JSON Schema object, as a tool's are:

{
  "name": "place_card",
  "description": "A photo and a short summary of a place.",
  "parameters": {
    "type": "object",
    "properties": {
      "title": {
        "type": "string",
        "description": "The place's Wikipedia title."
      },
      "visit_date": { "type": "string", "format": "date" }
    },
    "required": ["title"],
    "x-order": ["title", "visit_date"]
  },
  "events": [
    { "name": "ask_about", "description": "The user asks about the place." }
  ]
}

Decode it with JSONDecoder to use the same file in your app:

let definitions = try JSONDecoder().decode([AIUIComponentDefinition].self, from: data)

Run your app's tools

A reply's UI can run your app's client side tools on the device, such as to load the user's calendar for the days of a plan. Pass them to the host as tools. They are apart from the tools the agent calls, and they run without asking the user, so the host runs a tool by itself only when its definition says it's read-only, with MCP's readOnlyHint:

let definition = AIClientToolDefinition(
    name: "calendar_events",
    description: "The user's calendar events between two days.",
    inputSchema: [
        "type": "object",
        "properties": [
            "from": ["type": "string", "format": "date"],
            "to": ["type": "string", "format": "date"]
        ],
        "required": ["from", "to"]
    ],
    annotations: .init(readOnlyHint: true)
)

A read-only tool runs whenever the UI asks, such as while it shows. Any other tool runs only when the user taps something that runs it, and fails otherwise. The renderers follow this: an OpenUI Query runs only read-only tools, and a Mutation runs from a tap. The model's instructions list the host's tools, and say which ones are read-only.

A component of yours runs a tool through its context's host, and gets the tool's output as JSON. Pass userInitiated: true when the user asked for it, such as with a tap, and false when the component loads data while it shows:

let events = try await context.host.runTool(
    "calendar_events",
    arguments: ["from": "2026-11-14", "to": "2026-11-15"],
    userInitiated: false
)

The host's open closure opens the links of a reply's UI. By default, web and email links open and others don't, since the agent writes the links, and another kind could call someone or act in another app.

What the user does

When the user does something in a reply's UI that's for the agent, such as tapping a place, choosing a follow-up or submitting a form, the host calls your send closure with an AIUIAction. Send it as the user's next message. The host sends a reply's actions one after another, in the order they happened.

customData holds the action under the ai_ui_action key, for the message's extra data, as above. json is the action alone, for a backend that takes it beside the question. Have your backend tell the model what the user did after their words, and which reply it was in, since the same UI can show in several replies, such as a plan for each of two trips.

The SDK's renderers label an action with the control the user tapped, never with text the agent wrote for it, so the conversation shows only what the user chose. OpenUI sends only the values of the form the action belongs to, and never a password field's value.

Tell the agent what the app shows

The model has to know which formats and components your app shows, and which tools its UI can run. The host's capabilities describes them: each format, with what its renderer adds, and your components' definitions:

let capabilities = ui.capabilities
// ["v": 1, "renderers": [["format": "a2ui", "fence": ..., "instructions": ...], ...], "components": [...]]

Each renderer adds the fence the model writes its UI in, such as openui-lang, and the instructions for the model: the instructions you gave the renderer, then the format's own, written from its components, yours and the host's tools. A2UI also adds the catalog_id of the catalog with your components.

Send the capabilities to your backend, for example when a conversation starts, and add the instructions to what the model is told. You can also write the model's instructions on your backend, from the same component definitions.

Show it in your own views

To show replies in views of your own rather than with AIConversationView, show a reply's UI with AIUIView, and its text when the view's canShow is false:

@ViewBuilder
func reply(_ message: ChatAIMessage) -> some View {
    let generated = message.uiDocument.map { AIUIView(host: ui, document: $0, messageID: message.id) }
    if let generated, generated.canShow {
        generated
    } else {
        StreamingMessageView(content: message.text, isGenerating: message.isGenerating)
    }
}

Each time your messages change, pass them to ui.receive(_:) before the replies show, so the renderers get the UI of every reply, also of those that aren't on screen.

Motion

aiUIEntrance() brings a part of a reply's UI in when it shows: the part fades in, rising into place and coming into focus, after the parts that came in just before it. Parts that show together come in together, so a reply written before it showed shows at once. With Reduce Motion on, parts fade in without moving.

The SDK's renderers give it to each part they place, and every component has it. Use it on the parts of your own renderer's views, as CardsView does above.