# Generative UI An agent can answer with UI rather than text: a weekend plan with a map of the places, the user's own calendar for those days and a form to change the pace. StreamChatAI shows a reply's UI in place of its text, brings it in as the agent writes it, and sends what the user does in it back to the agent as their next message. StreamChatAI knows no UI format of its own. You give a conversation a renderer for each format your agent writes, and the views of your app the agent can place, such as a map, a calendar or a card over your own data. The SDK ships renderers of [A2UI](https://a2ui.org) and [OpenUI](https://github.com/thesysdev/openui), and you can write one for any other format. A component works whichever format the agent writes. > > **Info:** Generative UI is part of StreamChatAI, and the A2UI and OpenUI renderers are the StreamChatA2UI and StreamChatOpenUI libraries, all available from version 5.16.0 of the Stream Chat Swift SDK. See [Installation](https://getstream.io/chat/docs/sdk/ios/guides/ai-integrations/#installation). > ## How a reply carries its UI The agent's backend writes a reply's UI in the message's `ai_ui` custom field. The reply's text stays readable on its own, for wherever the UI can't show, such as a push notification or an app without a renderer for the format: ```json { "text": "Here's a weekend in Ohrid, with a map and a plan for each day.", "ai_ui": { "v": 1, "format": "openui", "status": "streaming", "content": "root = Card([title, map])\ntitle = CardHeader(\"A weekend in Ohrid\")" } } ``` | Field | Description | | --------- | ----------------------------------------------------------------------------------------------- | | `v` | The version of the field, `1`. An app that doesn't know a newer version shows the reply's text. | | `format` | The format the UI is written in, such as `a2ui` or `openui`. | | `version` | The format's version, such as `1.0`, when the agent names one. Optional. | | `status` | `streaming` while the agent is still writing the UI, then `complete`. | | `content` | The UI in its format: text, such as an OpenUI program or A2UI messages one per line, or JSON. | A model usually writes the UI into its reply, for example in a fenced code block. Have your backend move it out of the text into `ai_ui` as the reply streams, so the text keeps only the words around it. A message's custom data holds at most 50 KB, so leave the UI out when it doesn't fit beside the rest of it. Read the field on your messages with `AIUIDocument(customData:)`, as the `uiDocument` of your `AIMessage`: ```swift struct ChatAIMessage: AIMessage { let message: ChatMessage // The id, role, text and the rest, as in the Conversation View guide. var uiDocument: AIUIDocument? { AIUIDocument(customData: message.extraData) } } ``` ## Show it in a conversation Make an `AIUIHost` with the renderers of the formats your agent writes and your components, and pass it to `AIConversationView` as `ui`. Keep one host for as long as its conversation shows, for example in the model behind your screen, so each reply's UI keeps its state, such as what the user typed into a form: ```swift import StreamChatA2UI import StreamChatAI import StreamChatOpenUI let ui = AIUIHost(components: [PlaceCardComponent()]) { A2UIRenderer() OpenUIRenderer() } send: { action in try await chat.sendMessage(with: action.label, extraData: action.customData) } AIConversationView(messages: messages, ui: ui) ``` Here `chat` is the Stream Chat `Chat` of the conversation's channel. A reply shows its UI in place of its text when one of the host's renderers reads its format and version, the first one listed asked first. Otherwise, it shows its text. The UI comes in as the agent writes it: each part fades in, rising into place, after the parts before it. A reply written before it showed, such as one loaded from history, shows at once. [Reply actions](https://getstream.io/chat/docs/sdk/ios/guides/ai-integrations/conversation-view/#reply-actions) show under a reply once its UI is written, as they do under text. Their `text` is the reply's words, which a reply that only shows UI doesn't have, so copying and sharing are left out for it. With an [`AIConversation`](https://getstream.io/chat/docs/sdk/ios/guides/ai-integrations/conversation-state/#generative-ui), pass the host the same way: ```swift AIConversationView(conversation: conversation, ui: ui) ``` `OpenUIRenderer` requires iOS 17. If your app supports earlier versions, make a host with it only on iOS 17 and later, and one with the other renderers before that. ## Renderers A renderer reads one format, and makes the views of each reply's UI written in it. Add the library of each format your agent writes to your app target, next to StreamChatAI: | Library | Renderer | Format | Requires | | ------------------ | ---------------- | --------------- | -------- | | `StreamChatA2UI` | `A2UIRenderer` | A2UI 1.0 | iOS 15 | | `StreamChatOpenUI` | `OpenUIRenderer` | OpenUI Lang 1.0 | iOS 17 | Both renderers place your app's [components](#components) beside their own, run your app's [tools](#run-your-apps-tools), and style their views with the conversation's [`AIAppearance`](https://getstream.io/chat/docs/sdk/ios/guides/ai-integrations/#customizing-the-appearance), so a reply's UI looks like the rest of the conversation. They take the same options: | Parameter | Description | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `instructions` | What the agent's model is told ahead of the format's own instructions, such as which answers to write as UI and when to use your components. See [Tell the agent what the app shows](#tell-the-agent-what-the-app-shows). | | `theme` | The look of the format's own components, an `A2UITheme` or an `OpenUITheme`. By default it's made from the appearance, with `A2UITheme(appearance:)` or `OpenUITheme(appearance:)`. | | `mediaURL` | The URL an image (and with A2UI, a video or an audio clip) loads from, given the one the agent wrote, or `nil` to not load it, such as to allow only your own image hosts. | | `onError` | Gets the problems with a reply's UI, such as a component the format doesn't know or a tool that failed, with the reply's id. A chat has no channel for them, so forward them to your backend if the agent should hear about them. They are logged by default. | The renderers treat what the agent writes as untrusted: a reply's UI can't crash or freeze the app, its links open through the host, which opens only web and email links by default, and images download at most once, capped at 20 MB. ### A2UI `A2UIRenderer` reads [A2UI 1.0](https://a2ui.org), with its basic catalog of components and functions drawn in SwiftUI. The agent writes with the basic catalog, which your components join under the `getstream.io:app-catalog` id (`catalogId` changes it). A button reaches the host as an `AIUIAction` named after its event, labeled with the button's text, with the surface's data model when the agent asks for it with `sendDataModel`. A function the catalog doesn't define runs as one of the host's tools. By default, only `http` and `https` media load (`A2UIRenderer.webURL(_:)`). To change how one of the basic catalog's components looks, give the renderer a catalog in which your view shows it. The view reads the component's properties from its context, and shows its children with `childView`: ```swift let catalog = BasicCatalog.catalog.replacingView(of: "Card") { context in AnyView(BrandCard { context.childView(context.childId()) }) } A2UIRenderer(catalog: catalog) ``` `catalog.removing(components:)` leaves out components you don't want the agent to use. To see every component of a catalog with its examples, such as while you restyle them, show `DebugCatalogView(catalog: catalog)`. ### OpenUI `OpenUIRenderer` reads [OpenUI Lang 1.0](https://github.com/thesysdev/openui). The agent writes with the chat library by default, `ComponentLibrary.openUIChat`, where every reply is one card. Pass `library: .openUI` for the standard library, with flexible layouts and modals, for surfaces such as dashboards. Each reply keeps its own state, so its forms and `$variables` survive scrolling. A follow-up or a form's submit button reaches the host as an `AIUIAction` named after OpenUI's action, labeled with the control the user tapped, with the form's values in its payload. A `Query` runs one of the host's read-only tools while the reply shows, and a `Mutation` runs any of them when the user taps. By default, every image link the model wrote, or a tool returned, loads. To change how one of the library's components looks, give the renderer a library in which your view shows it. The view's context holds the component's props, and the library's own view of it, `defaultView`, to wrap: ```swift let library = ComponentLibrary.openUIChat.replacingView(of: .button) { context in context.defaultView.tint(.orange) } OpenUIRenderer(library: library) ``` `library.removing(_:)` leaves out components you don't want the agent to use. The library's texts, such as its form labels, go through `OpenUITheme.localizationProvider`, which works like [StreamChatAI's](https://getstream.io/chat/docs/sdk/ios/guides/ai-integrations/#localization). ### Write a renderer For any other format, write a renderer of your own. A renderer makes an engine for each conversation, which makes the view of each reply's UI: ```swift struct CardsRenderer: AIUIRenderer { let format: AIUIFormat = "cards" func makeEngine(host: any AIUIHosting) -> CardsEngine { CardsEngine() } } final class CardsEngine: AIUIEngine { func view(for document: AIUIDocument, messageID: String) -> CardsView? { guard let cards = document.content.arrayValue else { return nil } return CardsView(cards: cards) } } struct CardsView: View { let cards: [RawJSON] var body: some View { VStack(alignment: .leading) { ForEach(cards.indices, id: \.self) { index in Text(cards[index].dictionaryValue?["title"]?.stringValue ?? "") .aiUIEntrance() } } } } ``` List it in the host like any other renderer. The views keep their types, so SwiftUI updates the UI of a reply as it does any other view. - `view(for:messageID:)` is asked again each time the reply's document changes, such as while the agent writes it. Return `nil` to show the reply's text instead. - The host keeps its engines for as long as it's kept, so an engine can hold each reply's state by its `messageID`. An engine that needs the host keeps it weakly, since the host keeps the engine. - `receive(_:messageID:)` gets each reply's document as it arrives, oldest first, including replies that aren't on screen. Use it for a format whose later replies change an earlier one's UI. `remove(messageID:)` tells the engine a reply, or its UI, left the conversation, so it can let go of what it kept for it. - `supports(version:)` says which versions of the format it reads. It reads every version by default. - `capabilities(components:tools:)` says what the agent's backend needs to write the format, such as the instructions for its model. See [Tell the agent what the app shows](#tell-the-agent-what-the-app-shows). A renderer places the app's components by name, from the host's `components`. A component's `view(arguments:context:)` makes its view for the arguments the agent wrote, or returns `nil` while they don't fit its parameters yet. Give it an `AIUIComponentContext` that sends its events the way your format reports what the user did. ## Components A component is a view of your app the agent can place in a reply's UI. You describe it once, with the parameters the agent fills in and the events it reports, and say how it looks for the arguments the agent wrote: ```swift struct PlaceCardComponent: AIUIComponent { struct Arguments: Decodable { let title: String let visitDate: Date? } let definition = AIUIComponentDefinition( name: "place_card", description: "A photo and a short summary of a place.", parameters: [ .required("title", .string, description: "The place's Wikipedia title."), .optional("visit_date", .date) ], events: [.init(name: "ask_about", description: "The user asks about the place.")] ) func body(arguments: Arguments, context: AIUIComponentContext) -> some View { PlaceCard(title: arguments.title, date: arguments.visitDate) { context.emit("ask_about", label: "Tell me more about \(arguments.title)") } } } ``` Every renderer places it by its name, so it works whichever format the agent writes. A component named like one of the format's own, such as `card` in A2UI, would replace it, so the renderer leaves it out and logs it. The arguments decode into your `Arguments` type, with snake_case keys, and dates as a day, such as `2026-11-14`, or a moment in RFC 3339. While the agent is still writing them and they don't decode yet, the component waits to show. `context.isStreaming` tells whether they may still change. The parameter types are `.string`, `.number`, `.integer`, `.boolean`, `.date`, `.dateTime`, `.url`, `.enumeration(_:)`, `.array(_:)` and `.object(_:)`. `AIUIParameterType(schema:)` takes any other JSON Schema. A component can load what it shows on the device, such as photos or the user's calendar, which then never reaches the agent. ### One description for every platform A component's definition reads and writes JSON, so your iOS, Android and web apps, and the agent's backend, can share one description of it. Its parameters are a JSON Schema object, as a tool's are: ```json { "name": "place_card", "description": "A photo and a short summary of a place.", "parameters": { "type": "object", "properties": { "title": { "type": "string", "description": "The place's Wikipedia title." }, "visit_date": { "type": "string", "format": "date" } }, "required": ["title"], "x-order": ["title", "visit_date"] }, "events": [ { "name": "ask_about", "description": "The user asks about the place." } ] } ``` Decode it with `JSONDecoder` to use the same file in your app: ```swift let definitions = try JSONDecoder().decode([AIUIComponentDefinition].self, from: data) ``` ## Run your app's tools A reply's UI can run your app's [client side tools](https://getstream.io/chat/docs/sdk/ios/guides/ai-integrations/client-side-tools/) on the device, such as to load the user's calendar for the days of a plan. Pass them to the host as `tools`. They are apart from the tools the agent calls, and they run without asking the user, so the host runs a tool by itself only when its definition says it's read-only, with MCP's `readOnlyHint`: ```swift let definition = AIClientToolDefinition( name: "calendar_events", description: "The user's calendar events between two days.", inputSchema: [ "type": "object", "properties": [ "from": ["type": "string", "format": "date"], "to": ["type": "string", "format": "date"] ], "required": ["from", "to"] ], annotations: .init(readOnlyHint: true) ) ``` A read-only tool runs whenever the UI asks, such as while it shows. Any other tool runs only when the user taps something that runs it, and fails otherwise. The renderers follow this: an OpenUI `Query` runs only read-only tools, and a `Mutation` runs from a tap. The model's instructions list the host's tools, and say which ones are read-only. A component of yours runs a tool through its context's host, and gets the tool's output as JSON. Pass `userInitiated: true` when the user asked for it, such as with a tap, and `false` when the component loads data while it shows: ```swift let events = try await context.host.runTool( "calendar_events", arguments: ["from": "2026-11-14", "to": "2026-11-15"], userInitiated: false ) ``` The host's `open` closure opens the links of a reply's UI. By default, web and email links open and others don't, since the agent writes the links, and another kind could call someone or act in another app. ## What the user does When the user does something in a reply's UI that's for the agent, such as tapping a place, choosing a follow-up or submitting a form, the host calls your `send` closure with an `AIUIAction`. Send it as the user's next message. The host sends a reply's actions one after another, in the order they happened. | Property | Description | | ----------- | ------------------------------------------------------------------------------------------------------------------------------ | | `messageID` | The reply whose UI it came from. | | `format` | The format of that UI. | | `name` | What happened, such as a component's event (`ask_about`) or an action of the format, such as OpenUI's `continue_conversation`. | | `label` | What the user did, in their words, such as "Tell me more about St. Naum". Use it as the message's text. | | `payload` | The details, in the format's own shape, such as what was chosen and a form's values. | `customData` holds the action under the `ai_ui_action` key, for the message's extra data, as above. `json` is the action alone, for a backend that takes it beside the question. Have your backend tell the model what the user did after their words, and which reply it was in, since the same UI can show in several replies, such as a plan for each of two trips. The SDK's renderers label an action with the control the user tapped, never with text the agent wrote for it, so the conversation shows only what the user chose. OpenUI sends only the values of the form the action belongs to, and never a password field's value. ## Tell the agent what the app shows The model has to know which formats and components your app shows, and which tools its UI can run. The host's `capabilities` describes them: each format, with what its renderer adds, and your components' definitions: ```swift let capabilities = ui.capabilities // ["v": 1, "renderers": [["format": "a2ui", "fence": ..., "instructions": ...], ...], "components": [...]] ``` Each renderer adds the `fence` the model writes its UI in, such as `openui-lang`, and the `instructions` for the model: the `instructions` you gave the renderer, then the format's own, written from its components, yours and the host's tools. A2UI also adds the `catalog_id` of the catalog with your components. Send the capabilities to your backend, for example when a conversation starts, and add the instructions to what the model is told. You can also write the model's instructions on your backend, from the same component definitions. ## Show it in your own views To show replies in views of your own rather than with `AIConversationView`, show a reply's UI with `AIUIView`, and its text when the view's `canShow` is false: ```swift @ViewBuilder func reply(_ message: ChatAIMessage) -> some View { let generated = message.uiDocument.map { AIUIView(host: ui, document: $0, messageID: message.id) } if let generated, generated.canShow { generated } else { StreamingMessageView(content: message.text, isGenerating: message.isGenerating) } } ``` Each time your messages change, pass them to `ui.receive(_:)` before the replies show, so the renderers get the UI of every reply, also of those that aren't on screen. ## Motion `aiUIEntrance()` brings a part of a reply's UI in when it shows: the part fades in, rising into place and coming into focus, after the parts that came in just before it. Parts that show together come in together, so a reply written before it showed shows at once. With Reduce Motion on, parts fade in without moving. The SDK's renderers give it to each part they place, and every component has it. Use it on the parts of your own renderer's views, as `CardsView` does above. --- For the most recent version of this documentation, visit [https://getstream.io/chat/docs/sdk/ios/guides/ai-integrations/generative-ui/](https://getstream.io/chat/docs/sdk/ios/guides/ai-integrations/generative-ui/).