Generative UI

An agent can answer with UI rather than text: a weekend plan with a map of the places, the user's own calendar for those days and a form to change the pace. The AI components show a reply's UI in place of its text, bring it in as the agent writes it, and send what the user does in it back to the agent as their next message.

The AI components know no UI format of their own. You give a conversation a renderer for each format your agent writes, and the composables of your app the agent can place, such as a map, a calendar or a card over your own data. The SDK ships renderers of A2UI and OpenUI, and you can write one for any other format. A component works whichever format the agent writes.

How a reply carries its UI

The agent's backend writes a reply's UI in the message's ai_ui custom field. The reply's text stays readable on its own, for wherever the UI can't show, such as a push notification or an app without a renderer for the format:

{
  "text": "Here's a weekend in Ohrid, with a map and a plan for each day.",
  "ai_ui": {
    "v": 1,
    "format": "openui",
    "status": "streaming",
    "content": "root = Card([title, map])\ntitle = CardHeader(\"A weekend in Ohrid\")"
  }
}

A model usually writes the UI into its reply, for example in a fenced code block. Have your backend move it out of the text into ai_ui as the reply streams, so the text keeps only the words around it. A message's custom data holds at most 50 KB, so leave the UI out when it doesn't fit beside the rest of it.

Read the field with AIUIDocument.from, as the uiDocument of your AIMessage, such as the StreamAIMessage from Messages from Stream Chat:

override val uiDocument: AIUIDocument? = AIUIDocument.from(message.extraData)

It returns null when the message carries no ai_ui field the SDK can read, and the reply shows its text.

Show it in a conversation

Make an AIUIHost with the renderers of the formats your agent writes and your components, and pass it to AIConversation as ui. Keep one host for as long as its conversation shows, for example in the ViewModel behind your screen, so each reply's UI keeps its state, such as what the user typed into a form, across scrolling and configuration changes:

class AssistantViewModel(private val cid: String) : ViewModel() {

    val ui: AIUIHost = AIUIHost(
        renderers = listOf(A2UIRenderer(), OpenUIRenderer()),
        components = listOf(placeCard),
    ) { action ->
        ChatClient.instance().channel(cid)
            .sendMessage(Message(text = action.label, extraData = mapOf(AIUIAction.CUSTOM_DATA_KEY to action.json)))
            .enqueue()
    }
}

AIConversation(messages = messages, ui = viewModel.ui)

placeCard is one of your components. A reply shows its UI in place of its text, after its steps, when one of the host's renderers reads its format and version, the first one listed asked first. Otherwise, it shows its text. The UI comes in as the agent writes it: each part fades in, rising into place, after the parts before it. A reply written before it showed, such as one loaded from history, shows at once.

A reply with UI the host can show counts as showing something, so the conversation stops showing the agent thinking as soon as its UI starts. The reply actions slot shows under a reply once its UI is written, as it does under text. Its text is still the reply's text, which the reply doesn't show while it shows its UI.

The host is confined to the main thread.

Renderers

A renderer reads one format, and shows each reply's UI written in it. Add the artifact of each format your agent writes next to stream-chat-android-ai-compose, which both depend on:

dependencies {
    implementation("io.getstream:stream-chat-android-ai-a2ui:$stream_version")
    implementation("io.getstream:stream-chat-android-ai-openui:$stream_version")
}

Both require minSdk 23, like the AI components. Both renderers place your app's components beside their own, run your app's tools, and draw their own components with Material 3, so a reply's UI follows your MaterialTheme. They take these options:

A2UIRenderer also takes a catalog, a catalogId and a mediaUrl, and OpenUIRenderer a library, as described below.

The renderers treat what the agent writes as untrusted: links in a reply's UI open through the host, which opens only web and email links by default, and images load only from the web.

A2UI

A2UIRenderer reads A2UI 1.0, with every component and function of its basic catalog drawn in Compose. The agent writes with the basic catalog, which your components join under the getstream.io:app-catalog id (catalogId changes it). A surface that names the basic catalog's id shows them as well. Each reply has surfaces of its own, kept while the host is, so what the user enters in them survives scrolling.

A button reaches the host as an AIUIAction named after its event, labeled with the button's text, or else with the event's name as words (submit_form is "Submit form"). Its payload holds the surfaceId, the sourceComponentId, the event's resolved context, the event's userMessage when it has one, and the surface's dataModel when the agent created the surface with sendDataModel. A function the catalog doesn't define runs as one of the host's tools, with the tool's output as the function's value.

By default, only web links load as images, video and audio. Pass mediaUrl to return the URL an image, video or audio clip loads from, given the one the agent wrote, or null to not load it, such as to load media through your own CDN. The agent writes these URLs, so return only ones your app trusts. Images load only from web links, whatever it returns:

val renderer: A2UIRenderer = A2UIRenderer(
    mediaUrl = { url -> url.takeIf { it.startsWith("https://media.example.com/") } },
)

To change how one of the basic catalog's components looks, give the renderer a catalog in which your composable draws it. It reads the component through an A2UIComponentScope, with its bindings resolved, and draws its children with the same catalog:

val catalog: A2UICatalog = A2UICatalog.Basic.with("Card") { scope ->
    BrandCard(modifier = scope.modifier) {
        scope.child()?.let { scope.Child(it) }
    }
}

val renderer: A2UIRenderer = A2UIRenderer(catalog = catalog)

Here BrandCard is a composable of your app.

Apply scope.modifier to the outermost element: it carries what the parent asks of the component, such as its weight in a Row, and its accessibility label. The scope reads properties with value, string, number, boolean and stringList, writes an input's value to the surface's data model with write, and runs what the component does with performAction.

removing leaves out components you don't want the agent to use. The agent isn't told about them, and a reply that still uses one shows nothing for it:

val renderer: A2UIRenderer = A2UIRenderer(catalog = A2UICatalog.Basic.removing("Video", "AudioPlayer"))

OpenUI

OpenUIRenderer reads OpenUI Lang 1.0. The agent writes with the chat library by default, openuiChatLibrary, where every reply is one card. Pass library = openuiLibrary for the standard library, with a Stack root, layouts and modals, for surfaces such as dashboards. Each reply keeps its own state while the host is kept, so its forms and $variables survive scrolling.

A follow-up or a form's submit button reaches the host as an AIUIAction named after OpenUI's action, such as continue_conversation, labeled with the control the user tapped. Its payload holds the action's context, its other params, the form and the form's values. A Query runs one of the host's read-only tools while the reply shows, and a Mutation runs any of them when the user taps.

Images load only from public web hosts, at about the size they show at. To change how one of the library's components looks, give the renderer a library in which your composable draws it. It gets the component's ComponentRenderProps, with its props, isStreaming, RenderProp for a prop that holds components, and DefaultView, the library's own view of it, to wrap:

val library: Library = openuiChatLibrary
    .replacingView("Button") { props ->
        Box(Modifier.padding(4.dp)) { props.DefaultView() }
    }
    .removing(setOf("Carousel"))

val renderer: OpenUIRenderer = OpenUIRenderer(library = library)

replacingView keeps the component's props and what the agent is told about it. removing leaves out components you don't want the agent to use, also from every slot that takes several types. componentNames lists the components a library has.

Write a renderer

For any other format, write a renderer of your own. A renderer makes an engine for each conversation's host, which makes the content of each reply's UI:

class CardsRenderer : AIUIRenderer {

    override val format: AIUIFormat = AIUIFormat("cards")

    override fun makeEngine(host: AIUIHost): AIUIEngine = CardsEngine()
}

class CardsEngine : AIUIEngine {

    override fun content(document: AIUIDocument, messageId: String): (@Composable () -> Unit)? {
        val cards = document.content as? List<*> ?: return null
        return { Cards(cards) }
    }
}

@Composable
fun Cards(cards: List<*>) {
    Column {
        cards.forEach { card ->
            AIUIEntrance {
                Text(text = (card as? Map<*, *>)?.get("title") as? String ?: "")
            }
        }
    }
}

List it in the host like any other renderer. A document's content is the agent's text, which text reads, or JSON as plain Kotlin values: maps, lists, strings, numbers, booleans and null.

• content(document, messageId) is asked again each time the reply's document changes, such as while the agent writes it, while the conversation composes. Return null to show the reply's text instead.
• The host makes one engine per renderer and keeps it for as long as it's kept, so an engine can hold each reply's state by its messageId. Keep what shows on screen in snapshot state, so the UI follows it. The host calls the engine on the main thread.
• receive(document, messageId) gets each reply's document as it arrives or changes, oldest first, including replies that aren't on screen. Use it for a format whose later replies change an earlier one's UI. remove(messageId) tells the engine a reply, or its UI, left the conversation, so it can let go of what it kept for it.
• supports(version) says which versions of the format it reads. It reads every version by default.
• capabilities(components, tools) says what the agent's backend needs to write the format, such as the instructions for its model. See Tell the agent what the app shows.

A renderer places the app's components by name, with host.component(name), among the ones its format has. A component's Content(arguments, context) draws it for the arguments the agent wrote. Give it an AIUIComponentContext whose emit sends its events the way your format reports what the user did.

Components

A component is a composable of your app the agent can place in a reply's UI. You describe it once, with the parameters the agent fills in and the events it reports, and say how it looks for the arguments the agent wrote:

val placeCard: AIUIComponent = AIUIComponent(
    AIUIComponentDefinition(
        name = "place_card",
        description = "A photo and a short summary of a place.",
        parameters = listOf(
            AIUIParameter.required("title", AIUIParameterType.String, description = "The place's Wikipedia title."),
            AIUIParameter.optional("visit_date", AIUIParameterType.Date),
        ),
        events = listOf(AIUIComponentDefinition.Event("ask_about", "The user asks about the place.")),
    ),
) { arguments, context ->
    val title = arguments["title"] as? String ?: return@AIUIComponent
    PlaceCard(
        title = title,
        visitDate = arguments["visit_date"] as? String,
        onClick = { context.emit("ask_about", label = "Tell me more about $title") },
    )
}

PlaceCard is a composable of your app. Every renderer places it by its name, so it works whichever format the agent writes. A2UI and OpenUI place it under its name in PascalCase, so place_card is PlaceCard. A component named like one of the format's own, such as Card in A2UI, would replace it, so the renderer leaves it out.

The arguments are the JSON the agent wrote, as plain Kotlin values: maps, lists, strings, numbers, booleans and null. Read what you need, and show nothing while they don't fit, such as while the agent is still writing them. A date arrives as the text the agent wrote, such as 2026-11-14. context.isStreaming tells whether the arguments may still change, and context.messageId which reply the component is in. An event's payload carries what it reports, and the renderer adds which component it came from as component. You can also implement AIUIComponent on a class of your own, with its definition and a Content composable.

The parameter types are AIUIParameterType.String, Number, Integer, Boolean, Date, DateTime and Url, and enumeration(values), array(items) and obj(fields). AIUIParameterType(schema) takes any other JSON Schema.

A component can load what it shows on the device, such as photos or the user's calendar, which then never reaches the agent. context.openUrl(url) opens a link through the host, which decides the kinds of link that open.

One description for every platform

A component's definition reads and writes JSON, so every platform your app runs on, and the agent's backend, can share one description of it. Its parameters are a JSON Schema object, as a tool's are, which keeps their order in x-order:

{
  "name": "place_card",
  "description": "A photo and a short summary of a place.",
  "parameters": {
    "type": "object",
    "properties": {
      "title": {
        "type": "string",
        "description": "The place's Wikipedia title."
      },
      "visit_date": { "type": "string", "format": "date" }
    },
    "required": ["title"],
    "x-order": ["title", "visit_date"]
  },
  "events": [
    { "name": "ask_about", "description": "The user asks about the place." }
  ]
}

Read it with AIUIComponentDefinition.fromJsonString(text), or fromJson(map), to use the same file in your app. Both return null for a definition without a name. toJson() and toJsonString() write it.

Run your app's tools

A reply's UI can run your app's client-side tools on the device, such as to load the user's calendar for the days of a plan. Pass them to the host as tools. They are apart from the tools the agent calls, and they run without asking the user, so the host runs a tool by itself only when it says it's read-only:

class CalendarEventsTool : AIClientTool {

    override val name: String = "calendar_events"

    override val description: String = "The user's calendar events between two days."

    override val inputSchema: Map<String, Any?> = mapOf(
        "type" to "object",
        "properties" to mapOf(
            "from" to mapOf("type" to "string", "format" to "date"),
            "to" to mapOf("type" to "string", "format" to "date"),
        ),
        "required" to listOf("from", "to"),
    )

    override val isReadOnly: Boolean = true

    override suspend fun run(call: AIToolCallPart): AIClientToolResult {
        val arguments = call.arguments?.let { runCatching { JSONObject(it) }.getOrNull() }
            ?: return AIClientToolResult.failed("The days are missing")
        val events = DeviceCalendar.eventsJson(arguments.optString("from"), arguments.optString("to"))
        return AIClientToolResult.completed(outputJson = events, summary = "Read the calendar")
    }
}

Here DeviceCalendar stands for your app's own access to the calendar, and sendToAgent for sending an action as shown in Show it in a conversation.

val ui: AIUIHost = AIUIHost(
    renderers = listOf(A2UIRenderer(), OpenUIRenderer()),
    components = listOf(placeCard),
    tools = listOf(CalendarEventsTool()),
) { action -> sendToAgent(action) }

A read-only tool runs whenever the UI asks, such as while it shows. Any other tool runs only when the user taps something that runs it, and fails otherwise. The renderers follow this: an OpenUI Query runs only read-only tools, and a Mutation runs from a tap, and an A2UI function runs a tool that isn't read-only only from a tap. The model's instructions list the host's tools, with their descriptions and arguments, and say which ones are read-only.

A component of yours runs a tool through its context's host, and gets the tool's output, JSON, as plain Kotlin values. runTool suspends, so call it from a coroutine, such as a LaunchedEffect. Pass userInitiated = true when the user asked for it, such as with a tap, and false when the component loads data while it shows:

var events by remember { mutableStateOf<Any?>(null) }
LaunchedEffect(context.messageId) {
    events = try {
        context.host.runTool(
            name = "calendar_events",
            arguments = mapOf("from" to "2026-11-14", "to" to "2026-11-15"),
            userInitiated = false,
        )
    } catch (e: AIUIToolException) {
        null
    }
}

runTool throws an AIUIToolException when the host has no such tool, the tool isn't read-only and the user didn't ask for it, it fails, or its output isn't JSON.

The host's linkSchemes are the kinds of link a reply's UI may open: http, https and mailto by default, since the agent writes the links, and another kind could call someone or act in another app. Add another scheme only when your app wants it.

What the user does

When the user does something in a reply's UI that's for the agent, such as tapping a place, choosing a follow-up or submitting a form, the host calls your send function with an AIUIAction. Send it as the user's next message.

json is the action as the JSON object an agent's backend reads. Put it in the message's custom data under the ai_ui_action key (AIUIAction.CUSTOM_DATA_KEY), as above, or send it to a backend that takes it beside the question. customData is the same, as a map. Have your backend tell the model what the user did after their words, and which reply it was in, since the same UI can show in several replies, such as a plan for each of two trips.

The SDK's renderers label an action with the control the user tapped, never with text the agent wrote for it, so the conversation shows only what the user chose. OpenUI sends only the values of the form the action belongs to, and never what a password field holds, which it also leaves out of the state it keeps for the reply. What the user types into an A2UI text field that's obscured is in none of the action's details.

Tell the agent what the app shows

The model has to know which formats and components your app shows, and which tools its UI can run. The host's capabilities describes them, as plain Kotlin values: each format, with what its renderer adds, and your components' definitions:

val capabilities: Map<String, Any?> = ui.capabilities
// {"v": 1, "renderers": [{"format": "a2ui", "fence": ..., "catalog_id": ..., "instructions": ...}, ...], "components": [...]}

Each renderer adds the fence the model writes its UI in, ui:a2ui@1.0 or openui-lang, and the instructions for the model: the instructions you gave the renderer, then the format's own, written from its components, yours and the host's tools. A2UI also adds the catalog_id of the catalog with your components.

Send the capabilities to your backend, for example when a conversation starts, and add the instructions to what the model is told. You can also write the model's instructions on your backend, from the same component definitions.

Theming

The renderers' own components follow your MaterialTheme. To restyle one of them, replace its view, with A2UI's catalog or OpenUI's library.

OpenUI's components read their colors, spacing, corner radii and text styles from an OpenUITheme, which by default OpenUITheme.fromMaterial() makes from the MaterialTheme. To change it, provide a theme of your own around the conversation:

val material: OpenUITheme = OpenUITheme.fromMaterial()
val theme: OpenUITheme = material.copy(
    colors = material.colors.copy(accent = BrandOrange),
    radius = OpenUITheme.Radius(small = 4.dp, medium = 6.dp, large = 8.dp),
)
ProvideOpenUITheme(theme) {
    AIConversation(messages = messages, ui = viewModel.ui)
}

Here BrandOrange is a color of your app. The theme applies to the OpenUI components drawn inside ProvideOpenUITheme.

The renderers' texts, such as form validation messages, are string resources whose names start with stream_ai_a2ui_ and stream_ai_openui_, which you can override or translate in your app's strings.xml.

Errors

A reply's UI can't show everything the agent wrote, such as a component the format doesn't know, a message that doesn't read or a tool that failed. Each renderer passes these problems to its onError, with the reply's id. A chat has no channel for them, so forward them to your backend if the agent should hear about them, for example to have the model correct its UI. Without onError, they are logged:

val renderer: OpenUIRenderer = OpenUIRenderer(
    onError = { error: OpenUIError, messageId: String ->
        agentBackend.reportUIError(messageId, error.toString())
    },
)

Here agentBackend stands for your app's connection to the agent's backend.

• A2UIRenderer passes an A2UIError, the problems A2UI reports to the agent: its code in A2UI 1.0, such as VALIDATION_FAILED, the message, written for the agent, and the surfaceId and the path, a JSON Pointer to the part of the agent's message at fault, when they apply.
• OpenUIRenderer passes an OpenUIError once the reply is complete: its source (Parser, Runtime, Query or Mutation), a code, such as unknown-component or tool-not-found, the message, and, when they apply, the statementId, component, path and toolName, a hint to fix it, and isWarning for a problem the reply still shows despite.

Show it in your own views

To show replies with views of your own rather than with AIConversation, show a reply's UI with AIUIView, and its text when showsUI is false:

@Composable
fun Reply(message: AIMessage, ui: AIUIHost) {
    val document = message.uiDocument
    if (document != null && ui.showsUI(document, message.id)) {
        AIUIView(host = ui, document = document, messageId = message.id)
    } else {
        StreamingText(text = message.text, animate = message.isGenerating)
    }
}

canShow(document) tells whether one of the host's renderers reads a document, and showsUI(document, messageId) whether it also shows something of its own for the reply. Each time your messages change, pass them to ui.receive(messages), oldest first, before the replies show, so the renderers get the UI of every reply, also of those that aren't on screen. AIConversation does it for you.

Motion

AIUIEntrance brings a part of a reply's UI in when it shows: the part fades in, rising into place and coming into focus, after the parts that came in just before it. Parts that show together come in together, so a reply written before it showed shows at once. With the system's animations off, parts show at once.

The SDK's renderers give it to each part they place, and to every component. Use it on the parts of your own renderer's content, as Cards does above. Outside a reply's UI, it shows its content as it is.