# Reasoning and Tool Calls Before an agent answers, it often reasons, and it may call tools: search the web, look up an order, or ask the user's device for their location. The AI components show these steps in the reply, in the order they happened, with the final answer after them. An agent describes each step as a custom attachment on its reply message: - `ai_reasoning` - a round of the model's reasoning. - `ai_tool_call` - a tool the agent called, run by the agent's backend or by a user's device. The order of the attachments is the order of the steps, and the final answer stays in the message text. The agent's backend adds the steps to the reply's `attachments` and updates them as it works, with the same message updates that stream the text. > > **Info:** The views and types on this page are available from version 0.2.0 of `stream_chat_flutter_ai`. See [Install the SDKs](https://getstream.io/chat/docs/sdk/flutter/ai-components/#step-2-install-the-sdks). > ## Show the steps of a reply `AIMessagePart.parseAll` decodes the steps among a message's attachments, given as `(type:, payload:)` records, skipping attachments that aren't AI steps, such as images. `AIMessagePartsView` shows them in order. Render it before the reply's text: ```dart import 'package:flutter/material.dart'; // stream_chat_flutter has older versions of some AI widgets, so hide those. import 'package:stream_chat_flutter/stream_chat_flutter.dart' hide StreamingMessageView, AITypingIndicatorView, TypewriterState, TypewriterValue, TypewriterController, TypewriterWidgetBuilder; import 'package:stream_chat_flutter_ai/stream_chat_flutter_ai.dart'; List aiParts(Message message) => AIMessagePart.parseAll([ for (final attachment in message.attachments) if (attachment.rawType case final type?) (type: type, payload: {'id': attachment.id, ...attachment.extraData}), ]); class AssistantReply extends StatelessWidget { const AssistantReply({super.key, required this.message, required this.isGenerating}); final Message message; final bool isGenerating; @override Widget build(BuildContext context) { return Column( crossAxisAlignment: CrossAxisAlignment.start, spacing: 8, children: [ AIMessagePartsView(parts: aiParts(message)), StreamingMessageView(text: message.text ?? '', expectMoreText: isGenerating), ], ); } } ``` Read the type from `rawType`, because the Stream Chat SDK can rewrite `type`, for example into a link preview. Spread `extraData` last, so the step's own `id` wins over `Attachment.id`: a message from the network carries the step's `id` in `extraData`, while the offline database stores it as `Attachment.id`. Your backend must set an `id` on every step. Without one, each update gets a new random `Attachment.id`, and the step starts over. If you have a single step as a JSON map that carries its `type` alongside its fields, such as the output of `toJson`, use `AIMessagePart.fromJson`. Each step shows through its kind's default view: a round of reasoning as a `StreamingReasoningView`, a tool call as an `AIToolCallView`, and a neutral `AIUnsupportedPartView` ("A step this app version can't show") for a step it can't read. The [conversation view](https://getstream.io/chat/docs/sdk/flutter/ai-components/conversation-view/) renders the steps for you. To show them in the Chat SDK's message list instead, see the [Stream Chat integration](https://getstream.io/chat/docs/sdk/flutter/ai-components/stream-chat-integration/#rendering-reasoning-and-tool-calls). ### Steps are an open set New kinds of steps can appear without breaking your code. A part's `kind` is an `AIMessagePartKind`, a `String` extension type holding the attachment type (`AIMessagePartKind.reasoning`, `AIMessagePartKind.toolCall`, or any other `ai_` type), and `isKnown` says whether the package knows it. The kinds the package reads have typed views, `part.reasoning` (an `AIReasoningPart`) and `part.toolCall` (an `AIToolCallPart`). Anything else keeps its `payload`, a map of its fields, which you can read with `part.decode(fromJson)`. Statuses and executors are open in the same way (`AIReasoningStatus`, `AIToolCallStatus`, `AIToolExecutor`), so give a `switch` over them a default case. Decoding is lenient: missing fields get defaults, a field of the wrong type or an empty string reads as missing, and a step in a newer format version (`v`) keeps its payload but has no typed view (`isSupported` is `false`). Each step has a stable `id` (a tool call uses the model provider's tool-call ID), so the list updates smoothly while it streams. To show some steps your own way, such as reasoning your backend streams in full, or a kind of step of your own, return a widget from `partBuilder` (any step) or `toolCallBuilder` (tool calls only). Return `null` to keep the default view, or `AIMessagePartsView.hidden` to leave the step out of the list: ```dart AIMessagePartsView( parts: parts, partBuilder: (context, part) { if (part.reasoning case final reasoning?) { return StreamingReasoningView.part(reasoning, text: fullReasoning[reasoning.id]); } if (part.kind == const AIMessagePartKind('ai_citation')) { if (part.decode(Citation.fromJson) case final citation?) return CitationView(citation: citation); } return null; // the default view }, toolCallBuilder: (context, call) => switch (call.name) { 'get_weather' => WeatherCard(call: call), 'log_event' => AIMessagePartsView.hidden, _ => null, // the default row }, ) ``` `partBuilder` is asked first, and `toolCallBuilder` only sees the tool calls it returned `null` for. A hidden tool call that waits for this user's approval or sign-in still asks it, without its row. To replace a kind's view everywhere instead, use the `reasoning`, `toolCall` and `unsupportedPart` builders of `streamAIComponentBuilders`, registered in a `StreamComponentFactory`, as described in [Replacing components](https://getstream.io/chat/docs/sdk/flutter/ai-components/#replacing-components). The builders you pass to `AIMessagePartsView` are asked before it. `AIMessagePartsView` keys its children by their `id`, so each keeps its state, such as whether its reasoning is open, as the reply updates. ## Reasoning A round of reasoning (`AIReasoningPart`) carries a capped `preview` of the model's thoughts (the latest ones while it streams, the opening once it's done) and a one-line `summary`. `AIMessagePartsView` shows it with `StreamingReasoningView.part`: while the model thinks, the reasoning streams into an open panel under a "Thinking... 7s" header, and once it's done, the view folds into "Thought for 12s" and the summary. Tapping the header opens or closes the reasoning. You can also use `StreamingReasoningView` on its own, with reasoning you get some other way, for example streamed from your backend: ```dart StreamingReasoningView( text: reasoning, isThinking: answer.isEmpty, duration: thinkingDuration, ) ``` The header counts for as long as `isThinking` is `true`, so if your stream stalls or fails, pass `false` yourself. Reasoning can run to tens of kilobytes and grow many times a second, so the view only lays out what changes: it splits the text into paragraphs that render lazily, so only the paragraph still being written is laid out again. Blank lines separate paragraphs, and inline Markdown (bold, italics, strikethrough, code and links) is rendered. | Parameter | Description | Default | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | | `text` | The reasoning so far. | - | | `isThinking` | Whether the model is still thinking. While it is, the header counts seconds. | - | | `duration` | How long the model thought. While it thinks, the header counts on from it. Without one, the view uses what it measured itself. | `null` | | `summary` | A one-line summary shown beside the header while the finished reasoning is folded. | `null` | | `footnote` | A note under the open reasoning, such as how long it's kept. Shown once the model is done, when the reasoning is open. | `null` | | `expandedWhenDone` | Whether the reasoning stays open once the model is done. | `false` | | `showsLiveReasoning` | Whether the reasoning is open while the model thinks. | `true` | | `theme` | A `ReasoningThemeData` for this view, layered over the ambient theme. Its `maxExpandedHeight` (260 by default) sets how tall the reasoning grows before it scrolls. | `null` | | `onExpandedChanged` | Called when the reasoning opens or folds, by the user or by itself. | `null` | | `onTapLink` | Called when the user taps a link in the reasoning. Without it, links are drawn but not tappable. | `null` | `StreamingReasoningView.part(reasoning)` builds the view from an `AIReasoningPart`: it reads `isThinking`, `duration` and `summary` from the step, and shows its `preview` (or else its `summary`) unless you pass the full reasoning as `text`. Once the reader opens or closes the reasoning themselves, the view keeps their choice. `StreamingReasoningView.foldDuration` (450 ms) and `StreamingReasoningView.foldCurve` are how the view folds and opens: use them for your own changes that should move along with it, or to let the reasoning fold before the answer starts typing, rather than the two moving against each other. With reduced motion on (`MediaQueryData.disableAnimations`), nothing shimmers and the reasoning opens and folds at once. ## Tool calls `AIToolCallView` shows a single tool call (`AIToolCallPart`): what it's doing (its `display_title`, such as "Checking your location", or else its name with underscores as spaces), its outcome (`summary`), and its duration once it's finished. ```dart if (part.toolCall case final call?) { return AIToolCallView(call: call); } ``` The icon follows the call's status, and without a `summary`, a call that waits or didn't complete says so in words under its title: | Status | Icon | Line under the title, without a `summary` | | ------------------------ | ------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | `running` | A spinner | - | | `awaiting_approval` | A hand | "Waiting for approval" | | `awaiting_authorization` | A key | "Waiting for sign-in", or "Waiting for you to sign in" for the user it waits for, when the sign-in is offered on their device | | `awaiting_client` | A phone | "Waiting for the device" | | `completed` | A check mark | - | | `failed` | An exclamation mark | "Didn't complete" | | `cancelled` | A cross | "Cancelled", or "Declined" for a call the user declined | A status the package doesn't know shows as still running. The duration shows once the call has completed, failed or was cancelled, such as `0.4s`, `12s` or `1m 5s`. When the agent asks a user's device to run a tool, `AIClientToolRunner` runs it there and sends the result back, as described in [Tools on the device](#tools-on-the-device). ## Tools on the device An agent can ask the user's device to run a tool, such as reading their calendar or opening a screen of your app. Only the tool's answer leaves the device. With [Stream Agents](https://getstream.io/agents/docs/quickstart/?client=flutter), pass the session `AgentTool`s in its `SessionOptions` when you open it, and the agents SDK runs them, as the [Personal assistant](https://getstream.io/chat/docs/sdk/flutter/ai-components/use-cases/personal-assistant/#give-the-agent-the-calendar-and-reminders) use case shows. With a backend of your own, the AI components run them: 1. Implement `AIClientTool` for each tool, or extend `AIResultClientTool` for a tool that returns a result to the agent. 2. Give your tools to an `AIClientToolRunner`, and tell your backend about them, so the agent offers them to the model. 3. Hand the runner the steps of each reply. It runs the calls addressed to this device. ### Define a tool A tool has a `definition`, an `AIToolDefinition`: the `name` the agent knows it by, a `description` of what it does, and the JSON Schema of its arguments as `parameters`, in plain Dart maps. A tool that returns a result to the agent extends `AIResultClientTool`. When the agent calls the tool, `run` does the work and returns an `AIClientToolResult`: ```dart class OpenScreenTool extends AIResultClientTool { const OpenScreenTool({required this.onOpen}); static const screens = {'settings', 'profile'}; final ValueChanged onOpen; @override AIToolDefinition get definition => const AIToolDefinition( name: 'openScreen', description: 'Open a screen of the app', parameters: { 'type': 'object', 'properties': { 'screen': { 'type': 'string', 'enum': ['settings', 'profile'], }, }, 'required': ['screen'], }, ); @override Future run(AIToolCallPart call) async { // Null when the call arrived without its arguments. final screen = call.arguments?['screen']; if (screen is! String || !screens.contains(screen)) { return const AIClientToolResult.failed('The screen to open is missing'); } onOpen(screen); return AIClientToolResult.completed( {'opened': screen}, summary: 'Opened $screen', ); } } ``` - `AIClientToolResult.completed(output, summary: ...)` carries the `output` for the model and an optional short `summary` of the outcome. The output can be any JSON-encodable value, `null` included, and an object with a `toJson` method works too. It's stored in its JSON form, so changing the object afterwards doesn't change the result. A value that can't be encoded throws a `JsonUnsupportedObjectError`. Thrown from `run`, it makes the runner answer the call with the failure "The result couldn't be read". - `AIClientToolResult.failed(reason)` reports that the device couldn't run the call, such as "Location not shared". The reason is told to the model. If `run` throws, the runner reports the call as failed (`The "openScreen" tool failed on this device`), so the agent always gets an answer. - `run` runs on the same isolate as your widgets, so a tool can update your UI state directly, such as to show a dialog or open a screen, as `OpenScreenTool` does through `onOpen`. - `toJson(toolCallId: ..., clientId: ...)` gives the result in the shape your backend receives: `{tool_call_id, client_id, output | failure, summary}`. A completed result always has an `output` key, even when it's `null`, so it can't be read as a failure. - `run` receives the call as an `AIToolCallPart`, with its `id`, `name` and `arguments`. The model can send values outside your schema, so validate the arguments before acting on them. A call can also reach the device without its arguments, for example when the backend left out arguments too large to copy onto the step, so check for missing `arguments`, as `OpenScreenTool` does. - `instructions` give the agent extra guidance on when and how to use the tool, apart from the `description` the model reads. - A tool that only reads data and changes nothing, such as one that loads the user's calendar, can say so with `isReadOnly: true` on its `AIToolDefinition`. A reply's [generative UI](https://getstream.io/chat/docs/sdk/flutter/ai-components/generative-ui/#run-your-apps-tools) runs a tool by itself, such as to load data while it shows, only when it's read-only. Any other tool runs from it only when the user taps something that runs it. The flag stays on the device: it isn't part of a tool's registration payload. - `showExternalSourcesIndicator: true` asks the agent to report that it checks external sources while the tool runs (`false` by default), for a tool that consults something remote. The flag goes to your backend with the tool's registration, and this package never reads it. An agent that honors it reports the state as `AITypingState.checkingSources`, which an `AITypingIndicatorView` shows, as described in [Showing the AI indicator](https://getstream.io/chat/docs/sdk/flutter/ai-components/stream-chat-integration/#showing-the-ai-indicator). The summary and the failure reason are shown on the step to every channel member, so keep private data out of them: a summary like "Shared approximate location" rather than the coordinates. A tool that reads the user's location returns it like this: ```dart class LocationTool extends AIResultClientTool { const LocationTool(); @override AIToolDefinition get definition => const AIToolDefinition( name: 'getLocation', description: "Read the user's approximate location", instructions: 'Use getLocation when the answer depends on where the user is.', ); @override Future run(AIToolCallPart call) async { final position = await locator.approximatePosition(); if (position == null) { return const AIClientToolResult.failed('Location not shared'); } return AIClientToolResult.completed( {'lat': position.lat, 'lng': position.lng}, // Every channel member sees the summary. summary: 'Shared approximate location', ); } } ``` `locator` stands for whatever your app uses to read the location. A tool that takes no arguments omits `parameters`, which defaults to an empty object schema. A tool that only acts in your app, and has nothing to tell the agent, can implement `AIClientTool` directly. It has two members: the `definition`, and `List handleInvocation(AIToolInvocation invocation)`, which runs when the model calls it. `AIResultClientTool` implements `handleInvocation` for you, on top of `run`. `handleInvocation` returns a list of `AIToolAction`s (callbacks returning `FutureOr`) instead of performing the work itself. That way the decision lives in the tool, and the effect lives in your widget tree. The actions run in order, and one that fails doesn't stop the ones after it. An action must complete on its own: awaiting a dialog that only completes when it's dismissed holds up every action after it. This tool shows a greeting when the user asks to be greeted: ```dart class GreetUserTool implements AIClientTool { const GreetUserTool({required this.onGreet}); final VoidCallback onGreet; @override AIToolDefinition get definition => const AIToolDefinition( name: 'greetUser', description: 'Display a native greeting to the user', instructions: 'Use the greetUser tool when the user asks to be ' 'greeted. The tool shows a greeting alert in the Flutter app.', ); @override List handleInvocation(AIToolInvocation invocation) => [onGreet]; } ``` `onGreet` is where your widget tree reacts, for example by calling `showDialog` with an `AlertDialog` that says hello. When the runner runs a plain `AIClientTool` like this one, the call completes with no output once its actions have run. A plain tool reads its arguments from `invocation.args`: ```dart class SetThemeModeTool implements AIClientTool { const SetThemeModeTool({required this.onChange}); final ValueChanged onChange; @override AIToolDefinition get definition => const AIToolDefinition( name: 'setThemeMode', description: 'Switch the app between its light and dark theme', instructions: 'Use setThemeMode when the user asks for dark mode, ' 'light mode, or to change how the app looks.', parameters: { 'type': 'object', 'properties': { 'mode': { 'type': 'string', 'enum': ['light', 'dark', 'system'], 'description': 'The theme the app should switch to.', }, }, 'required': ['mode'], 'additionalProperties': false, }, ); @override List handleInvocation(AIToolInvocation invocation) { final mode = ThemeMode.values.asNameMap()[invocation.args['mode']]; // Outside the schema's enum: better to do nothing than to guess. if (mode == null) return const []; return [() => onChange(mode)]; } } ``` When the runner runs it, an unknown mode completes the call with no output, so the agent isn't told the value was rejected. A tool whose agent should hear about it extends `AIResultClientTool`, as `OpenScreenTool` does. A plain tool would read missing arguments as empty, so the runner doesn't run one whose schema lists `required` arguments when its call arrives without them. The call fails with "This call's arguments didn't reach the device" instead. A plain tool whose schema requires nothing still runs, with empty `args`. An `AIResultClientTool` always runs. ### Run the calls Give your tools to an `AIClientToolRunner`. It knows who is signed in and which install it runs on, so it only runs the calls addressed to this device: ```dart final runner = AIClientToolRunner( identity: AIClientIdentity(userId: currentUser.id, clientId: installId), tools: [ GreetUserTool(onGreet: showGreeting), SetThemeModeTool(onChange: setThemeMode), OpenScreenTool(onOpen: openScreen), const LocationTool(), ], send: sendToolResult, ); ``` `AIClientIdentity` names the signed-in user (`userId`, your chat client's current user) and this install (`clientId`). `await AIClientIdentity.installId()` gives you an install id: the package creates it on first use, stores it on the device, and returns the same one in later launches. It lasts until the app's data is cleared, or the app is uninstalled on Android and iOS. On Android, iOS and macOS it is kept out of backups, so a device restored from one gets its own id. If your app already keeps an id per install, you can use that instead. Read it once per sign-in, before you create the runner, and use that same value for the [approver](#tool-approvals) and as the `client_id` of every message, so they always agree. A call only runs on the install it names, so a runner without a `clientId` never runs a tool, and says so in the console of a debug build. `sendToolResult` delivers each result to your backend, as shown below. The tools are fixed when the runner is created. When two tools share a name, the last one wins. Keep one runner per conversation: it remembers which calls it already ran, so a new runner could run a call again. When a conversation opens, send `runner.registrations` to your backend, so the agent can offer the tools to the model. Each registration, a map ready for `jsonEncode`, carries the tool's `name`, `description`, `instructions` (left out when the tool has none), `parameters` (its JSON schema) and `showExternalSourcesIndicator`. `runner.toolNames` lists their names. The keys are camelCase, so check that your backend expects the same casing: a mismatch fails quietly, as a tool that never fires. The payloads are plain maps, so remapping keys takes a couple of lines. A call with `executor: client` and `status: awaiting_client` asks a device to run a tool. It names the user (`target_user_id`) and the install (`target_client_id`), which your backend copies from the custom data (`client_id`) of the user's message, so send the install's identifier with every message: ```dart ChatComposer( controller: controller, onSendPressed: (text, selectedOption, attachments) => channel.sendMessage( Message(text: text, extraData: {'client_id': installId}), ), onStopPressed: () => channel.stopAIResponse(), ); ``` With `StreamMessageComposer`, add it in `preMessageSending`: ```dart StreamMessageComposer( preMessageSending: (message) => message.copyWith( extraData: {...message.extraData, 'client_id': installId}, ), ); ``` Then hand the runner each reply's steps whenever they change. This class does it for every message of a channel: ```dart class ClientToolSession { ClientToolSession({required Channel channel, required this.runner}) { _subscription = channel.state!.messagesStream.listen(_onMessages); } final AIClientToolRunner runner; late final StreamSubscription> _subscription; void _onMessages(List messages) { for (final message in messages) { if (message.attachments.isEmpty) continue; final parts = AIMessagePart.parseAll([ for (final attachment in message.attachments) if (attachment.rawType case final type?) (type: type, payload: {'id': attachment.id, ...attachment.extraData}), ]); unawaited(runner.handle(parts)); } } Future dispose() async { await _subscription.cancel(); runner.close(); // Calls already running still finish and send their results. await runner.whenIdle(); } } ``` `AIMessagePart.parseAll` decodes the reply's steps, as shown in [Show the steps of a reply](#show-the-steps-of-a-reply). Without a step's `id`, the runner can't tell one update of a call from the next, and can run the tool again. `handle` is safe to call on every update, and never completes with an error. It runs a call only when the call awaits this user and this install (`AIClientIdentity.canRun`) and the runner has a tool with its name, and it runs each call only once, by `id`. A call for a tool the runner doesn't have is left alone, so several runners can share one `AIClientIdentity`, each with its own tools. The flip side: when no runner has the tool, nothing answers the call and the agent keeps waiting. That can happen when an older version of your app registered a tool this build dropped. A runner that holds every tool of your app can set `failUnknownTools: true`, to answer such a call once with a failure, and report it through `onToolError`. The runner hands each result to `send`, which delivers it to your backend: ```dart Future sendToolResult( AIToolCallPart call, AIClientToolResult result, ) async { final response = await http.post( Uri.parse('$myBackend/tool-results'), headers: {'content-type': 'application/json'}, body: jsonEncode(result.toJson(toolCallId: call.id, clientId: installId)), ); // Throw when the backend didn't accept the result, so the runner tries again. if (response.statusCode >= 300) { throw Exception('Tool result rejected: ${response.statusCode}'); } } ``` `send` completes once your backend has the result, and throws when it wasn't accepted. If it throws, the result is offered again (up to `maxAttempts`, 3 by default), without running the tool again. Your backend updates the step and gives the `output` or the `failure` to the model; the step then shows the `summary` or the `failure`. A retried `send` can post the same result twice, so make the endpoint idempotent on `tool_call_id`. It should also accept a result only from the targeted user and install, only while the call waits, and only once. Every channel member can see a call's arguments and summary, so keep private data out of them. The runner answers every call it takes on, even when the tool fails, and retries a result your backend didn't accept. These options tune that: ```dart final runner = AIClientToolRunner( identity: identity, tools: tools, send: sendToolResult, runTimeout: const Duration(seconds: 30), sendTimeout: const Duration(seconds: 10), maxAttempts: 5, retryDelay: const Duration(seconds: 2), failureMessage: (call, error) => error is TimeoutException ? 'The device took too long to answer' : 'The tool failed on the device', onToolError: (call, error, stack) => showSnackBar("Couldn't run ${call.displayName}"), onDeliveryError: (call, result, error, stack) => showSnackBar("Couldn't send the result of ${call.displayName}"), ); ``` - A tool that throws, or runs longer than `runTimeout`, is answered with a generic failure that names the tool. `runTimeout` defaults to no limit. The tool keeps running past it, and a failure it ends with later is still reported. - `failureMessage` words the failure the agent is told, from the call and the error. Once set, it words every failure the runner makes, including a missing-arguments call and a result that can't be encoded. An `AIClientToolResult.failed` your tool returns keeps its own reason. If it throws, a generic failure is used. - `onToolError` is called when a tool fails: it threw, timed out, or its call arrived without the arguments it requires. Use it to tell the user something didn't work. The error goes to `FlutterError.onError` too, and never to the agent. - `send` is tried up to `maxAttempts` times (3 by default). The wait before the first retry is `retryDelay` (1 second by default), doubled for each retry after it, up to `AIClientToolRunner.maxRetryDelay` (30 seconds). `sendTimeout` bounds each attempt, and defaults to no limit. - When the attempts run out, `onDeliveryError` is called, once until that result is sent, and `FlutterError.onError` hears about it too. The result is kept: a later `handle` that still sees the call waiting sends it again without rerunning the tool, and `runner.retryUndelivered()` sends it again on demand, for example once the device is back online. `hasUndelivered` tells you whether a result is waiting to be sent. A new runner only takes on calls that still wait for this install, with status `awaiting_client` and addressed to its identity's `userId` and `clientId`, so a call whose result has reached the backend is never run again. A result the old runner was still sending, or had given up on, is lost with it, and a new runner runs that call once more. So a new runner is safe once the old one has nothing in flight and nothing undelivered. When the conversation closes, `close()` stops the runner from taking on new calls. Calls already running still finish and send their results, retries included, so `send` must not depend on a disposed widget. `whenIdle()` completes once they're done, and `hasPendingWork` tells you whether any are left. `ClientToolSession.dispose` above does both. A call whose tool asks the user first reaches the device only once they allowed it, as described in [Tool approvals](#tool-approvals). ## Tool approvals Some tool calls should wait for the user: the agent wants their location, or to send an email on their behalf. The agent's backend writes such a call with status `awaiting_approval`, addresses it to that user (`target_user_id`, and `target_client_id` for a client tool), and adds the question as its `approval`: ```json { "type": "ai_tool_call", "id": "toolu_01A", "name": "get_location", "status": "awaiting_approval", "executor": "client", "target_user_id": "u_123", "target_client_id": "flutter-7F3A9C2E", "approval": { "title": "Share your location?", "message": "Only your city is shared.", "reason": "to check the local weather", "allow_title": "Share location", "decline_title": "Don't share" } } ``` `AIToolApprovalView` asks the question under the call. It shows only to that user, on the install the call names (or on any of their devices, when the call names none, as for a server tool), and only while the call waits. Give it an `AIToolApprover`, which says who is signed in on this device (an `AIClientIdentity`) and sends their answer to your backend through `decide`: ```dart class _AssistantScreenState extends State { late final approver = AIToolApprover( identity: AIClientIdentity(userId: currentUser.id, clientId: installId), // Throw to let the user answer again. decide: (call, decision) => backend.answerToolCall(call.id, decision), ); @override void dispose() { approver.dispose(); super.dispose(); } @override Widget build(BuildContext context) { return AIMessagePartsView(parts: widget.parts, approver: approver); } } ``` `installId` is this install's identifier, from `await AIClientIdentity.installId()`, as described in [Run the calls](#run-the-calls). Without a `clientId`, the user can still answer calls that name no install. `decision` is an `AIToolApprovalDecision`, `allowed` or `declined`, a `String` extension type you can send as is. The approver remembers where each answer is, so a question scrolled out of a lazy list and back still shows its answer as sent. Create it once, in a `State` or a service rather than in `build`, and `dispose` it when you're done. Its identity is fixed, so make a new approver when the signed-in user changes. With [Stream Agents](https://getstream.io/agents/docs/quickstart/?client=flutter), a session tool that asks first waits in `session.approvals`, so `decide` answers through the session, with `session.decide(call.id, allowed: decision == AIToolApprovalDecision.allowed)`, as the [Quickstart](https://getstream.io/chat/docs/sdk/flutter/ai-components/) does. Passing the approver to `AIMessagePartsView` asks the question under every tool call of the reply that waits for this user. The [conversation view](https://getstream.io/chat/docs/sdk/flutter/ai-components/conversation-view/) takes an `approver` as well. To ask under a call you show yourself, put an `AIToolApprovalView` under it. It's an empty box while there's nothing to ask, so it can sit under every tool call: ```dart Column( crossAxisAlignment: CrossAxisAlignment.start, spacing: 8, children: [ AIToolCallView(call: call), AIToolApprovalView(call: call, approver: approver), ], ) ``` A call that `partBuilder` or `toolCallBuilder` draws with your own widget gets no question or sign-in from `AIMessagePartsView`, so add an `AIToolApprovalView` to your widget then, and an [`AIToolAuthorizationView`](#tool-sign-ins) if the call can wait for a sign-in. A call they hide with `AIMessagePartsView.hidden` still gets its question and sign-in, without its row. Where the answer is, `AIToolApprovalState`, is one of: - `AIToolApprovalIdle` - nothing sent yet, and the buttons are enabled. - `AIToolApprovalSending` - the answer is on its way, and the buttons are disabled. - `AIToolApprovalSent` - your backend has the answer, and the buttons stay disabled until the call's step changes and the question goes away. - `AIToolApprovalFailed` - `decide` threw. The card says the answer couldn't be sent, and the user can answer again. `decide` must not hang, as the answer stays sending until it completes. To bound it, pass a `sentTimeout`: past it, an answer still sending, or sent while the call still waits, turns failed. Your backend holds the call until it gets the answer, and accepts one only from the targeted user (and install, for a client tool), only while the call waits, and only once. It then updates the step: - **Allowed:** `approval.decision` becomes `allowed` and the call goes on. A client tool's call moves to `awaiting_client`, so the device's `AIClientToolRunner` runs it. - **Declined:** the call becomes `cancelled`, with `approval.decision` set to `declined`, and never runs. `isDeclined` tells it apart from other cancelled calls, and `AIToolCallView` shows it as "Declined". The question is visible to every channel member, so keep private data out of it. To ask in your own design, pass a `builder`. It gets the call's `approval` (`null` when the call waits without one, so ask in your own words then), the `AIToolApprovalState`, and an `answer` function that does nothing while an answer is on its way or sent: ```dart AIToolApprovalView( call: call, approver: approver, builder: (context, approval, state, answer) => MyApprovalCard( title: approval?.title ?? 'Allow this?', isBusy: state.isBusy, failed: state.hasFailed, onAllow: () => answer(AIToolApprovalDecision.allowed), onDecline: () => answer(AIToolApprovalDecision.declined), ), ) ``` `AIToolApprovalCard` is the default design, which you can also wrap inside a `builder`. Without `title`, `allow_title` and `decline_title`, it asks with a question naming the tool, and its buttons read "Allow" and "Don't Allow". To replace the card everywhere, including in `AIMessagePartsView`, register a `toolApproval` builder with `streamAIComponentBuilders`, whose `AIToolApprovalProps` carry the `call`, its `approval`, the `toolName`, the `state`, `onAnswer` and the `theme` the view was given. ## Tool sign-ins Some tools need the user's own account at a provider, such as "What are my most important Linear tickets?" before the user has connected Linear. The agent's backend then holds the call with status `awaiting_authorization`, addresses it to that user (`target_user_id`), and adds the sign-in as its `authorization`: ```json { "type": "ai_tool_call", "id": "call-1", "name": "linear__list_issues", "display_title": "Using Linear", "status": "awaiting_authorization", "target_user_id": "u_123", "authorization": { "type": "oauth", "provider": "linear", "provider_name": "Linear", "title": "Connect Linear", "message": "You sign in at Linear, and the agent never sees your password.", "url": "https://agent.example.com/connectors/oauth/launch/linear", "handoff_token": "h_7f3a9c", "expires_at": "2026-10-21T21:42:19.834Z" } } ``` `AIToolAuthorizationView` asks the user to sign in under the call, with **Connect** and **Not now**. It shows only to that user (on any of their devices), only while the call waits, only for an `oauth` sign-in, and only when the approver has an `AIToolAuthorizer`. The package opens no browser itself, so it adds no browser dependency: the authorizer's `connect` gets the sign-in page and your app opens it, and its `decline` sends the user's "Not now" to your backend: ```dart late final approver = AIToolApprover( identity: AIClientIdentity(userId: currentUser.id, clientId: installId), decide: (call, decision) => backend.answerToolCall(call.id, decision), authorizer: AIToolAuthorizer( // Opens the sign-in page, and completes with whether the user connected. connect: (call, url) => signIn(url), // "Not now": your backend cancels the call. Throw to let the user answer again. decline: (call) => backend.declineSignIn(call.id), ), ); ``` ```dart Column( crossAxisAlignment: CrossAxisAlignment.start, spacing: 8, children: [ AIToolCallView(call: call, userId: currentUser.id), AIToolAuthorizationView(call: call, approver: approver), ], ) ``` `AIMessagePartsView` and the [conversation view](https://getstream.io/chat/docs/sdk/flutter/ai-components/conversation-view/#tool-calls) show the sign-in under every tool call that waits for it, so an approver with an authorizer is all they need. For a call you show yourself, put an `AIToolAuthorizationView(call: call, approver: approver)` under it, as with approvals. To tell the user the call waits for them ("Waiting for you to sign in"), pass `userId` to that call's `AIToolCallView`, as `AIMessagePartsView` does when it offers the sign-in. Without an authorizer, no sign-in is offered on the device, and the call's row reads "Waiting for sign-in" for everyone. When the user taps **Connect**: 1. `connect` gets the authorization's `launchUrl`: its `url`, with the handoff token in the fragment, which no server sees. Your app opens it in a browser. 2. The user signs in at the provider, which hands the grant to the agent's backend, never to the app. The step holds no credential. 3. Your backend stores the grant, updates the step, and sends the page back to the return URL it gives its sign-ins, with `status=connected` in its query. Complete `connect` with `true` then. The card shows "Connecting..." until the step changes, and offers **Connect** again if it still hasn't after 30 seconds. 4. The call goes back to `running`, with `authorization.status` set to `connected`. The sign-in hides, and the call carries on in the same reply. Complete `connect` with `false` when the user came back without connecting, so they can connect again, and throw when the page couldn't be opened, so the card says so. `connect` must not hang: until it completes, the user can't open the page again. Only a sign-in page on `https` with a host is offered, or one on `http` on this device itself (`localhost` or a loopback address), for an agent backend you run locally. The handoff token travels in the URL, which a cleartext page elsewhere would expose, so an `authorization` with any other `url`, or without a `title`, is ignored. Give the sign-in a return URL in your app's own scheme, in reverse-domain form such as `com.example.app:/connected`. With [`flutter_web_auth_2`](https://pub.dev/packages/flutter_web_auth_2), which opens the page in an in-app browser session and catches the return URL: ```dart Future signIn(Uri url) async { try { final back = await FlutterWebAuth2.authenticate( url: url.toString(), callbackUrlScheme: 'com.example.app', ); return Uri.parse(back).queryParameters['status'] == 'connected'; } on PlatformException catch (e) { if (e.code == 'CANCELED') return false; // closed without connecting rethrow; // the card says the sign-in couldn't be opened } } ``` iOS needs nothing more. On Android, register the package's callback activity for your scheme in `AndroidManifest.xml`: ```xml ``` Any other way of opening the page works too, such as [`url_launcher`](https://pub.dev/packages/url_launcher) with a deep link back into your app: complete `connect` once your app receives the return URL, or once the user comes back without it. **Not now** calls the authorizer's `decline`. Complete it once your backend has the "Not now", and throw when it couldn't be sent, so the user can answer again. A `decline` that returns before the send finishes, such as one that doesn't await it, never shows a failed "Not now". Your backend ends the call: it becomes `cancelled`, with `authorization.status` set to `declined`, and never runs. `isDeclined` is `true` for it, and `AIToolCallView` shows it as "Declined". **Not now** stays enabled while the sign-in page is open, so a sign-in that never comes back still leaves the user a way out. The approver's `sentTimeout` bounds sending the "Not now", but never `connect`, since signing in can take minutes. Once the sign-in's `expires_at` passes, the card says it expired and disables **Connect**, leaving **Not now**. A sign-in can also end as `expired` or `failed`, which cancels the call too. To ask in your own design, pass a `builder` to `AIToolAuthorizationView`. It gets the `AIToolAuthorization`, where the user's answer is (`AIToolAuthorizationState`, with `isConnecting`, `isDeclining`, `connectFailed`, `declineFailed`, `isExpired`, and `isBusy` and `canConnect`), and `connect` and `decline` callbacks: ```dart AIToolAuthorizationView( call: call, approver: approver, builder: (context, authorization, state, connect, decline) => MySignInCard( title: authorization.title, isBusy: state.isBusy, onConnect: connect, onNotNow: decline, ), ) ``` To replace the card everywhere, register a `toolAuthorization` builder with `streamAIComponentBuilders`, whose `AIToolAuthorizationProps` carry the same, as `onConnect` and `onDecline`, plus the `theme`: ```dart Widget signInCard(BuildContext context, AIToolAuthorizationProps props) => MySignInCard( title: props.authorization.title, canConnect: props.state.canConnect, isConnecting: props.state.isConnecting, onConnect: props.onConnect, onNotNow: props.onDecline, ); void main() => runApp( StreamComponentFactory( builders: StreamComponentBuilders( extensions: streamAIComponentBuilders(toolAuthorization: signInCard), ), child: const MyApp(), ), ); ``` `AIToolAuthorizationCard` is the default design, and uses the approval card's theme. ## Step format The steps are custom attachments, with their fields at the top level of the attachment. Every step has: | Field | Description | | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | The kind of step: `ai_reasoning`, `ai_tool_call`, or a kind of your own starting with `ai_`. | | `id` | The step's stable identity. Tool calls use the model provider's tool-call ID. A step without one is named after its type and position (`-`), which is fine for a message you build yourself, but with Stream Chat every update gets a new random `Attachment.id`, so set it. | | `v` | The step's format version, a positive whole number, `1` by default. It changes only when a kind changes incompatibly. A step in a newer version has no typed view. | A round of reasoning (`ai_reasoning`) also has: | Field | Description | | ------------- | ----------------------------------------------------------------------------- | | `status` | `streaming` while the model thinks, or `completed` (the default). | | `summary` | A one-line summary of the reasoning, once it's done. | | `preview` | A capped excerpt: the latest thoughts while streaming, the opening once done. | | `duration_ms` | How long the model thought, in milliseconds. | A tool call (`ai_tool_call`) also has: | Field | Description | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name` | The tool's name. | | `display_title` | What the call is doing, in words for people, such as "Checking your location". | | `status` | `running` (the default), `awaiting_approval`, `awaiting_authorization`, `awaiting_client`, `completed`, `failed` or `cancelled`. | | `executor` | Who runs the tool: `server` (the default) or `client`. | | `target_user_id` | The user whose device must run a client tool, or who must approve the call or sign in. | | `target_client_id` | The install that must run a client tool, copied from the `client_id` in the custom data of the user's message. | | `arguments` | The call's arguments as a JSON object (or a string holding one), present for client tools, which need them to run. `AIToolCallPart.arguments` holds them as a map. | | `summary` | A short, shareable outcome, such as "Found your location". | | `duration_ms` | How long the call took, in milliseconds. | | `approval` | The question the call asks before it runs: `title`, `message`, `reason`, `allow_title`, `decline_title`, and the user's `decision`. | | `authorization` | The sign-in the call waits for: `type` (`oauth`), `provider`, `provider_name`, `title`, `message`, `url`, `handoff_token`, `expires_at`, and how it ended (`status`: `connected`, `declined`, `expired` or `failed`). | `expires_at` is an RFC 3339 time with a `Z` or an offset, such as `2026-10-21T21:42:19.834Z`. A call that waits for approval without an `approval` still asks, in the card's own words. Every channel member can see the steps, their arguments and their summaries, so keep private data out of them. ## Appearance The steps take their colors from the Stream theme, `StreamTheme`. To change them, register an `AITheme` extension, as described in [Theming](https://getstream.io/chat/docs/sdk/flutter/ai-components/#theming). Every field is optional, and one you leave out follows the ambient theme, in light and dark mode alike: ```dart MaterialApp( theme: ThemeData( extensions: const [ AITheme( reasoningTheme: ReasoningThemeData(shimmerColor: Color(0xFF005FFF), maxExpandedHeight: 320), toolCallTheme: ToolCallThemeData(accentColor: Color(0xFF005FFF), iconSize: 18), toolApprovalTheme: ToolApprovalThemeData(accentColor: Color(0xFF005FFF)), ), ], ), home: const AssistantScreen(), ) ``` - `ReasoningThemeData`, for `StreamingReasoningView`: `titleStyle` (the header), `textStyle` (the reasoning and the summary), `footnoteStyle`, `shimmerColor` (the header's shimmer, the rule while the model thinks, and links), `ruleColor` (the rule once it's done), `iconColor` and `maxExpandedHeight` (260 by default). - `ToolCallThemeData`, for `AIToolCallView`: `titleStyle`, `detailStyle` (the line under the title and the duration), `accentColor` (the spinner and the icons of a call that waits), `successColor`, `failureColor`, `iconColor` (the cross of a cancelled call) and `iconSize` (16 by default). `AIUnsupportedPartView` follows it too. - `ToolApprovalThemeData`, for `AIToolApprovalCard` and `AIToolAuthorizationCard`: `titleStyle`, `messageStyle`, `backgroundColor`, `borderColor` (the card's border and the outline of the decline or **Not now** button), `accentColor` (the buttons and the spinner) and `failureStyle`. To narrow one to part of the tree, wrap it in `ReasoningTheme`, `ToolCallTheme` or `ToolApprovalTheme` (or their `merge`, to add to an enclosing one rather than replace it): ```dart ToolApprovalTheme( data: const ToolApprovalThemeData(backgroundColor: Colors.white), child: AIMessagePartsView(parts: parts, approver: approver), ) ``` To style one view, pass its `theme`: `StreamingReasoningView`, `AIToolCallView`, `AIToolApprovalView`, `AIToolApprovalCard`, `AIToolAuthorizationView` and `AIToolAuthorizationCard` all take one. Its text styles merge over the themed ones rather than replacing them. The steps' texts, such as "Thinking... 7s", "Waiting for approval", the default "Allow" and "Don't Allow" buttons, or the sign-in's "Connect" and "Not now", are members of `AITranslations`, so you can translate them as described in [Localization](https://getstream.io/chat/docs/sdk/flutter/ai-components/#localization): `unsupportedMessagePart`, and the members whose names start with `reasoning`, `toolCall`, `toolApproval` and `toolAuthorization`. `DefaultAITranslations.reasoningDuration` spells the durations of both reasoning headers, and `toolCallDuration` falls back to it from a second up. --- For the most recent version of this documentation, visit [https://getstream.io/chat/docs/sdk/flutter/ai-components/reasoning-and-tool-calls/](https://getstream.io/chat/docs/sdk/flutter/ai-components/reasoning-and-tool-calls/).