Client Side Tools

Client-side tools

The Stream Chat AI components allow you to execute tools on the client. What that means, is that when the agent determines that a tool should be called, it's also able to delegate that task to the client, in this case the mobile device.

This opens up a lot of possibilities - for example your app can access the user's device data (calendar access, user location, health data, etc). You can also execute custom UI, like showing screens, forms, popups, alerts and similar. You can also potentially call app intents to execute other app's actions.

Info:

The client tool APIs are part of StreamChatAI, available from version 5.13.0 of the Stream Chat Swift SDK. They replace ClientTool and ClientToolRegistry from the standalone stream-chat-swift-ai package, as described in Migrating from stream-chat-swift-ai.

Client tools are described like Model Context Protocol (MCP) tools: a name, a description and a JSON schema of their input. StreamChatAI doesn't depend on the MCP SDK, though: AIClientToolDefinition reads and writes the same JSON as the MCP SDK's Tool.

Running client tools takes three steps:

  1. Conform each tool to AIClientTool.
  2. Give your tools to an AIClientToolRunner, and register them with your backend, so the agent knows it can call them.
  3. Run the calls the agent sends to the device. An agent that writes its tool calls on the reply as ai_tool_call steps reaches the runner directly. The Stream Chat AI SDK and the Stream Chat LangChain SDK send a custom_client_tool_invocation event instead.

Create your own client tool

In order to create your own client tool, you need to implement the AIClientTool protocol:

@MainActor
public protocol AIClientTool: AnyObject {
    var definition: AIClientToolDefinition { get }
    var instructions: String? { get }
    var showExternalSourcesIndicator: Bool { get }

    func run(_ call: AIToolCallPart) async -> AIClientToolResult
}

Each tool provides a definition describing its name, description and input schema. Optional instructions tell the model when to use the tool, and showExternalSourcesIndicator says whether the app shows that the tool reaches outside the conversation while it runs. Both have defaults (nil and false). When the agent calls the tool, run(_:) does the work and returns an AIClientToolResult. A tool's name is its definition's name.

Greet Tool

In this guide, we will create an example greet tool, that will show a "greet" alert when the user asks to be greeted.

@MainActor
final class GreetClientTool: AIClientTool {
    let definition = AIClientToolDefinition(
        name: "greetUser",
        description: "Display a native greeting to the user",
        inputSchema: [
            "type": "object",
            "properties": [:],
            "required": [],
            "additionalProperties": false
        ]
    )

    let instructions: String? =
        "Use the greetUser tool when the user asks to be greeted. The tool shows a greeting alert in the iOS app."

    func run(_ call: AIToolCallPart) async -> AIClientToolResult {
        ClientToolActionHandler.shared.presentAlert(
            ClientToolAlert(
                title: "Greetings!",
                message: "Hello there! The assistant asked me to greet you."
            )
        )
        return .completed(["greeted": true], summary: "Greeted the user")
    }
}

The input schema is a RawJSON, StreamCore's JSON value, which you can write with Swift literals, as above, or with its cases (.dictionary, .string, .array and so on). StreamChatAI re-exports StreamCore, so it needs no other import.

Declare instructions as a String?, as above. A property of type String doesn't fulfill the requirement, so the tool would silently send the default, nil.

run(_:) runs on the main actor, so it can update your UI directly. The greet tool shows an alert through a small observable object:

@MainActor
final class ClientToolActionHandler: ObservableObject {
    static let shared = ClientToolActionHandler()

    @Published var presentedAlert: ClientToolAlert?

    private init() {}

    func presentAlert(_ alert: ClientToolAlert) {
        presentedAlert = alert
    }
}

struct ClientToolAlert: Identifiable {
    let id = UUID()
    let title: String
    let message: String
}

When the presentedAlert value changes, you can use it in a SwiftUI view to render the alert:

@ObservedObject var toolActionHandler = ClientToolActionHandler.shared

var body: some View {
    /* ... chat UI ... */
    .alert(item: $toolActionHandler.presentedAlert) { alert in
        Alert(
            title: Text(alert.title),
            message: Text(alert.message),
            dismissButton: .default(Text("OK"))
        )
    }
}

Different tools can expose additional published properties (banners, navigation triggers, etc.) through a similar handler.

Tool results

AIClientToolResult carries the output for the model (a JSON object), a short summary of the outcome, or a failure, such as "Location not shared". Create one with .completed(_:summary:), which encodes any Encodable value as the output, or with .failed(_:).

For tools that take arguments, decode the call's arguments into a type of your own with call.decodeArguments(as:):

@MainActor
final class OpenScreenTool: AIClientTool {
    struct Arguments: Decodable {
        let screen: String
    }

    let definition = AIClientToolDefinition(
        name: "openScreen",
        description: "Open a screen of the app",
        inputSchema: [
            "type": "object",
            "properties": [
                "screen": ["type": "string", "enum": ["settings", "profile"]]
            ],
            "required": ["screen"]
        ]
    )

    func run(_ call: AIToolCallPart) async -> AIClientToolResult {
        guard let arguments = try? call.decodeArguments(as: Arguments.self) else {
            return .failed("The screen to open is missing")
        }
        AppRouter.shared.open(arguments.screen)
        return .completed(["opened": arguments.screen], summary: "Opened \(arguments.screen)")
    }
}

Arguments and summaries can be visible to every channel member, so keep private data out of them: a summary like "Shared approximate location" rather than the coordinates.

Tools defined with the MCP SDK

If you already define your tools with the Swift MCP SDK, you don't need to rewrite them. AIClientToolDefinition reads the same name, description and inputSchema keys as an MCP Tool, so you can convert one:

let definition = try AIClientToolDefinition(encoding: mcpTool)

Register the tools

Give your tools to an AIClientToolRunner. It knows who is signed in and which install it runs on, so it only runs the calls addressed to this device:

let runner = AIClientToolRunner(
    userID: currentUserID,
    clientID: AIClientIdentity.installID,
    tools: [GreetClientTool(), OpenScreenTool()]
)

AIClientIdentity.installID is a stable identifier for this install, created on first use. Create one runner and keep it while the user is signed in: it remembers which calls it already ran, so a new runner could run a call again.

Registering the tools server-side

You also need to notify your server-side agent about the client tools you are going to use. Our server-side integrations with the AI SDK and Langchain already expose a method that lets you register tools.

You need to expose your endpoint for registration with a code similar to this:

app.post('/register-tools', (req, res) => {
  const { channel_id, tools } = req.body ?? {};

  if (typeof channel_id !== 'string' || channel_id.trim().length === 0) {
    res.status(400).json({ error: 'Missing or invalid channel_id' });
    return;
  }

  if (!Array.isArray(tools)) {
    res.status(400).json({ error: 'Missing or invalid tools array' });
    return;
  }

  const channelIdNormalized = normalizeChannelId(channel_id);
  if (!channelIdNormalized) {
    res.status(400).json({ error: 'Invalid channel_id' });
    return;
  }

  const sanitizedTools: ClientToolDefinition[] = [];
  const invalidTools: string[] = [];

  tools.forEach((rawTool, index) => {
    const tool = rawTool ?? {};
    const rawName = typeof tool.name === 'string' ? tool.name.trim() : '';
    const rawDescription =
      typeof tool.description === 'string' ? tool.description.trim() : '';

    if (!rawName || !rawDescription) {
      invalidTools.push(
        typeof tool.name === 'string'
          ? tool.name
          : `tool_${index.toString().padStart(2, '0')}`,
      );
      return;
    }

    const instructions =
      typeof tool.instructions === 'string' && tool.instructions.trim().length > 0
        ? tool.instructions.trim()
        : undefined;

    const parameters = isPlainObject(tool.parameters)
      ? (JSON.parse(JSON.stringify(tool.parameters)) as ClientToolDefinition['parameters'])
      : undefined;

    let showExternalSourcesIndicator: boolean | undefined;
    if (typeof tool.showExternalSourcesIndicator === 'boolean') {
      showExternalSourcesIndicator = tool.showExternalSourcesIndicator;
    } else if (typeof tool.show_external_sources_indicator === 'boolean') {
      showExternalSourcesIndicator = tool.show_external_sources_indicator;
    }

    sanitizedTools.push({
      name: rawName,
      description: rawDescription,
      instructions,
      parameters,
      showExternalSourcesIndicator,
    });
  });

  if (!sanitizedTools.length && tools.length > 0) {
    res.status(400).json({
      error: 'No valid tools provided',
      invalid_tools: invalidTools,
    });
    return;
  }

  agentManager.registerClientTools(channelIdNormalized, sanitizedTools);

  const responsePayload: Record<string, unknown> = {
    message: 'Client tools registered',
    channel_id: channelIdNormalized,
    count: sanitizedTools.length,
  };
  if (invalidTools.length) {
    responsePayload.invalid_tools = invalidTools;
  }

  res.json(responsePayload);
});

Client-side, you can call this endpoint with the following code:

func registerTools(channelId: String, tools: [AIClientToolRegistration]) async throws {
    guard !tools.isEmpty else { return }
    try await executePostRequest(
        body: ToolRegistrationRequest(channelId: channelId, tools: tools),
        endpoint: "register-tools"
    )
}

private func executePostRequest<RequestBody: Encodable>(body: RequestBody, endpoint: String) async throws {
    let url = URL(string: "\(baseURL)/\(endpoint)")!
    var request = URLRequest(url: url)
    request.httpMethod = "POST"
    request.setValue("application/json", forHTTPHeaderField: "Content-Type")
    request.httpBody = try jsonEncoder.encode(body)
    _ = try await urlSession.data(for: request)
}

struct ToolRegistrationRequest: Encodable {
    let channelId: String
    let tools: [AIClientToolRegistration]

    enum CodingKeys: String, CodingKey {
        case channelId = "channel_id"
        case tools
    }
}

Then, for example, when the conversation opens, you can perform the registration with the following code:

try await registerTools(channelId: "your_channel_id", tools: runner.registrations)

Each registration carries the tool's name, description (its instructions, when it has no description), instructions, parameters (its input schema) and showExternalSourcesIndicator. It's the same JSON the stream-chat-swift-ai package sent, so an existing endpoint keeps working.

Run tool calls from agent steps

An agent can write each tool call on its reply, as an ai_tool_call step (see Reasoning and Tool Calls). A call with executor: client and status: awaiting_client asks a user's device to run a tool. It names the user (target_user_id) and the install (target_client_id) that should run it, which your backend copies from the custom data (client_id) of that user's message. Send the install's identifier with every message:

channelController.createNewMessage(
    text: messageData.text,
    extraData: ["client_id": .string(AIClientIdentity.installID)]
)

Then hand the runner each AI reply's steps whenever they change, for example from the view that renders the reply:

let parts = AIMessagePart.parts(
    from: message.allAttachments.map { ($0.type.rawValue, $0.payload) }
)

runner.run(parts) { call, result in
    try await backend.send(result, for: call, in: message)
}

The runner runs a call only when it awaits this user and this install, and only once. Your send closure delivers the result to your backend, which updates the step and gives the output to the model; the step then shows the summary or the failure. If send throws, the result is offered again on a later update (up to maxAttempts, 3 by default), without running the tool again. Your backend should still accept a result only from the targeted user and install, only while the call is waiting, and only once.

The steps, with their arguments and summaries, are visible to every channel member.

A call whose tool asks the user first reaches the device only once they allowed it, as described in Tool approvals.

Run tool calls from events

The Stream Chat AI SDK and the Stream Chat LangChain SDK ask the app to run a client tool with a custom_client_tool_invocation event on the channel, carrying the tool's name and its args. Decode the event with a custom event payload:

struct ClientToolInvocationPayload: CustomEventPayload {
    static let eventType = EventType(rawValue: "custom_client_tool_invocation")

    struct Tool: Codable, Hashable {
        let name: String
    }

    let messageId: String?
    let tool: Tool
    let args: RawJSON?

    enum CodingKeys: String, CodingKey {
        case messageId = "message_id"
        case tool
        case args
    }
}

Then listen for it with the client-wide events controller, find the tool by name, and run it with a call built from the event:

@MainActor
final class ClientToolEventHandler: EventsControllerDelegate {
    private let tools: [any AIClientTool]
    private let eventsController: EventsController

    init(chatClient: ChatClient, tools: [any AIClientTool]) {
        self.tools = tools
        eventsController = chatClient.eventsController()
        eventsController.delegate = self
    }

    func eventsController(_ controller: EventsController, didReceiveEvent event: Event) {
        guard
            let event = event as? UnknownChannelEvent,
            let payload = event.payload(ofType: ClientToolInvocationPayload.self),
            let tool = tools.first(where: { $0.name == payload.tool.name })
        else { return }

        let call = AIToolCallPart(
            id: UUID().uuidString,
            name: tool.name,
            status: .awaitingClient,
            executor: .client,
            arguments: try? JSONEncoder().encode(payload.args ?? [:])
        )
        Task {
            _ = await tool.run(call)
        }
    }
}

Keep the handler alive while the user is signed in, as you would the runner. These SDKs don't wait for a result: the tool does its work on the device, and the agent goes on with its reply.