Client Side Tools

The Stream Chat AI components allow you to execute client-side tools. When the agent decides that a tool should be called, it can delegate that task to the client, in this case your Flutter app.

This opens up a lot of possibilities. Your app can open a screen, show a form, popup or alert, switch the theme, or read data from the device, such as the user's location.

A client tool is a side effect, not a function call with a result. The protocol has no channel for reporting a result back to the model, so this API doesn't offer one either.

Create your own tool

To create a tool, implement AIClientTool. It has two members: a definition that describes the tool to the model, and handleInvocation, which runs when the model calls it.

Greet tool

This example tool shows a greeting when the user asks to be greeted:

class GreetUserTool implements AIClientTool {
  const GreetUserTool({required this.onGreet});

  final VoidCallback onGreet;

  @override
  AIToolDefinition get definition => const AIToolDefinition(
        name: 'greetUser',
        description: 'Display a native greeting to the user',
        instructions: 'Use the greetUser tool when the user asks to be '
            'greeted. The tool shows a greeting alert in the Flutter app.',
      );

  @override
  List<AIToolAction> handleInvocation(AIToolInvocation invocation) => [onGreet];
}

The instructions help the model decide when the tool is appropriate. Set showExternalSourcesIndicator: true on the definition for a tool that consults something remote, and the agent will report a "checking external sources" state while it runs.

handleInvocation returns a list of AIToolActions (plain callbacks) instead of performing the work itself. That way the decision lives in the tool, and the effect lives in your widget tree. Each action runs in order.

The greet tool takes no arguments, so it omits parameters.

Tools with arguments

For a tool that takes parameters, describe them with a JSON schema in parameters, and read them from invocation.args:

class SetThemeModeTool implements AIClientTool {
  const SetThemeModeTool({required this.onChange});

  final ValueChanged<ThemeMode> onChange;

  @override
  AIToolDefinition get definition => const AIToolDefinition(
        name: 'setThemeMode',
        description: 'Switch the app between its light and dark theme',
        instructions: 'Use setThemeMode when the user asks for dark mode, '
            'light mode, or to change how the app looks.',
        parameters: {
          'type': 'object',
          'properties': {
            'mode': {
              'type': 'string',
              'enum': ['light', 'dark', 'system'],
              'description': 'The theme the app should switch to.',
            },
          },
          'required': ['mode'],
          'additionalProperties': false,
        },
      );

  @override
  List<AIToolAction> handleInvocation(AIToolInvocation invocation) {
    final mode = ThemeMode.values.asNameMap()[invocation.args['mode']];
    // Outside the schema's enum: better to do nothing than to guess.
    if (mode == null) return const [];
    return [() => onChange(mode)];
  }
}

The model can send values outside your schema, so validate the arguments before acting on them.

Register your tools

Add the tools to an AIToolRegistry, and send the registrations to your backend:

final registry = AIToolRegistry()
  ..register(GreetUserTool(onGreet: showGreeting))
  ..register(SetThemeModeTool(onChange: setThemeMode));

await http.post(
  Uri.parse('$myBackend/register-tools'),
  headers: {'content-type': 'application/json'},
  body: jsonEncode({
    'channel_id': channel.cid,
    'tools': registry.registrationPayloads(),
  }),
);

The /register-tools endpoint is yours. The package ships no HTTP client. Its job is to call registerClientTools(channelId, tools) from the Stream Chat AI SDK, which persists the definitions on the server and re-applies them the next time the channel's agent starts.

registrationPayloads() emits camelCase keys such as showExternalSourcesIndicator. Check that your backend expects the same casing. A mismatch fails quietly, as a tool that never fires. The payloads are plain maps, so remapping keys takes a couple of lines.

Handle invocations

When the model calls a tool, the agent sends a custom_client_tool_invocation event over the chat connection. Listen for it, parse it into an AIToolInvocation, and hand it to the registry:

class ClientToolListener {
  ClientToolListener({required StreamChatClient client, required AIToolRegistry registry})
      : _registry = registry {
    _subscription = client.on(kClientToolInvocationEventType).listen(_onEvent);
  }

  final AIToolRegistry _registry;
  late final StreamSubscription<Event> _subscription;

  Future<void> _onEvent(Event event) async {
    final invocation = AIToolInvocation.tryParse(event.toClientToolPayload());
    if (invocation == null) return;

    final handled = await _registry.dispatch(invocation);
    if (!handled) debugPrint('No client tool registered as "${invocation.tool.name}"');
  }

  void dispose() => unawaited(_subscription.cancel());
}

Event has no tool or args fields, so the SDK puts them in extraData, while cid and message_id are regular fields. This extension rebuilds the flat map that tryParse reads:

extension EventClientToolPayload on Event {
  Map<String, Object?> toClientToolPayload() => {
        'type': type,
        'cid': cid,
        'message_id': messageId,
        ...extraData,
      };
}

Forget message_id and invocation.messageId is silently null.

dispatch runs the tool's actions in order and guards each one. A few behaviors to know about:

• It returns whether a tool was registered under the invoked name, not whether it succeeded. A false is normal: registrations outlive the build that made them, so an older version of your app can register a tool this build no longer has.
• Failures inside a tool go to FlutterError.onError, and to the registry's onToolError if you pass one. Use that callback to tell the user something didn't work.
• If you want to schedule the actions yourself, call resolve instead, which returns them unrun. Run them with runActions to keep the same guarding.
• A payload that announced itself as an invocation and then failed to parse is reported to FlutterError.onError. The tool the agent asked for will not run, and the agent is never told.

Listening on the client, as above, keeps invocations arriving while the user switches between conversations. Listen on a channel instead if you only care about the open one.

A complete example, including both tools above, is in the sample app.