# Tool calling & Dispatch

Tool calling makes it easy to connect your agent to your own systems.
A support agent looks up the customer and checks their order status. A legal AI searches your case database.

## Tool calling

Register a function in your code and the model can call it. Your process runs the function and sends the result back, so it can reach your database and internal APIs:

```ts label="JavaScript"
agent.tools.register<{ order_id: string }>({
  name: "lookup_order",
  description: "Look up an order by its number",
  parameters: {
    type: "object",
    properties: {
      order_id: {
        type: "string",
        description: "the order number the caller read out",
      },
    },
    required: ["order_id"],
  },
  run: ({ order_id }, signal) => orders.find(order_id, signal),
});
```

```kotlin label="Kotlin"
import kotlinx.serialization.json.jsonPrimitive

val lookupOrder = AgentTool(
    name = "lookup_order",
    description = "Look up an order by its number",
    parameters = AgentTool.strings(
        mapOf("order_id" to "the order number the caller read out"),
        required = listOf("order_id"),
    ),
) { args -> orders.find(args["order_id"]?.jsonPrimitive?.content.orEmpty()) }

val session = agents.agent("support").chat(lookupOrder)
```

```swift label="Swift"
let lookupOrder = AgentTool(
    name: "lookup_order",
    description: "Look up an order by its number",
    parameters: .strings(["order_id": "the order number the caller read out"], required: ["order_id"])
) { arguments in
    try await orders.find(arguments["order_id"]?.stringValue ?? "")
}

let session = try await agents.chat(agent: "support", tools: [lookupOrder])
await session.start()
```

```dart label="Dart"
final lookupOrder = AgentTool(
  name: 'lookup_order',
  description: 'Look up an order by its number',
  parameters: AgentTool.strings(
    {'order_id': 'the order number the caller read out'},
    required: ['order_id'],
  ),
  run: (arguments) => orders.find(arguments['order_id'] as String),
);

final session = await agents.agent('support').chat(SessionOptions(tools: [lookupOrder]));
```

```python label="Python"
@agent.llm.register_function(description="Look up an order by its number")
async def lookup_order(order_id: str) -> Order:
    return await orders.find(order_id)
```

```ruby label="Ruby"
agent.tools.register("lookup_order",
                     description: "Look up an order by its number",
                     params: { order_id: "the order number the caller read out" }) do |order_id:|
  Orders.find(order_id)
end
```

```php label="PHP"
use GetStream\VisionAgents\Describe;

$agent->tools->register(
    'lookup_order',
    'Look up an order by its number',
    fn (#[Describe('the order number the caller read out')] string $orderId) => $orders->find($orderId),
);
```

```go label="Go"
type LookupOrder struct {
	OrderID string `json:"order_id" schema:"the order number the caller read out"`
}

err := tools.Register(agent.Tools(), "lookup_order", "Look up an order by its number",
	func(ctx context.Context, args LookupOrder) (any, error) {
		return orders.Find(ctx, args.OrderID)
	})
```

```csharp label="C#"
agent.Tools.Register<LookupOrder, Order>("lookup_order",
    "Look up an order by its number",
    (args, cancellationToken) => orders.FindAsync(args.OrderId, cancellationToken));

record LookupOrder([property: Description("the order number the caller read out")] string OrderId);
```

```rust label="Rust"
use serde_json::json;

let lookup_order = json!({
    "type": "object",
    "properties": {
        "order_id": { "type": "string", "description": "the order number the caller read out" },
    },
    "required": ["order_id"],
});

agent.tools.register("lookup_order", "Look up an order by its number", lookup_order, |args| async move {
    orders::find(args["order_id"].as_str().unwrap_or_default()).await
});
```

The arguments are described to the model by a type: a Zod schema in JavaScript, type hints or a Pydantic model in Python, and a struct or class in most other SDKs. You can also pass a JSON Schema directly.
The return value is sent back as the tool result. Errors are sent to the model too, so it can tell the user what went wrong.

Services like Slack, Sentry or Google Calendar don't need any code. See the [catalog](https://getstream.io/agents/docs/flutter/connectors/#catalog).

### Tools that run in the browser

In JavaScript, `agent.tools.register` hosts a tool in a [Dispatch](#dispatch) worker on your backend, for every session of the agent. A page can also bring tools of its own, which run in the browser for the session it opens, like the Kotlin and Swift tools above run on the device. Only their answers leave the page:

```ts label="JavaScript"
import { Client, Tools } from "@stream-io/calypso";
import { z } from "zod";

const tools = new Tools()
  .register({
    name: "get_local_context",
    description: "The person's current date, local time and timezone.",
    parameters: z.object({}),
    run: () => ({
      now: new Date().toISOString(),
      timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
    }),
  })
  .register({
    name: "get_location",
    description:
      "Where the person is. Only call it when the answer depends on it.",
    parameters: z.object({
      reason: z.string().describe("why you need it, in a few words"),
    }),
    displayTitle: "Checking where you are",
    approval: {
      title: "Share your location?",
      reasonArgument: "reason",
      allowTitle: "Share",
      declineTitle: "Not now",
    },
    run: () =>
      new Promise((resolve, reject) =>
        navigator.geolocation.getCurrentPosition(
          ({ coords }) =>
            resolve({ latitude: coords.latitude, longitude: coords.longitude }),
          reject,
        ),
      ),
  });

// `client` holds your apiKey and a user from setUser.
const session = await client.agent("support").sessions.create({ tools });

// A session opened before, such as after a reload, gets them back with its id.
const resumed = await client
  .agent("support")
  .sessions.resume(sessionId, { tools });
```

The session's socket runs them, so they answer while the page holds the session open. A tool with an `approval` waits in `session.approvals` until the user answers it, as in [Approval](#approval).

### Input schema

The input schema of a tool is always a JSON Schema object. Depending on the SDK, you can describe it the idiomatic way instead: Zod in JavaScript, Pydantic in Python and struct tags in Go.

The arguments the model sends are checked against the schema before your function runs. Arguments that don't fit fail the call with what was wrong, such as `seats: Input should be less than or equal to 9`, and the model can ask again. Your function never sees them.

For simple tools, annotate the arguments and the SDK generates the JSON Schema:

```ts label="JavaScript"
import { z } from "zod";

agent.tools.register({
  name: "get_weather",
  description: "Current weather in a city",
  parameters: z.object({
    city: z.string().describe("City name, e.g. Amsterdam"),
  }),
  run: async ({ city }) => weather(city),
});
```

```kotlin label="Kotlin"
// AgentTool.strings writes the schema of a tool whose arguments are all strings.
// For any other schema, pass a JsonObject. Read each argument defensively.
val getWeather = AgentTool(
    name = "get_weather",
    description = "Current weather in a city",
    parameters = AgentTool.strings(mapOf("city" to "City name, e.g. Amsterdam"), required = listOf("city")),
) { args -> weather(args["city"]?.jsonPrimitive?.content.orEmpty()) }
```

```swift label="Swift"
// .strings writes the schema of a tool whose arguments are all strings; any other schema is
// a RawJSON literal. Arguments are not checked against it, so read each one defensively.
let getWeather = AgentTool(
    name: "get_weather",
    description: "Current weather in a city",
    parameters: .strings(["city": "City name, e.g. Amsterdam"], required: ["city"])
) { arguments in
    try await weather(arguments["city"]?.stringValue ?? "")
}
```

```python label="Python"
from typing import Annotated

from pydantic import Field


@agent.llm.register_function(description="Current weather in a city")
async def get_weather(
    city: Annotated[str, Field(description="City name, e.g. Amsterdam")],
) -> str:
    return await weather(city)
```

```go label="Go"
type Weather struct {
	City string `json:"city" schema:"City name, e.g. Amsterdam"`
}

err := tools.Register(agent.Tools(), "get_weather", "Current weather in a city",
	func(ctx context.Context, in Weather) (any, error) {
		return weather(ctx, in.City)
	})
```

For a tool with several fields and validation rules, describe them with Zod, Pydantic or Go struct tags. This is the preferred approach for anything that needs more than one field:

```ts label="JavaScript"
import { z } from "zod";

agent.tools.register({
  name: "book_flight",
  description: "Book a flight for the caller",
  parameters: z.object({
    flight_id: z
      .string()
      .regex(/^[A-Z]{2}\d{1,4}$/)
      .describe("e.g. LH400"),
    seats: z.number().int().min(1).max(9),
    cabin: z.enum(["economy", "business", "first"]).default("economy"),
  }),
  run: async ({ flight_id, seats, cabin }) =>
    bookings.book(flight_id, seats, cabin),
});
```

```python label="Python"
from typing import Literal

from pydantic import BaseModel, Field


class BookFlight(BaseModel):
    flight_id: str = Field(pattern=r"^[A-Z]{2}\d{1,4}$", description="e.g. LH400")
    seats: int = Field(ge=1, le=9)
    cabin: Literal["economy", "business", "first"] = "economy"


# A function that takes a single model gets the model's fields as the tool's
# arguments, and is handed the parsed model.
@agent.llm.register_function(description="Book a flight for the caller")
async def book_flight(booking: BookFlight) -> str:
    return await bookings.book(booking.flight_id, booking.seats, booking.cabin)
```

```go label="Go"
// Tags: schema (the description), enum, minimum, maximum, exclusiveMinimum,
// exclusiveMaximum, multipleOf, minLength, maxLength, minItems, maxItems,
// pattern, format, default and example. An invopop-style
// `jsonschema:"description=...,minLength=1"` tag works too.
// default is only told to the model: a field left out arrives as its zero value.
type BookFlight struct {
	FlightID string `json:"flight_id" schema:"e.g. LH400" pattern:"^[A-Z]{2}[0-9]{1,4}$"`
	Seats    int    `json:"seats" minimum:"1" maximum:"9"`
	Cabin    string `json:"cabin,omitempty" enum:"economy,business,first" default:"economy"`
}

err := tools.Register(agent.Tools(), "book_flight", "Book a flight for the caller",
	func(ctx context.Context, in BookFlight) (any, error) {
		return airline.Book(ctx, in.FlightID, in.Seats, cmp.Or(in.Cabin, "economy"))
	})
```

You can also pass the JSON Schema directly:

```ts label="JavaScript"
// The schema is told to the model as written; arguments are not checked against it,
// so the type parameter is a promise, not a guarantee.
agent.tools.register<{ flight_id: string; seats: number; cabin?: string }>({
  name: "book_flight",
  description: "Book a flight for the caller",
  parameters: {
    type: "object",
    properties: {
      flight_id: {
        type: "string",
        pattern: "^[A-Z]{2}[0-9]{1,4}$",
        description: "e.g. LH400",
      },
      seats: { type: "integer", minimum: 1, maximum: 9 },
      cabin: {
        type: "string",
        enum: ["economy", "business", "first"],
        default: "economy",
      },
    },
    required: ["flight_id", "seats"],
  },
  run: async ({ flight_id, seats, cabin = "economy" }) =>
    bookings.book(flight_id, seats, cabin),
});
```

```kotlin label="Kotlin"
// The schema is told to the model as written.
val bookFlight = AgentTool(
    name = "book_flight",
    description = "Book a flight for the caller",
    parameters = Json.parseToJsonElement(
        """
        {
          "type": "object",
          "properties": {
            "flight_id": {"type": "string", "pattern": "^[A-Z]{2}[0-9]{1,4}$", "description": "e.g. LH400"},
            "seats": {"type": "integer", "minimum": 1, "maximum": 9},
            "cabin": {"type": "string", "enum": ["economy", "business", "first"], "default": "economy"}
          },
          "required": ["flight_id", "seats"]
        }
        """,
    ).jsonObject,
) { args ->
    bookings.book(
        args["flight_id"]?.jsonPrimitive?.content.orEmpty(),
        args["seats"]?.jsonPrimitive?.intOrNull ?: 1,
        args["cabin"]?.jsonPrimitive?.content ?: "economy",
    )
}
```

```swift label="Swift"
// The schema is told to the model as written; arguments are not checked against it.
let bookFlight = AgentTool(
    name: "book_flight",
    description: "Book a flight for the caller",
    parameters: [
        "type": "object",
        "properties": [
            "flight_id": ["type": "string", "pattern": "^[A-Z]{2}[0-9]{1,4}$", "description": "e.g. LH400"],
            "seats": ["type": "integer", "minimum": 1, "maximum": 9],
            "cabin": ["type": "string", "enum": ["economy", "business", "first"], "default": "economy"],
        ],
        "required": ["flight_id", "seats"],
    ]
) { arguments in
    try await bookings.book(
        arguments["flight_id"]?.stringValue ?? "",
        Int(arguments["seats"]?.numberValue ?? 1),
        arguments["cabin"]?.stringValue ?? "economy"
    )
}
```

```python label="Python"
@agent.llm.register_function(
    description="Book a flight for the caller",
    parameters_schema={
        "type": "object",
        "properties": {
            "flight_id": {"type": "string", "pattern": "^[A-Z]{2}[0-9]{1,4}$", "description": "e.g. LH400"},
            "seats": {"type": "integer", "minimum": 1, "maximum": 9},
            "cabin": {"type": "string", "enum": ["economy", "business", "first"], "default": "economy"},
        },
        "required": ["flight_id", "seats"],
    },
)
# The schema is told to the model as written; arguments are not checked against it.
async def book_flight(flight_id: str, seats: int, cabin: str = "economy") -> str:
    return await bookings.book(flight_id, seats, cabin)
```

```go label="Go"
type BookFlightArgs struct {
	FlightID string `json:"flight_id"`
	Seats    int    `json:"seats"`
	Cabin    string `json:"cabin"`
}

schema := []byte(`{
  "type": "object",
  "properties": {
    "flight_id": {"type": "string", "pattern": "^[A-Z]{2}[0-9]{1,4}$", "description": "e.g. LH400"},
    "seats":     {"type": "integer", "minimum": 1, "maximum": 9},
    "cabin":     {"type": "string", "enum": ["economy", "business", "first"], "default": "economy"}
  },
  "required": ["flight_id", "seats"]
}`)

// The schema can be a map[string]any, raw JSON or another library's schema type.
// Arguments are checked against it, then decoded into the struct;
// tools.RegisterSchema hands the function the raw JSON instead.
err := tools.RegisterWithSchema(agent.Tools(), "book_flight", "Book a flight for the caller", schema,
	func(ctx context.Context, in BookFlightArgs) (any, error) {
		return airline.Book(ctx, in.FlightID, in.Seats, cmp.Or(in.Cabin, "economy"))
	})
```

In Go, a tool can also be a type. Its fields are the arguments, and `Run` is called with them filled in:

```go
type GetWeather struct {
	City string `json:"city" schema:"City name, e.g. Amsterdam"`
}

func (GetWeather) Name() string        { return "get_weather" }
func (GetWeather) Description() string { return "Current weather in a city" }
func (w GetWeather) Run(ctx context.Context) (any, error) {
	return weather(ctx, w.City)
}

err := agent.Tools().Add(GetWeather{})
```

Unexported fields are yours, for whatever `Run` needs to reach, such as a database handle.

### Tool call context

Tool functions are called with the input and the context of the call:

- `id`: the unique identifier of this call. It stays the same when the call is sent again, so use it to make tool calls idempotent and handle retries.
- `session_id`: the session the call comes from.
- `user`: the user whose turn triggered the call.
- `conversation`: the chat conversation.

**JavaScript**

It is the third argument of `run`, `toolCallContext`.

**Kotlin**

A tool runs on the user's device, and `run` is handed the arguments only, as a `JsonObject`.

**Swift**

A tool runs on the user's device, and `run` is handed the arguments only.

**Python**

It is `get_tool_call_context()`.

**Go**

It is `tools.ToolCallContextFrom(ctx)`.

### Approval

By default, a tool runs as soon as the model calls it. A tool can instead ask the user first. On a voice call the agent asks out loud and runs the tool only on a yes. In a text conversation, the question goes to your app to show. Each `{argument}` in the question is filled in from the call:

```ts label="JavaScript"
agent.tools.register({
  name: "book_flight",
  description: "Book the flight the user picked",
  parameters: BookFlight,
  approval: { title: "Book flight {flight_id}?" },
  run: async ({ flight_id }, signal, toolCallContext) => {
    const booking = await airline.book({
      flightId: flight_id,
      userId: toolCallContext.user.id,
    });
    return `Booked. Confirmation number ${booking.id}.`;
  },
});

// In a page, tools passed to sessions.create({ tools }) run in the browser, and in a
// text conversation each call waits in session.approvals until the user answers it.
const request = session.approvals[0];
if (request) {
  session.decide(request.id, { allowed: true });
}
```

```kotlin label="Kotlin"
val bookFlight = AgentTool(
    name = "book_flight",
    description = "Book the flight the user picked",
    parameters = AgentTool.strings(mapOf("flight_id" to "e.g. LH400"), required = listOf("flight_id")),
    approval = AgentTool.Approval(title = "Book flight {flight_id}?"),
) { args ->
    val booking = airline.book(args["flight_id"]?.jsonPrimitive?.content.orEmpty())
    "Booked. Confirmation number ${booking.id}."
}

// In a text conversation, each call waits in session.approvals until the user answers it.
session.approvals.value.firstOrNull()?.let { request ->
    session.decide(request.id, allowed = true)
}
```

```swift label="Swift"
let bookFlight = AgentTool(
    name: "book_flight",
    description: "Book the flight the user picked",
    parameters: .strings(["flight_id": "e.g. LH400"], required: ["flight_id"]),
    approval: AgentTool.Approval(title: "Book flight {flight_id}?")
) { arguments in
    let booking = try await airline.book(flightID: arguments["flight_id"]?.stringValue ?? "")
    return "Booked. Confirmation number \(booking.id)."
}

// In a text conversation, each call waits in session.approvals until the user answers it.
if let request = session.approvals.first {
    try await session.decide(request.id, allowed: true)
}
```

```python label="Python"
from vision_agents.core.llm import get_tool_call_context


@agent.llm.register_function(
    description="Book the flight the user picked",
    needs_approval="Book flight {flight_id}?",
)
async def book_flight(flight_id: str) -> str:
    tool_call_context = get_tool_call_context()
    booking = await airline.book(flight_id=flight_id, user_id=tool_call_context.user.id)
    return f"Booked. Confirmation number {booking.id}."
```

```go label="Go"
err = tools.Register(agent.Tools(), "book_flight", "Book the flight the user picked",
	func(ctx context.Context, args BookFlight) (any, error) {
		toolCallContext := tools.ToolCallContextFrom(ctx)
		booking, err := airline.Book(ctx, airline.BookRequest{
			FlightID: args.FlightID,
			UserID:   toolCallContext.User.ID,
		})
		if err != nil {
			return nil, err
		}
		return fmt.Sprintf("Booked. Confirmation number %s.", booking.ID), nil
	},
	tools.NeedsApproval("Book flight {flight_id}?"),
)
```

In a conversation kept in Stream Chat, the call's step in the reply carries the question too, and the AI components ask it there: pass an approver to `AIConversation` in React, as the [React assistant](https://github.com/GetStream/Vision-Agents/tree/accelerate/examples/voice_agents/react_assistant) sample does. `session.conversation` tells a page when the calls waiting change, including one the model withdrew.

### Interruptions

By default, a tool keeps running when the user talks over the agent, and its result reaches the model when it comes. A tool that only reads, such as a search, can ask to be cancelled instead:

```ts label="JavaScript"
agent.tools.register({
  name: "search_flights",
  description: "Search for flights",
  parameters: SearchFlights,
  interruptible: true,
  // signal is aborted when the user talks over the agent
  run: async (args, signal) => airline.search(args, { signal }),
});
```

```python label="Python"
@agent.llm.register_function(description="Search for flights", interruptible=True)
async def search_flights(destination: str) -> list[Flight]:
    # the task is cancelled when the user talks over the agent
    return await airline.search(destination)
```

```go label="Go"
err = tools.Register(agent.Tools(), "search_flights", "Search for flights",
	func(ctx context.Context, args SearchFlights) (any, error) {
		// ctx is cancelled when the user talks over the agent
		return airline.Search(ctx, args)
	},
	tools.Interruptible(true),
)
```

A tool that changes something can stay interruptible until the change starts. From then on it holds the call, so talking over the agent no longer cancels it and only its timeout does:

```ts label="JavaScript"
agent.tools.register({
  name: "book_flight",
  description: "Book the flight the user picked",
  parameters: BookFlight,
  interruptible: true,
  run: async ({ flight_id }, signal, toolCallContext) => {
    const seat = await airline.holdSeat(flight_id, { signal });
    // from here on, talking over the agent doesn't abort the booking
    const held = await toolCallContext.disallowInterruptions();
    return airline.book({ flightId: flight_id, seat, signal: held });
  },
});
```

```python label="Python"
from vision_agents.core.llm import get_tool_call_context


@agent.llm.register_function(description="Book the flight the user picked", interruptible=True)
async def book_flight(flight_id: str) -> str:
    seat = await airline.hold_seat(flight_id)
    # from here on, talking over the agent doesn't cancel the booking
    await get_tool_call_context().disallow_interruptions()
    booking = await airline.book(flight_id=flight_id, seat=seat)
    return f"Booked. Confirmation number {booking.id}."
```

```go label="Go"
err = tools.Register(agent.Tools(), "book_flight", "Book the flight the user picked",
	func(ctx context.Context, args BookFlight) (any, error) {
		seat, err := airline.HoldSeat(ctx, args.FlightID)
		if err != nil {
			return nil, err
		}
		// from here on, talking over the agent doesn't cancel ctx
		ctx = tools.ToolCallContextFrom(ctx).DisallowInterruptions(ctx)
		return airline.Book(ctx, airline.BookRequest{FlightID: args.FlightID, Seat: seat})
	},
	tools.Interruptible(true),
)
```

When the call was cancelled before it could be held, the change never starts: in JavaScript the signal is already aborted, Python raises `CallNotHeld`, and in Go the context comes back cancelled.

### Timeouts

By default, a tool call fails and is cancelled when it doesn't return within 30 seconds. You can set a timeout per tool:

```ts label="JavaScript"
agent.tools.register({
  name: "book_flight",
  description: "Book the flight the user picked",
  parameters: BookFlight,
  timeoutMs: 60_000,
  run: async ({ flight_id }, signal) =>
    airline.book({ flightId: flight_id, signal }),
});
```

```python label="Python"
@agent.llm.register_function(description="Book the flight the user picked", timeout=60)
async def book_flight(flight_id: str) -> str:
    return await airline.book(flight_id=flight_id)
```

```go label="Go"
err = tools.Register(agent.Tools(), "book_flight", "Book the flight the user picked",
	func(ctx context.Context, args BookFlight) (any, error) {
		return airline.Book(ctx, airline.BookRequest{FlightID: args.FlightID})
	},
	tools.Timeout(60*time.Second),
)
```

### Concurrency

In Go, up to 16 calls of a session's tools run at once. You can cap a single tool lower. A call over the cap waits its turn, and its timeout keeps counting while it does:

```go
err = tools.Register(agent.Tools(), "book_flight", "Book the flight the user picked", bookFlight,
	tools.Concurrency(2),
)
```

### Progress updates

A slow tool can tell the user how it's going. On a voice call the agent says it in its own words once nobody is talking. In a text conversation it shows on the tool call:

```ts label="JavaScript"
agent.tools.register({
  name: "book_flight",
  description: "Book the flight the user picked",
  parameters: BookFlight,
  run: async ({ flight_id }, signal, toolCallContext) => {
    toolCallContext.update("Checking seats on the flight");
    const booking = await airline.book({ flightId: flight_id, signal });
    return `Booked. Confirmation number ${booking.id}.`;
  },
});
```

```python label="Python"
from vision_agents.core.llm import get_tool_call_context


@agent.llm.register_function(description="Book the flight the user picked")
async def book_flight(flight_id: str) -> str:
    await get_tool_call_context().update("Checking seats on the flight")
    booking = await airline.book(flight_id=flight_id)
    return f"Booked. Confirmation number {booking.id}."
```

```go label="Go"
err = tools.Register(agent.Tools(), "book_flight", "Book the flight the user picked",
	func(ctx context.Context, args BookFlight) (any, error) {
		tools.ToolCallContextFrom(ctx).Update(ctx, "Checking seats on the flight")
		booking, err := airline.Book(ctx, airline.BookRequest{FlightID: args.FlightID})
		if err != nil {
			return nil, err
		}
		return fmt.Sprintf("Booked. Confirmation number %s.", booking.ID), nil
	},
)
```

A newer update replaces one that hasn't been said yet, and the result replaces both.
To have the agent say your exact words instead, use `say` (`Say` in Go). A speech-to-speech model may still paraphrase them.

### Idempotency

A call can be sent to your process again, for example when it reconnects. The call's `id` stays the same, so use it as the idempotency key for anything the tool changes:

```ts label="JavaScript"
agent.tools.register({
  name: "book_flight",
  description: "Book the flight the user picked",
  parameters: BookFlight,
  run: async ({ flight_id }, signal, toolCallContext) => {
    const booking = await airline.book({
      flightId: flight_id,
      userId: toolCallContext.user.id,
      idempotencyKey: toolCallContext.id,
    });
    return `Booked. Confirmation number ${booking.id}.`;
  },
});
```

```python label="Python"
from vision_agents.core.llm import get_tool_call_context


@agent.llm.register_function(description="Book the flight the user picked")
async def book_flight(flight_id: str) -> str:
    tool_call_context = get_tool_call_context()
    booking = await airline.book(
        flight_id=flight_id,
        user_id=tool_call_context.user.id,
        idempotency_key=tool_call_context.id,
    )
    return f"Booked. Confirmation number {booking.id}."
```

```go label="Go"
err = tools.Register(agent.Tools(), "book_flight", "Book the flight the user picked",
	func(ctx context.Context, args BookFlight) (any, error) {
		toolCallContext := tools.ToolCallContextFrom(ctx)
		booking, err := airline.Book(ctx, airline.BookRequest{
			FlightID:       args.FlightID,
			UserID:         toolCallContext.User.ID,
			IdempotencyKey: toolCallContext.ID,
		})
		if err != nil {
			return nil, err
		}
		return fmt.Sprintf("Booked. Confirmation number %s.", booking.ID), nil
	},
)
```

### Returning images

A tool can show the model pictures, such as a seat map. Return them with the text:

```ts label="JavaScript"
import { ToolOutput } from "@stream-io/calypso";

agent.tools.register({
  name: "seat_map",
  description: "Show the seat map for a flight",
  parameters: SeatMap,
  run: async ({ flight_id }) =>
    new ToolOutput({
      text: `Seat map for ${flight_id}`,
      images: [{ url: await airline.seatMapUrl(flight_id) }],
    }),
});
```

```python label="Python"
from vision_agents.core.llm import ImageContent


@agent.llm.register_function(description="Show the seat map for a flight")
async def seat_map(flight_id: str) -> list[str | ImageContent]:
    png = await airline.seat_map(flight_id)
    return [f"Seat map for {flight_id}", ImageContent(data=png, mime="image/png")]
```

```go label="Go"
err = tools.Register(agent.Tools(), "seat_map", "Show the seat map for a flight",
	func(ctx context.Context, args SeatMap) (any, error) {
		url, err := airline.SeatMapURL(ctx, args.FlightID)
		if err != nil {
			return nil, err
		}
		return tools.Output{Text: "Seat map for " + args.FlightID, Images: []tools.Image{{URL: url}}}, nil
	},
)
```

An image is an https URL or the image itself (bytes in Python, a data URI in JavaScript and Go), at most 5 MB. On a voice call the agent's `vision` skill looks at it.
In a text conversation a model that reads images sees it, and one that reads only text is told the tool failed.

## Dispatch

A dispatch worker is a process on your server that receives work from Stream: messages, [phone calls](https://getstream.io/agents/docs/flutter/telephony/accepting-calls/) and tool calls.
It connects out to Stream, so nothing you run needs to be publicly reachable.

### Running a worker

Register a handler and run the worker. When a message has a session id, `answer` lets the model reply on that session. Otherwise `sessionFor` gives you the session for that channel, and starts one from your agent if needed:

```ts label="JavaScript"
import { Agent, Dispatch } from "@stream-io/calypso";

const dispatch = new Dispatch({ capacity: 8 });
const support = () =>
  new Agent({ name: "support", pipeline: { config: "support" } });

dispatch.onMessage(async (message) => {
  if (message.sessionId) {
    await dispatch.answer(message);
    return;
  }
  const session = await dispatch.sessionFor(message, support);
  await session.responses.create(message.text);
});

await dispatch.run();
```

```python label="Python"
from vision_agents.core import Agent
from vision_agents.plugins import stream

dispatch = stream.Dispatch(capacity=8)


@dispatch.wait_for_message()
async def written(message: stream.InboundMessage) -> None:
    if message.session_id:
        await dispatch.answer(message)
        return
    agent = await dispatch.get_or_create_agent(message, lambda: Agent(config="support"))
    await agent.responses.create(message.text)


await dispatch.run()
```

```ruby label="Ruby"
dispatch = GetStream::VisionAgents::Dispatch.new(capacity: 8)

dispatch.wait_for_message do |message|
  next dispatch.answer(message) unless message.session_id.empty?

  dispatch.get_or_create_agent(message) { GetStream::VisionAgents::Agent.new(config: "support") }
          .reply(message)
          .responses.create(message.text)
end

dispatch.run
```

```php label="PHP"
use GetStream\VisionAgents\Agent;
use GetStream\VisionAgents\Inbound\InboundMessage;
use GetStream\VisionAgents\Worker\Dispatch;

$dispatch = new Dispatch(capacity: 8);
$support = fn (): Agent => new Agent(config: 'support');

$dispatch->waitForMessage(function (InboundMessage $message) use ($dispatch, $support): void {
    if ($message->sessionId !== '') {
        $dispatch->answer($message);
        return;
    }
    $dispatch->getOrCreateAgent($message, $support)->responses->create($message->text);
});

$dispatch->run();
```

```go label="Go"
dispatch, err := agents.NewDispatch(agents.DispatchOptions{Capacity: 8})
if err != nil {
	return err
}

support := func(ctx context.Context, message agents.InboundMessage) (*agents.Agent, error) {
	return agents.New(agents.Options{
		Name: "support",
		LLM:  stream.Accelerated(stream.Config{Agent: "support"}),
	})
}

dispatch.OnMessage(func(ctx context.Context, message agents.InboundMessage) error {
	if message.SessionID != "" {
		return dispatch.Answer(ctx, message)
	}
	conversation, err := dispatch.Conversation(ctx, message, support)
	if err != nil {
		return err
	}
	return conversation.Respond(message.Text)
})

return dispatch.Run(ctx)
```

```csharp label="C#"
var dispatch = new Dispatch(new DispatchOptions { Capacity = 8 });

dispatch.WaitForMessage(async message =>
{
    if (message.SessionId != "")
    {
        await dispatch.AnswerAsync(message);
        return;
    }
    var agent = await dispatch.GetOrCreateAgentAsync(message,
        () => new Agent(new AgentOptions { Config = "support" }));
    await agent.Responses.CreateAsync(message.Text);
});

await dispatch.RunAsync(cancellationToken);
```

```rust label="Rust"
use vision_agents::{Agent, Client, Dispatch, InboundMessage};

let dispatch = Dispatch::with_capacity(Client::from_env()?, 8);

let worker = dispatch.clone();
dispatch.wait_for_message(move |message: InboundMessage| {
    let dispatch = worker.clone();
    async move {
        if !message.session_id.is_empty() {
            return dispatch.answer(&message).await;
        }
        let session = dispatch
            .get_or_create_agent(&message, || async { Ok(Agent::new("support")) })
            .await?;
        session.responses.create(&message.text).await?;
        Ok(())
    }
});

dispatch.run().await
```

To reply in your own words instead of the model's, skip `answer` and send the reply from your backend.

The method names differ a bit per SDK:

| What it does                              | JavaScript   | Python, Ruby, Rust    | PHP                | Go             | C#                      |
| ----------------------------------------- | ------------ | --------------------- | ------------------ | -------------- | ----------------------- |
| Handle each message                       | `onMessage`  | `wait_for_message`    | `waitForMessage`   | `OnMessage`    | `WaitForMessage`        |
| Handle each incoming call                 | `onCall`     | `wait_for_call`       | `waitForCall`      | `OnCall`       | `WaitForCall`           |
| The session answering a message's channel | `sessionFor` | `get_or_create_agent` | `getOrCreateAgent` | `Conversation` | `GetOrCreateAgentAsync` |
| Have the model reply on a session         | `answer`     | `answer`              | `answer`           | `Answer`       | `AnswerAsync`           |
| Host an agent's tools                     | `host`       | `host`                | `host`             | `Host`         | `Host`                  |

### Hosting tools

Tools registered on an agent only run for the sessions that process opens. Sessions your users open from a browser or phone need a worker to host their tools. Every session of the agent can then use them:

```ts label="JavaScript"
const agent = client.agent("support");
agent.tools.register({
  name: "lookup_order",
  description: "Look up an order",
  run: lookupOrder,
});
dispatch.host(agent, { toolTimeoutMs: 60_000 });
```

```python label="Python"
agent = stream.Client().agent("support")


@agent.register(description="Look up an order by its number")
async def lookup_order(order_id: str) -> Order:
    return await orders.find(order_id)


dispatch.host(agent, tool_timeout=60)
```

```ruby label="Ruby"
agent = client.agent("support")
agent.tools.register("lookup_order", description: "Look up an order",
                     params: { order_id: "the order number" }) { |order_id:| Orders.find(order_id) }
dispatch.host(agent, tool_timeout: 60)
```

```php label="PHP"
$agent = $client->agent('support');
$agent->tools->register('lookup_order', 'Look up an order', $lookupOrder);
$dispatch->host($agent, toolTimeoutMs: 60_000);
```

```go label="Go"
agent := api.Agent("support")
err := tools.Register(agent.Tools(), "lookup_order", "Look up an order",
	func(ctx context.Context, args LookupOrder) (any, error) {
		return orders.Find(ctx, args.OrderID)
	})
dispatch.Host(agent, time.Minute)
```

```csharp label="C#"
var agent = client.Agent("support");
agent.Tools.Register<LookupOrder, Order>("lookup_order", "Look up an order",
    (args, cancellationToken) => orders.FindAsync(args.OrderId, cancellationToken));
dispatch.Host(agent, toolTimeout: TimeSpan.FromMinutes(1));
```

```rust label="Rust"
let agent = client.agent("support");
agent.tools.register("lookup_order", "Look up an order", lookup_order, |args| async move {
    orders::find(args["order_id"].as_str().unwrap_or_default()).await
});
dispatch.host(&agent, Some(Duration::from_secs(60)));
```

Host tools before calling `run`. The worker keeps running until you stop it. The tool timeout is how long Stream waits for one tool call to be answered before telling the model it failed: two minutes unless you set it.

### Leaving text to your server

By default the model answers every message. You can also have your server handle messages first, in `agent.yaml`:

```yaml
name: Stream product
mode: text
dispatch:
  incoming_call: enabled
  text: enabled
```

With `text: enabled` each message goes to your worker first. Check it, enrich it or route it, then call `answer` to let the model reply. The user sees the agent thinking in the meantime.

This adds a round trip before the model starts answering. Messages with images or video are refused, and a message is dropped when no worker is connected. Phone calls to your numbers always go to a worker, so `incoming_call` doesn't change anything yet.

### Capacity

Capacity is how many messages and calls a worker handles at once. Stream sends new work to the worker with the most room, so you can run several workers to share the load. Hosted tool calls don't count against capacity.

---

For the most recent version of this documentation, visit [https://getstream.io/agents/docs/flutter/tool-calling-and-dispatch/](https://getstream.io/agents/docs/flutter/tool-calling-and-dispatch/).