Stream Chat AI SDK

The Stream Chat AI SDK, @stream-io/chat-ai-sdk, hosts AI assistants inside Stream Chat channels. It runs on your Node.js backend and connects Stream's real-time messaging to the Vercel AI SDK, so you can use OpenAI, Anthropic, Gemini or xAI models, stream replies with AI indicators, run tool calls, and keep long-term memory with Mem0. This page covers installation, configuration, and the APIs the package exposes. The Stream Chat LangChain SDK has the same API, built on LangChain instead.

The ai-sdk-sample folder of the chat-ai-samples repository is a complete server built with it.

Installation

npm install @stream-io/chat-ai-sdk
# or
yarn add @stream-io/chat-ai-sdk

The package ships its TypeScript sources under src/, and a CommonJS build with type declarations under dist/.

Supported providers

Pick a provider at runtime with the AgentPlatform enum:

The model option of an agent overrides the default model.

Environment variables

Set these before creating an agent:

Quick start: a single agent

import {
  Agent,
  AgentPlatform,
  createDefaultTools,
  type ClientToolDefinition,
} from "@stream-io/chat-ai-sdk";

const agent = new Agent({
  userId: "ai-bot-weather",
  channelId: "support-room",
  channelType: "messaging",
  platform: AgentPlatform.OPENAI,
  model: "gpt-4o-mini",
  instructions: [
    "Answer in a friendly, concise tone.",
    "Prefer Celsius unless the user specifies otherwise.",
  ],
  serverTools: createDefaultTools(),
  clientTools: [
    {
      name: "openHelpCenter",
      description: "Open the help center in the app",
      parameters: {
        type: "object",
        properties: { articleSlug: { type: "string" } },
        required: ["articleSlug"],
      },
    } satisfies ClientToolDefinition,
  ],
  mem0Context: {
    channelId: "support-room",
    appId: "stream-chat-support",
  },
});

await agent.start();
// Later:
await agent.stop();

Many channels with AgentManager

AgentManager runs many agents at once. It caches agents by userId, starts and stops them on demand, and stops agents that have been idle for a while.

import {
  AgentManager,
  AgentPlatform,
  createDefaultTools,
} from "@stream-io/chat-ai-sdk";

const manager = new AgentManager({
  serverToolsFactory: () => createDefaultTools(),
  inactivityThresholdMs: 15 * 60 * 1000,
  cleanupIntervalMs: 5_000,
  agentIdResolver: (channelId) => `ai-${channelId}`,
});

await manager.startAgent({
  userId: "ai-support-bot-123",
  channelId: "support-room",
  channelType: "messaging",
  platform: AgentPlatform.OPENAI,
  instructions: "Answer with step-by-step troubleshooting tips.",
});

manager.registerClientTools("support-room", [
  {
    name: "openTicket",
    description: "Open the CRM ticket in the app",
    parameters: {
      type: "object",
      properties: { ticketId: { type: "string" } },
      required: ["ticketId"],
    },
  },
]);

// On shutdown:
await manager.stopAgent("ai-support-bot-123");
manager.dispose();

How the manager behaves:

• serverToolsFactory runs each time the manager creates an agent, so the tools can differ per channel.
• AgentManager.registerClientTools(channelId, tools) stores the definitions, applies them to the running agent whose userId is what agentIdResolver returns for the channel (the channel id by default), and applies them again each time an agent for the channel starts.
• inactivityThresholdMs (about 8 hours by default) and cleanupIntervalMs (5 seconds by default) stop an agent once Agent.getLastInteraction() is older than the threshold.
• AgentManager.dispose() clears the cleanup interval and the caches. Call it when the process shuts down, so no timer keeps it running.

Working with tools

Server tools (AgentTool)

Server tools run on your backend and receive the arguments the LLM passed, validated against the tool's schema but typed unknown, plus the channel and message they were called for and a sendEvent function that sends an event to the channel. Each tool has a name, a description, optional instructions that are added to the system prompt, a Zod parameters schema, and an async execute function that returns the tool's output as text.

import { z } from "zod";
import type { AgentTool } from "@stream-io/chat-ai-sdk";

const getOrderStatus: AgentTool = {
  name: "getOrderStatus",
  description: "Look up order details by ID.",
  instructions: "Invoke only when the user provides a valid order number.",
  parameters: z.object({
    orderId: z.string().describe("The e-commerce order number"),
  }),
  async execute(args, { sendEvent }) {
    // `args` is typed `unknown`.
    const { orderId } = args as { orderId: string };
    const record = await lookupOrder(orderId);
    if (!record) {
      throw new Error(`No order found for ${orderId}`);
    }
    await sendEvent({ type: "order.viewed", orderId });
    return `Status: ${record.status}. ETA ${record.eta}`;
  },
};

Register tools with serverTools in the Agent constructor, with Agent.addServerTools(...), or with Agent.registerServerTools([...], { replace: boolean }).

Client tools

Client tools are JSON-schema definitions of actions that run in the app, such as opening a ticket or navigating to a screen. The SDK turns the schema into a Zod validator, and when the LLM calls the tool, it sends a custom_client_tool_invocation event to the channel:

{
  "type": "custom_client_tool_invocation",
  "cid": "<channel-id>",
  "message_id": "<message-id>",
  "tool": { "name": "openTicket", ... },
  "args": { "ticketId": "123" }
}

In the app, listen for the event with Channel.on("custom_client_tool_invocation", handler), run the action, and optionally reply in the chat. Events explains subscribing to channel events. Declare the event type in CustomEventTypes, as Custom data types shows. Without it, the Channel.on() call doesn't type-check. stream-chat puts a custom event's fields under event.custom.

Default tools

createDefaultTools() returns getCurrentTemperature. It needs OPENWEATHER_API_KEY, and shows how a tool:

• calls an external REST API with axios;
• converts units based on its arguments;
• uses instructions, so the LLM only calls it when the user asks about the weather.

Use it as a starting point for tools of your own.

Streaming, indicators and cancellation

• Replies stream through streamText, reading textStream or fullStream depending on what the provider supports.
• ai_indicator.update events report the AI state: AI_STATE_THINKING, AI_STATE_GENERATING, AI_STATE_EXTERNAL_SOURCES and AI_STATE_ERROR.
• ai_indicator.clear is sent once the final reply is stored.
• The SDK listens for ai_indicator.stop and aborts the request to the LLM, which stops streamText mid-reply.
• While the reply streams, the text so far is sent with serverClient.ephemeralUpdateMessage, so the app shows the reply as it's written without a database write for every chunk.
• Once the reply is complete, the final text is stored with chatClient.partialUpdateMessage, so the full reply is in the channel's history. Use ephemeral updates while the response is streaming explains why the last update is never ephemeral.

In the app, SDK integration shows how the React Native components follow this: StreamingMessageView types out the reply, AITypingIndicatorView follows the ai_indicator events, and the composer's stop button sends ai_indicator.stop.

Memory and Mem0

Mem0 gives an agent long-term memory for each user and channel. Its configuration comes from these layers:

1. MEM0_CONFIG_JSON, a JSON string, sets defaults such as metadata.
2. MEM0_DEFAULT_USER_ID, MEM0_DEFAULT_AGENT_ID and MEM0_DEFAULT_APP_ID set fallback ids for the process.
3. The mem0Context option of an Agent overrides channelId, userId, appId or agentId, or passes configOverrides.
4. At runtime, the SDK adds the Stream ids to the metadata.

Mem0 runs only when the provider is openai, anthropic or google, and MEM0_API_KEY is set. Summaries don't use it, as described below.

Summaries and titles

agent.summarize(text), or the static Agent.generateSummary(text, platform, model?), asks the LLM for a short title of about six words for any text. Use it for channel names, inbox previews or push notification titles. The helper:

• creates a lightweight model with the agent's provider settings;
• turns Mem0 off, so nothing is stored;
• trims quotes around the result.

On your backend, for example:

const headline = await agent.summarize(latestTranscript);
await channel.updatePartial({ set: { subtitle: headline } });

Best practices

• Choose the provider based on your model needs and the API keys you have.
• Load environment variables before creating agents, so they don't fail at runtime.
• Use AgentManager when agents run in many channels, so idle agents are stopped.
• Keep tool definitions narrow, with tight schemas.
• Run privileged actions as server tools.