Stream Chat LangChain SDK

The Stream Chat LangChain SDK, @stream-io/chat-langchain-sdk, hosts AI assistants inside Stream Chat channels. It runs on your Node.js backend and connects Stream's real-time messaging to the LangChain JavaScript SDK, so you can use OpenAI, Anthropic, Gemini or xAI chat models, stream replies with AI indicators, run tool calls, and keep long-term memory with Mem0. It has the same API as the Stream Chat AI SDK, which is built on the Vercel AI SDK instead.

The langchain-sample folder of the chat-ai-samples repository is a complete server built with it.

Installation

npm install @stream-io/chat-langchain-sdk
# or
yarn add @stream-io/chat-langchain-sdk

The package ships its TypeScript sources under src/, and a CommonJS build with type declarations under dist/.

Supported providers

Pick a provider at runtime with the AgentPlatform enum:

The model option of an agent overrides the default model, to match your LangChain configuration.

Environment variables

Set these before creating an agent:

Quick start: a single agent

import {
  Agent,
  AgentPlatform,
  createDefaultTools,
  type ClientToolDefinition,
} from "@stream-io/chat-langchain-sdk";

const agent = new Agent({
  userId: "ai-bot-weather",
  channelId: "support-room",
  channelType: "messaging",
  platform: AgentPlatform.OPENAI,
  model: "gpt-4o-mini",
  instructions: [
    "Answer in a friendly, concise tone.",
    "Prefer Celsius unless the user specifies otherwise.",
  ],
  serverTools: createDefaultTools(),
  clientTools: [
    {
      name: "openHelpCenter",
      description: "Open the help center in the app",
      parameters: {
        type: "object",
        properties: { articleSlug: { type: "string" } },
        required: ["articleSlug"],
      },
    } satisfies ClientToolDefinition,
  ],
  mem0Context: {
    channelId: "support-room",
    appId: "stream-chat-support",
  },
});

await agent.start();
// Later:
await agent.stop();

Many channels with AgentManager

AgentManager runs many agents at once. It caches agents by userId, starts and stops them on demand, and stops agents that have been idle for a while.

import {
  AgentManager,
  AgentPlatform,
  createDefaultTools,
} from "@stream-io/chat-langchain-sdk";

const manager = new AgentManager({
  serverToolsFactory: () => createDefaultTools(),
  inactivityThresholdMs: 30 * 60 * 1000,
  cleanupIntervalMs: 5_000,
});

await manager.startAgent({
  userId: "ai-support-bot-123",
  channelId: "support-room",
  channelType: "messaging",
  platform: AgentPlatform.ANTHROPIC,
  instructions: [
    "Provide empathetic troubleshooting tips.",
    "Offer to escalate tough issues to a human.",
  ],
});

manager.registerClientTools("support-room", [
  {
    name: "openTicket",
    description: "Open the CRM ticket in the app",
    parameters: {
      type: "object",
      properties: { ticketId: { type: "string" } },
      required: ["ticketId"],
    },
  },
]);

// On shutdown:
await manager.stopAgent("ai-support-bot-123");
manager.dispose();

How the manager behaves:

• serverToolsFactory runs each time the manager creates an agent, so the tools can differ per channel.
• AgentManager.registerClientTools(channelId, tools) stores the definitions, applies them to the running agent whose userId is what agentIdResolver returns for the channel (the channel id by default), and applies them again each time an agent for the channel starts.
• inactivityThresholdMs (about 8 hours by default) and cleanupIntervalMs (5 seconds by default) stop an agent once Agent.getLastInteraction() is older than the threshold.
• AgentManager.dispose() clears the cleanup interval and the caches. Call it when the process shuts down, so no timer keeps it running.

Working with tools

Server tools (AgentTool)

Server tools run on your backend and receive the arguments the LLM passed, typed unknown and not validated against the schema, plus the channel and message they were called for and a sendEvent function that sends an event to the channel. Each tool has a name, a description, optional instructions that are added to the system prompt, a Zod parameters schema, and an async execute function that returns the tool's output as text.

import { z } from "zod";
import type { AgentTool } from "@stream-io/chat-langchain-sdk";

const getOrderStatus: AgentTool = {
  name: "getOrderStatus",
  description: "Look up order details by ID.",
  instructions: "Invoke only when the user provides a valid order number.",
  parameters: z.object({
    orderId: z.string().describe("The e-commerce order number"),
  }),
  async execute(args, { sendEvent }) {
    // `args` is typed `unknown`.
    const { orderId } = args as { orderId: string };
    const record = await lookupOrder(orderId);
    if (!record) {
      throw new Error(`No order found for ${orderId}`);
    }
    await sendEvent({ type: "order.viewed", orderId });
    return `Status: ${record.status}. ETA ${record.eta}`;
  },
};

Register tools with serverTools in the Agent constructor, with Agent.addServerTools(...), or with Agent.registerServerTools([...], { replace: boolean }).

Client tools

Client tools are JSON-schema definitions of actions that run in the app, such as opening a ticket or navigating to a screen. The SDK turns the schema into a Zod validator, and when the LLM calls the tool, it sends a custom_client_tool_invocation event to the channel:

{
  "type": "custom_client_tool_invocation",
  "cid": "<channel-id>",
  "message_id": "<message-id>",
  "tool": { "name": "openTicket", ... },
  "args": { "ticketId": "123" }
}

In the app, listen for the event with Channel.on("custom_client_tool_invocation", handler), run the action, and optionally reply in the chat. Events explains subscribing to channel events. Declare the event type in CustomEventTypes, as Custom data types shows. Without it, the Channel.on() call doesn't type-check. stream-chat puts a custom event's fields under event.custom.

Default tools

createDefaultTools() returns getCurrentTemperature. It needs OPENWEATHER_API_KEY, and shows how a tool:

• calls an external REST API with axios;
• converts units based on its arguments;
• uses instructions, so the LLM only calls it when the user asks about the weather.

Use it as a starting point, or pass a factory of your own tools.

Streaming, indicators and cancellation

• Replies stream through LangChain's async generators. Every 200 ms the SDK stores the text so far with chatClient.partialUpdateMessage, a regular update that writes to the database each time. It doesn't use ephemeral updates.
• ai_indicator.update events report the AI state: AI_STATE_THINKING, AI_STATE_GENERATING, AI_STATE_EXTERNAL_SOURCES and AI_STATE_ERROR.
• ai_indicator.clear is sent once the final reply is stored.
• The SDK listens for ai_indicator.stop and aborts the LangChain run, which stops the reply mid-stream.
• Once the reply is complete, the final text is stored with chatClient.partialUpdateMessage, so the full reply is in the channel's history. Use ephemeral updates while the response is streaming explains why the last update is never ephemeral.

In the app, SDK integration shows how the React Native components follow this: StreamingMessageView types out the reply, AITypingIndicatorView follows the ai_indicator events, and the composer's stop button sends ai_indicator.stop.

Memory and Mem0

Mem0 gives an agent long-term memory for each user and channel. Its configuration comes from these layers:

1. MEM0_CONFIG_JSON, a JSON string, sets defaults such as metadata.
2. MEM0_DEFAULT_USER_ID, MEM0_DEFAULT_AGENT_ID and MEM0_DEFAULT_APP_ID set fallback ids for the process.
3. The mem0Context option of an Agent overrides channelId, userId, appId or agentId, or passes configOverrides.
4. At runtime, the SDK adds the Stream ids to the metadata.

Mem0 runs for every provider. Before each reply the SDK asks Mem0 for the memories of the conversation and adds them to the system prompt, and after the reply it stores the exchange. A failed Mem0 call logs a warning, and the reply goes on without it. Summaries don't use it, as described below.

Summaries and titles

agent.summarize(text), or the static Agent.generateSummary(text, platform, model?), asks the LLM for a short title of about six words for any text. Use it for channel names, inbox previews or push notification titles. The helper:

• creates a lightweight LangChain chat model with the agent's provider settings;
• turns Mem0 off, so nothing is stored;
• trims quotes around the result.

On your backend, for example:

const headline = await agent.summarize(latestTranscript);
await channel.updatePartial({ set: { subtitle: headline } });

Best practices

• Choose providers by model latency and cost.
• Load all the API keys an agent needs before creating it.
• Use AgentManager when agents run in many channels, so idle agents are stopped.
• Keep tool schemas tight, so the LLM can't call tools with arguments you don't expect.
• Keep Mem0 optional, and store only what your privacy policy allows.