Stream Chat LangChain SDK

The Stream Chat LangChain SDK wires Stream’s real-time messaging primitives to the LangChain JavaScript SDK. With a single package you can host AI copilots inside Stream channels, connect OpenAI, Anthropic, Gemini, or xAI chat models, stream partial responses with typing indicators, dispatch tool calls, and optionally persist long-term memory via Mem0.

The code on this page runs on your backend server, not in the Flutter app. For a complete Node.js backend to pair with the Flutter app, see the sample backend langchain-sample.

Installation

npm install @stream-io/chat-langchain-sdk
# or
yarn add @stream-io/chat-langchain-sdk

The package ships TypeScript sources plus transpiled JavaScript under dist/. Import either ESM or CJS builds depending on your bundler.

Supported Providers

Select a provider at runtime through the AgentPlatform enum:

Override the model ID per agent via the model option to match your LangChain configuration.

Environment Variables

Set the following before instantiating an agent:

Tip: when running multiple agents, load environment variables once at process start-up so lazy imports such as serverClient do not throw intermittently.

Quick Start: Single Agent

import {
  Agent,
  AgentPlatform,
  createDefaultTools,
  type ClientToolDefinition,
} from "@stream-io/chat-langchain-sdk";

const agent = new Agent({
  userId: "ai-bot-weather",
  channelId: "support-room",
  channelType: "messaging",
  platform: AgentPlatform.OPENAI,
  model: "gpt-4o-mini",
  instructions: [
    "Answer in a friendly, concise tone.",
    "Prefer Celsius unless the user specifies otherwise.",
  ],
  serverTools: createDefaultTools(),
  clientTools: [
    {
      name: "openHelpCenter",
      description: "Open the help center in the web app",
      parameters: {
        type: "object",
        properties: { articleSlug: { type: "string" } },
        required: ["articleSlug"],
      },
    } satisfies ClientToolDefinition,
  ],
  mem0Context: {
    channelId: "support-room",
    appId: "stream-chat-support",
  },
});

await agent.start();
// Later...
await agent.stop();

Multi-Channel Deployments with AgentManager

Use AgentManager to orchestrate many concurrent agents without leaking resources. It caches instances by userId, supports lazy start/stop, and periodically disposes idle bots.

import {
  AgentManager,
  AgentPlatform,
  createDefaultTools,
} from "@stream-io/chat-langchain-sdk";

const manager = new AgentManager({
  serverToolsFactory: () => createDefaultTools(),
  inactivityThresholdMs: 30 * 60 * 1000,
  cleanupIntervalMs: 5_000,
});

await manager.startAgent({
  userId: "ai-support-bot-123",
  channelId: "support-room",
  channelType: "messaging",
  platform: AgentPlatform.ANTHROPIC,
  instructions: [
    "Provide empathetic troubleshooting tips.",
    "Offer to escalate tough issues to a human.",
  ],
});

manager.registerClientTools("support-room", [
  {
    name: "openTicket",
    description: "Open the CRM ticket in the dashboard",
    parameters: {
      type: "object",
      properties: { ticketId: { type: "string" } },
      required: ["ticketId"],
    },
  },
]);

// On shutdown:
await manager.stopAgent("ai-support-bot-123");
manager.dispose();

Key behaviors:

• serverToolsFactory runs each time a new agent is constructed, allowing per-channel config.
• registerClientTools(channelId, tools) persists definitions and re-applies them the next time the channel’s agent starts.
• inactivityThresholdMs (default ~8 hours) pairs with cleanupIntervalMs (default 5 seconds) to stop agents automatically once Agent#getLastInteraction() stales.
• dispose() clears the cleanup interval and caches; call it during process shutdown to avoid hanging timers.

Working with Tools

Server Tools (AgentTool)

Server tools run in Node/TS and receive typed arguments plus channel/message context. Each tool consists of a name, human-facing description, optional instructions (folded into the system prompt), a Zod parameters schema, and an async execute function that returns the tool output text.

import { z } from "zod";
import type { AgentTool } from "@stream-io/chat-langchain-sdk";

const getOrderStatus: AgentTool = {
  name: "getOrderStatus",
  description: "Look up order details by ID.",
  instructions: "Invoke only when the user provides a valid order number.",
  parameters: z.object({
    orderId: z.string().describe("The e-commerce order number"),
  }),
  async execute({ orderId }, { channel }) {
    const record = await lookupOrder(orderId);
    if (!record) {
      throw new Error(`No order found for ${orderId}`);
    }
    await channel.sendEvent({ type: "order.viewed", orderId });
    return `Status: ${record.status}. ETA ${record.eta}`;
  },
};

Register tools through serverTools in the Agent constructor, agent.addServerTools(...), or agent.registerServerTools([...], { replace: boolean }).

Client Tools

Client tools are JSON-schema definitions representing capabilities that must be executed on the front end (e.g., “open ticket in CRM”, “navigate to screen”). The SDK converts the schema to a Zod validator and exposes an internal tool that sends a custom_client_tool_invocation event containing:

{
  "type": "custom_client_tool_invocation",
  "cid": "<channel-id>",
  "message_id": "<message-id>",
  "tool": { "name": "openTicket", ... },
  "args": { "ticketId": "123" }
}

Listen for that event in your Flutter app and perform the action. Client tools are fire-and-forget: the protocol has no way to send a result back to the model. See Client-side tools for how to register tools and handle invocations in Flutter.

Default Tools

createDefaultTools() returns getCurrentTemperature. It requires OPENWEATHER_API_KEY and demonstrates:

• Fetching an external REST API (axios).
• Converting units based on tool arguments.
• Supplying instructions so the LLM only calls the tool when the user explicitly asks about weather.

Use it as inspiration for production tools or swap in your own factory.

Streaming, Indicators, and Cancellation

• Messages stream via LangChain’s async generator interfaces, and the SDK pipes partial tokens through serverClient.ephemeralUpdateMessage for live typing, without writing every chunk to the database.
• ai_indicator.update events surface typing states: AI_STATE_THINKING, AI_STATE_GENERATING, AI_STATE_EXTERNAL_SOURCES, and AI_STATE_ERROR.
• ai_indicator.clear flushes once the final response lands.
• The SDK listens for ai_indicator.stop events and aborts the underlying LangChain run, which cancels streaming mid-flight.
• Once the response completes, the final text is stored through chatClient.partialUpdateMessage, so the full reply shows up in channel history. See Best practices for why the last update is never ephemeral.

Memory & Mem0

Mem0 adds “long-term” recall for user+channel pairs. Configuration layers:

1. Optional MEM0_CONFIG_JSON (stringified JSON) provides defaults such as metadata.
2. Per-process env fallbacks: MEM0_DEFAULT_USER_ID, MEM0_DEFAULT_AGENT_ID, MEM0_DEFAULT_APP_ID.
3. Agent option mem0Context lets you override channelId, userId, appId, agentId, or supply configOverrides.
4. During runtime the SDK augments metadata with Stream-specific IDs.

Mem0 only runs when the chosen provider matches openai, anthropic, or google and MEM0_API_KEY is set. The SDK turns Mem0 off for you when it generates summaries, so they stay fast and deterministic. You don't need to pass anything.

Summaries and Titles

Use agent.summarize(text) (instance) or Agent.generateSummary(text, platform, model?) (static) to request a short, six-word “title” summarizing arbitrary text, which is ideal for channel list previews, inbox cards, or push notification headers. The helper:

• Spins up a lightweight LangChain chat model using the current provider settings.
• Disables Mem0 to avoid unnecessary persistence.
• Normalizes the resulting string by trimming surrounding quotes.

const headline = await agent.summarize(latestTranscript);
await channel.updatePartial({ set: { subtitle: headline } });