Memory & Persistence

When you use LLMs directly they all have a different approach to memory and chat persistence.

Here we standardize on a simple approach

  • By default a session always retains its history/context
  • To disable that behaviour, set a new session to incognito mode
  • Memory is stored at an app + user id level

Let's go over a few examples

Authenticated User

Open a session with a user_id and tell the agent your shoe size. A week later, in a new session, it still knows.

import { Client, type Session } from "@stream-io/calypso";

const support = new Client().agent("support");
const memory = { user_id: "user-123" };

const monday = await support.sessions.create({ memory });
await ask(monday, "I wear a size 10 shoe.");
await monday.close();

const friday = await support.sessions.create({ memory });
console.log(
  await ask(friday, "What size should I order in these running shoes?"),
);
// Size 10, going by what you told me before.

async function ask(session: Session, text: string): Promise<string> {
  await session.responses.create(text);
  let answer = "";
  for await (const event of session.events()) {
    if (event.kind === "response_delta") answer += event.text;
    if (event.kind === "responded") break;
  }
  return answer;
}

What the agent learns is saved as the conversation goes, and recalled when a session opens. Every memory is kept under your app and the user_id, so one person's memories never reach another. In a browser, as on a phone, pass memory: {}: it remembers for the user setUser named, since a user's device can't name anyone else. A backend names whichever of its users it means, or the one it acts for when it names none.

Incognito

An incognito session saves nothing. Tell it your shoe size and the next session has no idea, and the conversation itself is not stored anywhere.

const memory = { user_id: "user-456" };

const secret = await support.sessions.create({ memory, incognito: true });
await ask(secret, "I wear a size 10 shoe.");
await secret.close();

const later = await support.sessions.create({ memory });
console.log(
  await ask(later, "What size should I order in these running shoes?"),
);
// I don't know your shoe size yet. What size do you wear?

await support.sessions.get(secret.id); // throws: not found

An incognito session has no transcript, so it can't be listed, searched, forked or rewound. It still recalls what earlier sessions learned about the person; it just adds nothing.

Forgetting

Ending a session keeps what it learned. To forget one session, delete its memories. To forget a person, truncate their memories: everything learned about them, in every session and by every agent, is deleted.

import { Client } from "@stream-io/calypso";

const api = new Client();

await api.agent("support").sessions.deleteMemories(sessionId);
await api.memories.truncate("user-123");