Memory and persistence

When you use LLMs directly they all have a different approach to memory and chat persistence.

Here we standardize on a simple approach

  • By default a session always retains its history/context
  • To disable that behaviour, set a new session to incognito mode
  • Memory is stored at an app + user id level

Let's go over a few examples

Authenticated user

Open a session with memory on and tell the agent your shoe size. A week later, in a new session, it still knows. A client remembers for the user it signed in; a server names the user.

Not documented for Flutter yet. Showing JavaScript.

JavaScript
import { Client, type Session } from "@stream-io/stream-agents";

const client = new Client({ apiKey: "your_api_key" });
await client.setUser("user-123", () =>
  fetch("/api/agent-token").then((response) => response.text()),
);
const support = client.agent("support");
// Remembers for the user setUser named.
const memory = {};

const monday = await support.sessions.create({ memory });
await ask(monday, "I wear a size 10 shoe.");
await monday.close();

const friday = await support.sessions.create({ memory });
console.log(
  await ask(friday, "What size should I order in these running shoes?"),
);
// Size 10, going by what you told me before.

async function ask(session: Session, text: string): Promise<string> {
  await session.responses.create(text);
  let answer = "";
  for await (const event of session.events()) {
    if (event.kind === "response_delta") answer += event.text;
    if (event.kind === "responded") break;
  }
  return answer;
}
Kotlin
val agents = StreamAgents(apiKey = "your_api_key")
agents.setUser(User(id = "user-123")) { yourBackend.agentToken() }
val support = agents.agent("support")
// Remembers for the user setUser named.
val options = SessionOptions(memory = SessionOptions.Memory())

val monday = support.chat(options)
ask(monday, "I wear a size 10 shoe.")
monday.close()

val friday = support.chat(options)
println(ask(friday, "What size should I order in these running shoes?"))
// Size 10, going by what you told me before.

suspend fun ask(session: AgentSession, text: String): String = coroutineScope {
    val answer = async(start = CoroutineStart.UNDISPATCHED) {
        session.events()
            .takeWhile { it.type != "responded" }
            .filter { it.type == "response_delta" }
            .fold("") { soFar, event -> soFar + event.text }
    }
    session.responses.create(text)
    answer.await()
}
Swift
let agents = StreamAgents(apiKey: "your_api_key")
agents.setUser(User(id: "user-123")) { try await yourBackend.agentToken() }
let support = agents.agent("support")
var options = SessionOptions()
// Remembers for the user setUser named.
options.memory = SessionOptions.Memory()

let monday = try await support.sessions.create(options)
_ = try await ask(monday, "I wear a size 10 shoe.")
await monday.close()

let friday = try await support.sessions.create(options)
print(try await ask(friday, "What size should I order in these running shoes?"))
// Size 10, going by what you told me before.

@MainActor
func ask(_ session: AgentSession, _ text: String) async throws -> String {
    await session.start()
    let events = session.events()
    _ = try await session.responses.create(text)
    var answer = ""
    for await event in events {
        if event.kind == .responseDelta { answer += event.text }
        if event.kind == .responded { break }
    }
    return answer
}

What the agent learns is saved as the conversation goes, and recalled when a session opens. Every memory is kept under your app and the user_id, so one person's memories never reach another. In a client, pass memory: {}: it remembers for the user setUser named, since a client can't name anyone else. A server names whichever of its users it means, or the one it acts for when it names none.

Incognito

An incognito session saves nothing. Tell it your shoe size and the next session has no idea, and the conversation itself is not stored anywhere.

Not documented for Flutter yet. Showing JavaScript.

JavaScript
const memory = { user_id: "user-456" };

const secret = await support.sessions.create({ memory, incognito: true });
await ask(secret, "I wear a size 10 shoe.");
await secret.close();

const later = await support.sessions.create({ memory });
console.log(
  await ask(later, "What size should I order in these running shoes?"),
);
// I don't know your shoe size yet. What size do you wear?

await support.sessions.get(secret.id); // throws: not found
Kotlin
agents.setUser(User(id = "user-456")) { yourBackend.agentToken() }
// Remembers for user-456, the user setUser named.
val options = SessionOptions(memory = SessionOptions.Memory())

val secret = support.chat(options.copy(incognito = true))
ask(secret, "I wear a size 10 shoe.")
secret.close()

val later = support.chat(options)
println(ask(later, "What size should I order in these running shoes?"))
// I don't know your shoe size yet. What size do you wear?

support.sessions.get(secret.id) // throws: not found
Swift
agents.setUser(User(id: "user-456")) { try await yourBackend.agentToken() }
var options = SessionOptions()
// Remembers for user-456, the user setUser named.
options.memory = SessionOptions.Memory()

var secretOptions = options
secretOptions.incognito = true
let secret = try await support.sessions.create(secretOptions)
_ = try await ask(secret, "I wear a size 10 shoe.")
await secret.close()

let later = try await support.sessions.create(options)
print(try await ask(later, "What size should I order in these running shoes?"))
// I don't know your shoe size yet. What size do you wear?

_ = try await support.sessions.get(secret.id) // throws: not found

An incognito session has no transcript, so it can't be listed, searched, forked or rewound. It still recalls what earlier sessions learned about the person; it just adds nothing.

Forgetting

Ending a session keeps what it learned. To forget one session, delete its memories. To forget a person, truncate their memories.

Warning:

Truncating a person's memories deletes everything learned about them, in every session and by every agent.

Not documented for Python yet. Showing Node.js.

Not documented for Ruby yet. Showing Node.js.

Not documented for PHP yet. Showing Node.js.

Not documented for .NET yet. Showing Node.js.

Not documented for Rust yet. Showing Node.js.

Node.js
import { Client } from "@stream-io/stream-agents";

const api = new Client();

await api.agent("support").sessions.deleteMemories(sessionId);
await api.memories.truncate("user-123");
Go
api, err := client.New(stream.Backend{})
if err != nil {
	return err
}
if err := api.Agent("support").Sessions.DeleteMemories(ctx, sessionID); err != nil {
	return err
}
if err := api.Memories().Truncate(ctx, "user-123"); err != nil {
	return err
}