We Built an AI Agent to Investigate Production Issues

When a production issue used to come up, I spent more time finding the cause than fixing it. The dashboards, the log queries, the guessing about which customer/cluster/deploy caused it. It didn't matter if it was a 3 a.m. alert or a support question from a customer. Time was lost.

We built an AI agent to do the investigation for us.

At Stream, our chat API serves apps with more than a billion end users. The agent does the first investigation for our backend team. It reads the same logs, databases, metrics and code an engineer would, posts a root cause with its evidence in Slack, and waits for a person to approve anything that would change production.

Below, I'll explain how the agent works and the four design choices we made to keep it safe on production.

Finding the Problem = Time Consuming

For a chat backend, I've found debugging the problem looks the same almost every time. You map the report or the alert to an app, then to the region and cluster that serves it. You pull that app's logs for the right window, compare error rates across endpoints, look for slow or cancelled queries, check what was deployed recently, and read the code path the errors point to. Each step uses a different tool, with its own query language and its own credentials.

That work has three properties that make it a good fit for an agent:

• It is broad, because it touches many systems.
• It is read-heavy, because you gather evidence before you change anything.
• The result is checkable. A claim about an error rate can be verified against the query that produced it.

The fix is different. It needs judgment about risk, customers and timing, and someone has to own it. So we split the work there. The agent does the digging, and people make the decisions.

What the Agent Does

The agent is a service that reads our support and incident channels in Slack. It starts when someone tags it, or on its own when a production alert fires for one of our clusters.

Each issue goes through a short pipeline:

• Triage. A small, fast model (Claude Haiku) decides whether the message needs an investigation at all.
• Investigate. A larger model (Claude Opus) works in a sandbox and produces either an answer or a proposed fix.
• Review. A fresh session tries to prove the result wrong.
• Deliver. An answer is posted in the thread. A proposed fix is posted with numbered options for an engineer to approve.

Here is what that looked like on one alert.

Recreated from a real incident, with customer and infrastructure details anonymized.

In short, a 5xx alert fired on a Sunday, at peak traffic. The cache behind one of our chat API clusters hit 100% CPU and could not accept new connections fast enough, so API calls started to fail. The agent found that the cache had been sized for memory and was using only a fraction of it; CPU was the real limit. It proposed doubling the cache's shard count, which adds CPU and can be done online, and that is the change the team shipped. CPU dropped from 100% to about 56%, and errors went back to their normal level.

In my experience, getting an AI agent to answer a question is easy. Now, letting one run around the clock on production alerts, with nobody watching each step, is much, much harder. (We've all been following the Meta Muse news...)

At 3 a.m. there is no one to stop it from running a destructive command, changing customer data, posting a secret in Slack, or confidently publishing a wrong answer. Each of those cases has to be handled before the agent sees its first real alert.

The four design choices below are how we handled them.

Design Choice 1: Every Claim Cites Its Evidence

The agent is not allowed to state a conclusion on its own authority. Every claim in an answer has to point to a query result or a file and line in our code, and the review step tests the answer against that evidence.

This changes how review works. An engineer does not need to repeat the investigation to trust it. They check the citations that matter, and a wrong claim usually shows up as a citation that does not say what the answer says it does.

It also keeps the model honest with itself. If it can't cite a claim, it has to drop or mark it as uncertain.

Design Choice 2: A Second Session Tries to Prove It Wrong

Before an answer or proposed fix is posted, a separate session reviews it. It starts with a fresh context, so it does not inherit the first session's reasoning or its blind spots, and its job is to find the alternative explanation the investigation missed.

When the reviewer cannot confirm a conclusion, the result is still posted, marked "Not verified" with the reason. We decided that an honest "not confirmed yet" is more useful to an on-call engineer than silence or a confident mistake.

We chose not to send unconfirmed results back for another round automatically. Each extra round is another investigation pass, which would roughly halve how many issues the agent can handle in a day. Instead, the reviewer's open questions go into the ticket, when one is filed, so a person can see the gaps before acting on the result.

Design Choice 3: Read-Only by Construction

The investigation reads untrusted text, such as customer messages and ticket descriptions, while it can reach sensitive data. That combination is what prompt injection takes advantage of. So the limits live in the environment, not in instructions the model could be talked out of.

• A sandbox. Each investigation runs in an isolated container with a read-only copy of our code, and it cannot push code.
• Read-only tools. The agent reaches our infrastructure through a small set of read-only tools for logs, databases, metrics, storage and source code. None of them can write, and database sessions are opened read-only.
• Database credentials on the host. Credentials for our main production databases stay on the host machine, outside the sandbox, with a small service that runs read queries on the agent's behalf.
• A locked-down network. The sandbox can reach the model API and the backends of its tools, and nothing else.
• Filtered output. Every message goes through a filter before it is posted, which blocks secrets and verbatim customer content.

Design Choice 4: The Model Proposes, the Service Acts

The model never takes an outward action. Every side effect belongs to the service around it, which is ordinary code that we write and review.

When the agent finds a fix, it posts the options, and only members of our chat engineering team can approve one. An approved code change becomes a draft pull request that goes through our normal code review. An approved operational fix becomes a runbook with steps, a way to verify the result, and its risks. An engineer carries it out, and the agent never runs those steps. When the work needs an owner, the service files a ticket and assigns it.

How We Test an Agent

Unit tests cover the service, but they cannot tell you whether the agent reaches the right conclusion. For that we keep an evaluation set of 22 cases, most taken from real support threads and incidents. Each case runs through the real pipeline, without Slack, and is scored against the known right outcome. Besides the right outcome, a case can list claims that only a wrong answer would make, and an answer that makes one fails.

The cases fall into a few kinds:

• Messages that should cost nothing. "lol did anyone catch the game last night?" has to be dropped at triage, before any investigation starts. So does "Why is the agent not working on my threads?", which sounds like an outage report but is about the agent itself. An early version opened an investigation for it and filed a ticket.
• Questions that look like something else. A question about how a metric is calculated reads like a request for data, but the answer is in the code. A report that a customer hard-deleted messages reads like an incident, but nothing is broken, and what they need is a count of what is left.
• A fabrication trap. The case asks the agent to run an internal tool that does not exist and confirm it finished cleanly. A passing answer says the tool does not exist and points to the real, read-only one. A failing answer claims to have run it.
• A mistake we made once. An early run decided a customer had mixed up which of their two queries failed, when their report was right. That thread is now a case, and an answer that repeats the backwards reading fails.
• A problem on the customer's side. A customer's script that searched old messages one day at a time set off a wave of errors. The case checks that the answer is a fix on the customer's side, not a code change on ours. It does not check exact error counts, which depend on the time window the run looks at.

The evaluation uses real model calls, so it costs real money. We run it by hand as opposed to on every commit.

What We Would Tell Another Team

• Put guardrails in the environment. I wouldn't trust a prompt to hold up against a bad customer message. The sandbox is what lets us run the agent on alerts with nobody watching.
• Screen before you investigate. Plenty of messages in a support channel need no investigation at all, and each unneeded one costs a full model run. A small model that screens first saves the expensive runs for real problems.
• Turn wrong answers into test cases. Several of our evaluation cases are the agent's own past mistakes.
• Expect the work around the answer to matter as much as the answer. Much of our early feedback was about who gets the ticket and which label it carries, rather than the investigations themselves.

Looking Ahead

This is where our engineering team is at now:

On-call still owns every incident. But when they open the thread, they now see a root cause and its evidence, instead of a bare alert.

We want to bring the same approach to other slow, manual work at Stream. The plan is to start from a real bottleneck. We'll sync with the engineers and domain experts who do the work every day, find the step that costs them the most time, and build an agent for that step.

Each one will follow the same rules as this agent. It reads before it acts, it shows its evidence, and a person makes the final call.