Evaluations

Evaluations let you run a fixed batch of content through a moderation policy and see how it is handled. Re-run the same set after you change rules, filters, or LLM settings to see what improved or drifted.

Use evaluations in the dashboard (Moderation → Configuration → Evaluations) or from the server-side SDKs.

Concepts

Replay vs a single policy

When you create a set you pick how rows are executed:

• A specific policy. Every item runs against that policy's current configuration.
• Replay. Omit config_key. Seeded items keep the policy they originally ran under. Deleted policies are skipped for those rows.

Pasted (hand-written) content has no original policy, so it always needs a policy selected.

How scoring works

1. The first completed run is the baseline. It is not scored against anything.
2. Later runs compare each row's labels and recommended action to the baseline (or, for replay sets seeded from production, to the original production outcome stored on the row).
3. Rows with no expectations have empty expected labels and an empty expected action. They are safe or unlabeled rows you included so you can see what the policy did. They are not scored, and they stay out of the totals.

A difference from the baseline is a change, not automatically a mistake. Tightening a policy on purpose shows up the same way. The API records label_mismatch or action_mismatch on failure_reason, and passed: false, for that change. provider_error means the check failed. policy_not_found means the row's policy was deleted.

API

These methods are server-side only. They are not available from client-side SDKs.

Create an evaluation set

POST /api/v2/moderation/policy_tests/sets

Provide either rows or seed, not both. Names are unique per app.

The example below samples the newest production content for a policy. To paste items instead, pass rows (each with a text field). To replay each item under its original policy, omit config_key.

response = client.moderation.create_policy_test_set(
  GetStream::Generated::Models::CreatePolicyTestSetRequest.new(
    name: "Hate-speech sample",
    config_key: "chat:messaging",
    seed: GetStream::Generated::Models::PolicyTestSeedSpec.new(
      labels: [],
      limit: 100,
    ),
  )
)

Request parameters

seed fields:

Row fields (when using rows, and also returned on seeded sets):

Response

If seeding matches no production data, the request fails. Broaden labels or moderate more content first.

List evaluation sets

GET /api/v2/moderation/policy_tests/sets

Each set in sets may include last_run so you can show last-run status and pass rate without a second request.

response = client.moderation.list_policy_test_sets(limit: 50)

Request parameters

Response

Get an evaluation set

GET /api/v2/moderation/policy_tests/sets/{id}

Returns set (including rows), recent_runs (newest first, capped at 10), and baseline_run_id once the first run has completed.

response = client.moderation.get_policy_test_set("pts_...")

Request parameters

Response

Delete an evaluation set

DELETE /api/v2/moderation/policy_tests/sets/{id}

Cascades to runs and results. Fails if a run is still pending or running.

client.moderation.delete_policy_test_set("pts_...")

Request parameters

Start a run

POST /api/v2/moderation/policy_tests/sets/{id}/runs

The body is empty. Only one run may be in progress per set. When a new run is created, the oldest terminal runs beyond the 10-run cap are deleted.

response = client.moderation.start_policy_test_run("pts_...")

Request parameters

Response

Get a run (and results)

GET /api/v2/moderation/policy_tests/runs/{id}

Poll this while status is pending or running. rows_completed / rows_total track progress. Per-row results are omitted until the run is terminal, so polling stays cheap.

response = client.moderation.get_policy_test_run("ptr_...")

Request parameters

Response

Run object

metrics.totals:

metrics.by_label maps each label to { same, changed }. changed counts rows where that label was added or dropped versus the baseline. Rows where the label was absent in both runs are not counted.

Result object (terminal runs)

Compare two runs client-side: fetch both, join on row_index.

Billing and retention

Each item in a run executes real moderation calls (LLM, NLP, image checks, and so on) at standard rates, the same as production traffic.

Each set keeps the last 10 completed runs. Starting another after that deletes the oldest run and its per-row results. In-flight runs are not deleted.