Policies

Overview

Policies define how content is moderated. A policy names the engines that run, the labels those engines look for, and the action to take on a match. Each policy has a key. A team can have its own policy.

After you change a policy, use Evaluations to re-run a fixed batch of content and see what improved or drifted.

Upsert Config

Creates or updates a policy. If a policy with that key already exists, it is replaced. A policy can include any combination of engines: NLP text, LLM text, images, video files, audio, blocklists, flood detection, and live video rules.

client.moderation.upsert_config(
  GetStream::Generated::Models::UpsertConfigRequest.new(
    key: "my_config",
    ai_text_config: GetStream::Generated::Models::AITextConfig.new(
      rules: [
        { label: "SPAM", action: "flag" },
        {
          label: "HATRED",
          severity_rules: [
            { severity: "low", action: "flag" },
            { severity: "high", action: "remove" },
          ],
        },
      ],
    ),
    block_list_config: GetStream::Generated::Models::BlockListConfig.new(
      rules: [GetStream::Generated::Models::BlockListRule.new(name: "profanity_en", action: "remove")],
    ),
    ai_image_config: GetStream::Generated::Models::AIImageConfig.new(
      rules: [GetStream::Generated::Models::AWSRekognitionRule.new(label: "Non-Explicit Nudity", action: "flag")],
    ),
  )
)

Request Parameters

key	required	type	description
key	true	string	Unique identifier for the policy. Send it as `config_key` on a check.
team	false	string	Team identifier for multi-tenancy. Scopes the policy to a specific team.
async	false	boolean	When true, checks that use this policy run asynchronously.
ai_text_config	false	object	Configuration for AI text analysis. Define rules mapping harm labels to actions.
ai_image_config	false	object	Configuration for AI image moderation (e.g., nudity, violence detection).
ai_video_config	false	object	Configuration for AI video moderation.
ai_audio_config	false	object	Configuration for AI audio moderation.
block_list_config	false	object	Which blocklists to use and the action for each.
llm_config	false	object	LLM text moderation. See LLM config.
flood_config	false	object	Configuration for per-user flood detection (identical and similar message detectors).

Get Config

Retrieve one policy by its key, including every engine and rule on it.

client.moderation.get_config("my_config")

Request Parameters

key	required	type	description
key	true	string	The policy key.
team	false	string	Team identifier for multi-tenancy.

Response

key	type	description
config	object	The policy. The response field name is `config`.

Delete Config

Delete a policy. Checks that name this key no longer use it.

client.moderation.delete_config("my_config")

Request Parameters

key	required	type	description
key	true	string	The policy key.
team	false	string	Team identifier for multi-tenancy.

Query Configs

List policies, with sorting and pagination.

client.moderation.query_moderation_configs(GetStream::Generated::Models::QueryModerationConfigsRequest.new(
  filter: {},
  sort: [GetStream::Generated::Models::SortParam.new(field: "created_at", direction: -1)],
  limit: 10
))

Request Parameters

key	required	type	description
filter	false	object	Filter conditions.
sort	false	array	Sort parameters (e.g., by created_at).
limit	false	number	Maximum number of policies to return.
next	false	string	Cursor for pagination.

Response

key	type	description
configs	array	Policies. The response field name is `configs`.
next	string	Next cursor for pagination.

Policy fields

A policy can include any of these engines. The field names are the API names.

Config Field	Description	Engine Documentation
ai_text_config	NLP text classification. Each rule is a `label` and an `action`, for example `INSULT` or `HATRED`.	AI Text
llm_config	LLM classification. Each rule has a `label`, a `description`, and an `action`.	LLM Text
ai_image_config	Image classification. Every rule requires `label`, `action`, and `min_confidence` (0–100). `ocr_rules` classifies text found in the image.	Image Moderation
ai_video_config	Video file classification. Uses the same rule shape as images. Live video uses `video_call_rule_config`.	Video Moderation
ai_audio_config	Audio transcription plus `ai_text_config` or `llm_config` on the transcript. The top-level `rules` list is the legacy fallback.	Audio Moderation
block_list_config	Blocklist and regex filtering. The field name is `block_list_config`.	Blocklists and Regex Filters
flood_config	Per-user detection of repeated or near-duplicate messages	Flooding
rule_builder_config	At most 3 rules. Each rule or group has at most 5 conditions.	Rules
intent_config	Up to 20 conversation topics. A match sends `moderation.intent_detected`.	Webhooks
video_call_rule_config	Harm rules for live video and audio. The field name is `video_call_rule_config`.	Live video and audio

LLM config

llm_config sets up the AI Text (LLM) engine. Upserting the policy replaces the whole llm_config, so send every field you want to keep.

key	type	description
app_context	string	Short description of your application and how users interact, so the LLM understands the setting.
rules	array	Harm labels. Each rule has a `label` (required), a plain-language `description` of what to detect (optional; without it the LLM uses the label name alone), an `action` (`flag`, `shadow`, `remove`, `bounce`, `bounce_flag`, `bounce_remove` or `keep`) and optional `severity_rules`. The engine runs when `rules` contains at least one rule with an action.
severity_descriptions	object	Optional. What each severity means to the LLM, keyed `LOW`, `MEDIUM`, `HIGH` and `CRITICAL`. When omitted, Stream fills in defaults, for example `LOW` is "Content that needs review but may be acceptable" and `HIGH` is "Content that must be removed immediately".
async	boolean	Optional. Runs LLM moderation asynchronously. Defaults to `false`: the check waits for the LLM verdict.
context_custom_keys	array of strings	Top-level `moderation_payload.custom` keys passed to the LLM as background context. Never moderated themselves. Up to 10 keys, each 1-64 characters. See Context from custom data.
custom_keys_context_description	string	How the LLM should read the context keys. Up to 1000 characters (UTF-8 bytes). Requires `context_custom_keys`.

LLM severity rules

severity_rules is an array of { severity, action }, with severity one of low, medium, high or critical. When the verdict's severity has a severity rule, that action is used; otherwise the rule's own action applies. For example, { "severity": "low", "action": "flag" } sends low-severity matches to review instead of removing them.

Intent config

intent_config is a list of conversation topics, up to 20. Each topic has a label and an optional description. A topic does not take an action by itself. A rule with an intent_rule condition names the topic, the score that counts as a match, and the action. A match sends moderation.intent_detected. See Webhooks.

Rule actions

These are the actions an engine rule can return. The API accepts the ones marked for that engine.

Action	What it does	NLP text	LLM	Image, video, OCR	Blocklist
`keep`	Leave the content in place.	yes	yes
`flag`	Queue it for review.	yes	yes	yes	yes
`mask`	Replace the matched text.	yes			yes
`mask_flag`	Replace the matched text and queue it.	yes			yes
`shadow`	Show it only to the author.	yes	yes	yes	yes
`remove`	Do not publish it, and queue it.	yes	yes	yes	yes
`bounce`	Reject it before it is published.	yes	yes	yes	yes
`bounce_flag`	Reject it and queue it.	yes	yes	yes	yes
`bounce_remove`	Reject it, treat it as removed, and queue it.	yes	yes	yes	yes

queryModerationRules returns default_llm_labels and ai_image_label_definitions. Use those strings as rule labels. SPAM is a valid NLP label. HARASSMENT is not; use HATRED, INSULT, SEXUAL_HARASSMENT, or MORAL_HARASSMENT.

Severity Levels

For AI text analysis rules, you can define severity-based rules instead of a single action. This allows different actions depending on how severe the detected violation is. The available severity levels are:

Severity	Description
`low`	Minor or borderline violations. Typically used for flagging or monitoring.
`medium`	Moderate violations that may warrant content removal or closer review.
`high`	Serious violations that typically result in content removal.
`critical`	The most severe violations requiring immediate action.

When using severity rules, each severity level can be mapped to a different action. For example, you might flag low-severity harassment but automatically remove high-severity harassment, as shown in the Upsert Config example above.