Content

Send content to Stream and get a keep, flag, or remove decision. There are two endpoints. Both use the same policies and rules.

Which should I use?

Check Analyze
Start here if You have image URLs and want one action You need a verdict per field, or you have image bytes
Request client.moderation.check client.moderation.analyze
Images Public URLs in moderation_payload.images Raw bytes via HTTP multipart (image:<label>). Not in the SDK
Texts A list of strings A map of named fields (title, caption, …)
Result One recommended_action for the entity Per-field action on texts and images
SDKs All server-side SDKs All server-side SDKs for texts. Image bytes are HTTP only
Review queue Yes Stateful mode only (when you send entity fields)

Use Custom Check when your own model already classified the content and you want that result on the dashboard.

Examples below use the server-side SDKs. See the setup guides for Node, Python, Go, Java, PHP, Ruby, or .NET.

Check

Submit text, image URLs, and/or video URLs. Get back one recommended_action (keep, flag, or remove) and a review-queue item you can show on a custom dashboard.

  1. Create a policy with engines and actions
  2. Call check with your entity ids and config_key
  3. Apply recommended_action in your product
  4. Review flagged items in the dashboard or via the review queue API

API Usage

Info:

In the context of the Moderation API, the term "entity" refers to a piece of content that you send for moderation. This can be text, an image, and/or a video.

response, err := client.Moderation().Check(ctx, &getstream.CheckRequest{
    EntityType:      "entity_type",
    EntityID:        "entity_id",
    EntityCreatorID: "entity_creator_id",
    ModerationPayload: &getstream.ModerationPayload{
        Texts:  []string{"this is bullshit", "f*ck you as***le"},
        Images: []string{"example.com/test.jpg"},
    },
    ConfigKey: getstream.PtrTo("config_key"),
    Options:   getstream.PtrTo(map[string]any{"force_sync": true}),
})

Request Params

key required type description
entity_type true string This is identifier for the type of content you are sending for moderation. This helps with categorizing content on dashboard. E.g., if you have multiple products then you can set unique entity_type for content coming from each product. It could be any string.
entity_id true string Unique identifier for entity
entity_creator_id true string Unique identifier for user who created this entity. Generally this is the user id of the app user.
moderation_payload true object Entity or content to be moderated: texts, images, videos, and custom metadata. For comments, custom can carry comment context for the review queue
config_key true string Key of the config to be used for moderation. See Configuration for details.
country_code false string ISO 3166-1 alpha-2 (for example US, DE). Invalid values return 400. Gives the AI text engine country context for the post.

Response

key type description
status “complete” , “partial” Status of moderation. In case you have configured both synchronous and asynchronous (image moderation) moderations, then status will be “partial”. This is because async moderations are still running in background task.
task_id string Id of the task running the async moderation. You can check the status of the task itself using GetTask endpoint.
recommended_action “flag” , “remove” , “keep” Final result of moderation which suggest what action should be taken for moderated entity.
  • “flag” suggests that the content needs manual review either on Stream dashboard or custom dashboard you may have
  • “remove” suggests that this content should be removed from the platform. You will be able to access/review this content from Stream dashboard
  • “keep” suggests that this content is safe and doesn’t contain any harms
item object This is basically a json representation of the review queue item accessible on Stream dashboard. You can use this to compose your own dashboard UI.

Comment context in the review queue

When the entity is a comment, moderators decide faster if they can see what it was posted on and the conversation before it. Send that context in moderation_payload.custom and the dashboard's review queue shows it next to the flagged comment:

  • A content card: what the comment was posted on (type, id, title, tags, subtitle, description, link, thumbnail)
  • The thread: the top-level comment, the latest earlier replies and the moderated comment, highlighted with its action (Flagged, Removed, Shadow blocked, Blocked)

The format works for any product: a video, a news article, a marketplace listing. The dashboard recognizes it by shape, not by entity_type: a numeric schemaVersion marks it. Without one, custom shows as a plain list of keys and values.

await client.moderation.check({
  entity_type: "comment",
  entity_id: "a_r7",
  entity_creator_id: "driver_dan",
  moderation_payload: {
    texts: ["Cyclists can all go back where they came from"],
    custom: {
      schemaVersion: 1,
      depth: 1,
      content: {
        id: "news-48213",
        type: "article",
        title: "City council approves new bike lanes downtown",
        subtitle: "Local news",
        tags: [
          { key: "section", label: "Section", values: ["Transport"] },
          {
            key: "topics",
            label: "Topics",
            values: ["City council", "Cycling"],
          },
        ],
        description: "The plan adds 12 km of protected lanes over two years...",
        url: "https://example.com/news/bike-lanes",
        imageUrl: "https://example.com/news/bike-lanes.jpg",
      },
      thread: {
        topLevel: {
          id: "a_top",
          user: { id: "cyclist_jen", name: "Jen" },
          text: "Finally. I have been waiting for this for years.",
          createdAt: "2026-10-06T08:12:00Z",
        },
        earlierReplyCount: 7,
        earlierReplies: [
          {
            id: "a_r6",
            user: { id: "driver_dan", name: "Dan" },
            text: "Good luck finding parking now.",
            createdAt: "2026-10-06T08:40:00Z",
          },
        ],
      },
    },
  },
  config_key: "config_key",
});

The moderated comment itself is not part of custom: its text comes from texts and its author from entity_creator_id. Keys inside custom are camelCase.

key type description
schemaVersion positive integer Required. Version of this format, set to 1. Stream bumps it only when the shape changes. Newer versions still render, best-effort
contextVersion any Optional. Your own version of the context data, for your purposes. The dashboard never reads it
depth 0 or 1 0 for a top-level comment, 1 for a reply. Threads are one level deep. When omitted, the comment is treated as a reply if thread.topLevel is set
content.id string Your id for the content, shown next to the type so moderators can copy it
content.type string Free-form kind of content: video, article, listing, ...
content.title string Title of the content
content.subtitle string Shown after the tags, on one line under the title
content.tags array of tag groups { key, label, values }. Groups show on one line, separated by ·, values within a group by commas. Hovering a value shows its group label. key is not displayed
content.description string Longer description, truncated at 150 characters with a "Show more" toggle
content.url string Link to the content, opened from the card. Only http and https URLs are shown
content.imageUrl string Thumbnail. Only http and https URLs are shown
thread.topLevel comment The comment a reply sits under. Omit it for a top-level comment
thread.earlierReplies array of comments Replies posted before the moderated one, oldest first. Send the latest few, not the whole thread
thread.earlierReplyCount integer Total replies before the moderated one. When it is larger than earlierReplies, the dashboard shows "Showing last X of N earlier replies"
thread comment object Each thread comment has an id, a user ({ id, name, image }, name and image optional), its text and an optional createdAt timestamp (ISO 8601)

Every field except schemaVersion is optional, and unknown keys are ignored. A malformed tags value drops only the tags, never the rest of the context.

Analyze

Submit named text fields and get a keep / flag / remove decision per label. Use it when you need a verdict per field instead of one action for the whole entity.

Server-side only. Call analyze the same way as check for named texts. At least one text field or image is required.

Raw image bytes are not in the SDK. Send them with an HTTP multipart/form-data request: one JSON payload part plus image:<label> file parts. That is the only way to attach bytes today.

Modes

Stateful vs stateless. Send entity_type, entity_id, and entity_creator_id together to persist a review-queue row (when content is flagged) and evaluate rules. Omit all three for a stateless call: verdicts only, no queue, no webhooks. config_key is required when stateless. Partial entity fields return 400.

entity_type is yours (listing, external:call). Types that start with stream: are reserved. You may add a subtype for routing (external:call:keyframe); webhooks echo the canonical type (external:call).

Sync vs async. Default is sync: verdicts in the HTTP body. Set async_response: true (stateful only) to get status: "pending" and receive moderation.text_analysis.complete / moderation.image_analysis.complete webhooks instead. Do not send async_response: true without entity fields.

Calling Analyze

response, err := client.Moderation().Analyze(ctx, &getstream.AnalyzeRequest{
    EntityType:      getstream.PtrTo("listing"),
    EntityID:        getstream.PtrTo("listing_42"),
    EntityCreatorID: getstream.PtrTo("user_42"),
    ConfigKey:       getstream.PtrTo("catalog-v1"),
    ContentIds:      map[string]string{"caption": "cap_42"},
    Texts:           map[string]string{"caption": "Vintage watch, mint condition"},
})

Image bytes (HTTP only)

client.moderation.analyze sends the JSON fields above. It cannot attach files. To send raw image bytes, POST multipart/form-data yourself (max 10 images, 250 KB each by default, 20 MB for the whole request):

curl -X POST "https://chat.stream-io-api.com/api/v2/moderation/analyze?api_key=${API_KEY}" \
  -H "Authorization: ${TOKEN}" \
  -H "stream-auth-type: jwt" \
  -F 'payload={"entity_type":"listing","entity_id":"listing_42","entity_creator_id":"user_42","config_key":"catalog-v1","texts":{"caption":"Vintage watch, mint condition"},"content_ids":{"caption":"cap_42","main_photo":"img_42_main"}};type=application/json' \
  -F "image:main_photo=@frame.jpg;type=image/jpeg"

Analyze request

key required type description
entity_type with the other entity fields string Caller-defined type. Required with entity_id and entity_creator_id. Omit all three for stateless mode
entity_id with the other entity fields string Your identifier for this content
entity_creator_id with the other entity fields string User id of the content creator
config_key stateless: yes string Policy key. Optional in stateful mode (resolved from entity_type). Unknown keys return 422
texts no object Named text fields, keyed by your label. Max 20 keys, each 1 to 64 characters
content_ids no object Per-field ids keyed by the same labels as texts or image:<label>. Echoed as id on each verdict. Values: 1 to 128 chars, A-Z a-z 0-9 . _ -
content_published_at no string (RFC3339) When the content was produced. Used as the score for aggregation-rule windows
custom no object Metadata stored on the review-queue item and echoed on webhooks
async_response no bool Stateful only. When true, HTTP is pending and verdicts arrive via webhooks
country_code no string ISO 3166-1 alpha-2 (for example US, DE). Invalid values return 400. Gives the AI text engine country context for the post.

Images: HTTP only. Each file part is named image:<label>. Max 10 images, 250 KB each by default, 20 MB for the whole request. The SDK analyze method does not accept file parts.

Analyze response

Successful calls return 201.

key type description
status complete, partial, pending complete if every field was screened. partial if some have error. pending is the async ack
texts object Per-text verdicts keyed by your label. Absent on async pending
images object Per-image verdicts keyed by your label
duration string Server wall-clock duration

Each text or image field:

key description
id Echo of content_ids[<label>] when you sent one
action keep, flag, or remove. Omitted when error is set. Never treat a missing action as keep
classifications Text: { name, severity }. Image: { name, confidence, subclassifications }
severity Text only. Highest severity across classifications (low, medium, high, critical)
language Text only. Detected language code
confidence Image only. Highest confidence (0 to 1)
ocr_classifications Image only. Text labels extracted from the image (name, severity)
error Field could not be screened. Retry this label only
{
  "status": "complete",
  "texts": {
    "caption": {
      "id": "cap_42",
      "action": "flag",
      "classifications": [{ "name": "INSULT", "severity": "high" }],
      "severity": "high",
      "language": "en"
    }
  },
  "images": {
    "main_photo": {
      "id": "img_42_main",
      "action": "keep"
    }
  }
}

If some fields fail, status is partial (still 201). Retry only the labels with error. If nothing could be screened, the API returns 503 with a standard error envelope, not this body. Bad input is 400. A missing policy is 422.

Async pending body:

{ "status": "pending", "duration": "112.84ms" }

Wait for Analyze webhook events. A provider error stays on *.complete as field error. moderation.analysis.failed means Stream could not finish the listed content_ids after the pending ack. Treat those ids as unscreened.

Custom Check

Custom Check enables you to submit your own AI moderation results of a chat message/custom uploaded content for review by moderators on the Dashboard.

Add Custom Flags to a Chat Message

Custom Flags can be added to a Chat message. This enables you to submit your own AI moderation results of a chat message for review by moderators on the Dashboard.

response, err := client.Moderation().CustomCheck(ctx, &getstream.CustomCheckRequest{
    EntityType:      "stream:chat:v1:message",
    EntityID:        messageID,
    EntityCreatorID: "user_id",
    Flags: []getstream.CustomCheckFlag{
        {Type: "custom_check_image", Reason: getstream.PtrTo("Image was NSFW"), Labels: []string{"NSFW"}},
        {Type: "custom_check_text", Reason: getstream.PtrTo("Text was harmful"), Labels: []string{"harmful"}},
        {Type: "custom_check_video", Reason: getstream.PtrTo("Video contains copyright material"), Labels: []string{"copyright-violation"}},
    },
})

Add Custom Check Flags to the Custom Content

The Custom Check endpoint enables you to submit your own AI moderation results for the custom content for review by moderators on the Dashboard.

response, err := client.Moderation().CustomCheck(ctx, &getstream.CustomCheckRequest{
    EntityType:      "entity_type",
    EntityID:        "entity_id",
    EntityCreatorID: "entity_creator_id",
    ModerationPayload: &getstream.ModerationPayload{
        Texts: []string{"offensive"},
    },
    Flags: []getstream.CustomCheckFlag{
        {
            Type:   "custom_check_text",
            Reason: getstream.PtrTo("Text was offensive"),
            Custom: map[string]any{"explaination": "custom is a nullable field allowing you to attach a custom object to the flag for your specific use cases or requirements"},
        },
    },
})

Request Params

key required type description
entity_type true string This is identifier for the type of the entity. This helps with categorizing Content.
entity_id true string Unique identifier for entity.
entity_creator_id true string Unique identifier for user who created this entity. Generally this is the user id of the app user.
moderation_payload true object Entity or content to be moderated
flags true array List of custom flags
flag.type true "custom_check_text", "custom_check_image", "custom_check_video" Type of a custom flag

Response

key type description
status “complete” , “partial” Status of moderation. In case you have configured both synchronous and asynchronous (image moderation) moderations, then status will be “partial”. This is because async moderations are still running in background task.
id string id of the review queue item.
item object This is basically a json representation of the review queue item accessible on Stream dashboard. You can use this to compose your own dashboard UI.