Moderation APIs

Overview

Moderation APIs allow you to integrate moderation capabilities into your product or application. You can use a combination of these APIs along with Stream’s Moderation Dashboard to prevent app users from posting harmful content on your platform. Additionally, you can build a custom dashboard using these APIs.

Let’s examine each of the moderation APIs provided by Stream.

Prerequisites

Make sure you have created the app on Stream. If not, please refer to Guide to Getting Started with Stream
Following API documentation presents examples from our javascript client. Please refer to our guide on Stream API and client integration to setup client.

Moderation Check Endpoint

The Moderation Check Endpoint enables real-time content moderation by analyzing submitted content and providing actionable recommendations. This comprehensive API supports multiple content formats, including text messages, images, and video content.

The process follows these simple steps:

Set up your moderation policy with defined rules and actions (see Policy Setup)
Submit content for review along with your policy key
The API analyzes the content using configured moderation engines and rules
Receive an immediate recommendation based on the analysis

By integrating this endpoint into your application workflow, you can efficiently maintain content quality and ensure compliance with your platform’s standards while providing a safe environment for your users.

API Usage

In the context of the Moderation API, the term “entity” refers to a piece of content that you send for moderation. This can be text, an image, and/or a video.

await client.moderation.check(
  "entity_type",
  "entity_id",
  "entity_creator_id",
  {
    texts: ["this is bullshit", "f*ck you as***le"],
    images: ["example.com/test.jpg"],
  },
  "config_key",
  {
    force_sync: true,
  },
);

Request Params

key	required	type	description
entity_type	true	string	This is identifier for the type of content you are sending for moderation. This helps with categorizing content on dashboard. E.g., if you have multiple products then you can set unique entity_type for content coming from each product. It could be any string.
entity_id	true	string	Unique identifier for entity
entity_creator_id	true	string	Unique identifier for user who created this entity. Generally this is the user id of the app user.
moderation_payload	true	object	Entity or content to be moderated
config_key	true	string	Key of the config to be used for moderation. Please check Config Setup API for details

Response

key	type	description
status	“complete”, “partial”	Status of moderation. In case you have configured both synchronous and asynchronous (image moderation) moderations, then status will be “partial”. This is because async moderations are still running in background task.
task_id	string	Id of the task running the async moderation. You can check the status of the task itself using GetTask endpoint.
recommended_action	“flag”, “remove”, “keep”	Final result of moderation which suggest what action should be taken for moderated entity. “flag” suggests that the content needs manual review either on Stream dashboard or custom dashboard you may have “remove” suggests that this content should be removed from the platform. You will be able to access/review this content from Stream dashboard “keep” suggests that this content is safe and doesn’t contain any harms
item	object	This is basically a json representation of the review queue item accessible on Stream dashboard. You can use this to compose your own dashboard UI.

Upsert Config

We provide a separate API endpoint to enable or disable specific harm engines. Stream supports various moderation engines, both in-house and from subprocessors, such as toxicity detection, platform circumvention prevention, AI Text Engine, and AI Image Moderation Engine. We plan to introduce more in the future. Each engine processes content and provides relevant classifications. As a customer, you don’t need to handle each result individually—that would be overwhelming. Instead, we’ve simplified the process by allowing you to map these classifications to a few values representing “Recommended Actions”.

These recommended actions are handled automatically in Stream’s Chat and Activity Feeds products. For custom moderation, you’ll receive the final recommended action through the check endpoint (TODO: add link), allowing you to take appropriate action on the content.

API Usage

await client.moderation.upsertConfig({
  key: "unique_config_key",
  ai_text_config: {
    rules: [
	    {label: "SCAM", action: "flag"},
        {label: "SEXUAL_HARASSMENT", severity_rules [
            {severity: "low", action: "flag"},
            {severity: "medium", action: "flag"},
            {severity: "high", action: "remove"},
            {severity: "critical", action: "remove"}
        ]}
    ]
  },
  block_list_config: {
    rules: [
      {
        name: 'blocklistname',
        action: 'remove',
      },
    ],
  },
  automod_toxicity_config: {
    rules: [
		  { "label": "identity_attack", "threshold": 0.5, "action": "flag" },
		  { "label": "insult", "threshold": 0.5, "action": "remove" },
		  { "label": "obscene", "threshold": 0.5, "action": "flag" },
		  { "label": "sexual_explicit", "threshold": 0.5, "action": "flag" },
		  { "label": "threat", "threshold": 0.5, "action": "flag" }
		],
  },
  ai_image_config: {
    rules: [{ label: 'Non-Explicit Nudity', action: 'flag' }],
  },
  automod_platform_circumvention_config: {
	  rules: [{action: "remove", label:  "platform_circumvention", threshold: 0.5}]
  }
});

Request Parameters

Key	Required	Type	Description
key	true	string	Unique identifier for the config. You’ll need to specify this key when sending content for moderation check.
ai_text_config	false	object	Configuration for the AI Text engine.
block_list_config	false	object	Configuration for the block list. This allows you to disallow specific words or phrases on your platform.
automod_toxicity_config	false	object	Configuration for Stream’s internal toxicity engine.
ai_image_config	false	object	Configuration for image moderation. Please refer todocsfor a list of all the harms.
automod_platform_circumvention_config	false	object	Configuration for Stream’s platform circumvention engine. It helps prevent users from moving to different platforms for communication.

For out-of-the-box moderation in our Chat and Activity Feeds products, we use specific conventions for the configuration key:

Config key for a specific chat channel: chat:messaging:channel_id
Config key for a channel type: chat:messaging
Config key for all channels of any type: chat
Config key for the Activity Feeds product: feeds:default

Get Config

API Usage

You can retrieve the config for specific key as following

await client.moderation.getConfig("config_key");

Flag Content

Flagging content marks it for review by moderators. Flagged items appear on the Stream dashboard, where moderators can take appropriate action.

await client.moderation.flag(
  "entity_type",
  "entity_id",
  "entity_creator_id",
  "reason for flag",
  {
    custom: {},
    moderation_payload: { text: ["this is shit"] },
    user_id: "user_id", // only for server side usage
  },
);

Request Params

key	required	type	description
entity_type	true	string	This is identifier for the type of content you are sending for moderation. This helps with categorizing content on dashboard. E.g., if you have multiple products then you can set unique entity_type for content coming from each product. It could be any string.
entity_id	true	string	Unique identifier for entity
entity_creator_id	true	string	Unique identifier for user who created this entity. Generally this is the user id of the app user.
reason	true	string	Reason for flagging. This is necessary for any moderator to know the intent
options.moderation_payload	false	object	Content to be manually reviewed. This is used to display the content on dashboard for review.
options.custom	true	string	Any custom properties you may want to attach with this flag
options.user_id	false	string	Only needed if flag API is being used with server side usage.

Response

key	type	description
item_id	string	Id for the Review Queue Item which got created as a result of flagging. You can request the entire review queue object using GetReviewQueueItem endpoint [TODO: Add link]

Flag a user

When a user/moderator flags a user, it ends up in “Users” review list (as shown in following screen

await client.moderation.flagUser("target_user_id", "spam");

// Internally this method simply calls the flag endpoint as following
// await client.moderation.flag(
//	"stream:user",
//	"target_user_id",
//	"",
//	"spam",
//)

Flag a message

await client.moderation.flagMessage("message_id", "spam");

// Internally this method simply calls the flag endpoint as following
// await client.moderation.flag(
//	"stream:chat:v1:message",
//	"message_id",
//	"",
//	"spam",
//)

Query Review Queue

This endpoint allows you to query the review queue. It supports a wide range of filters and sorting options, enabling you to build a dashboard that’s easy for moderators to use.

const filters = { entity_type: "some_entity_type" };
const sort = [{ field: "created_at", direction: -1 }];
const options = { next: null }; // cursor pagination

const { items, next } = await client.moderation.queryReviewQueue(
  filters,
  sort,
  options,
);

Request Params

We support filtering by following params

key	type	description
id	string	Query items by specific id
created_at	datetime	Query review items created on specific date
updated_at	datetime
status	partial, complete	Query items for which moderation check has completed
entity_type	string	Query by entity type. E.g., Query message type review queue items: - entity_type: `stream:chat:v1:message` Query user type review queue items: - entity_type: `stream:user`
entity_id	string
entity_creator_id	string
user_report_reason	string	Query items which were flagged by users manually with given reason
reviewed	boolean	Query items which have been reviewed. Calling SubmitAction API on any item marks it as reviewed
has_text	boolean	Query items which has flagged text content
has_image	boolean	Query items which has flagged image content
has_video	boolean	Query items which has flagged video content
category	ai_image, automod_toxicity, automod_platform_circumvention, automod_semantic_filters, ai_text, block_list	Query items which are flagged by specific moderation engine.
label	string	Query items which were classified with certain label by moderation engine E.g., spam
reporter_type	moderator, user, automod	E.g., Query items which were flagged by user with moderator role
reporter_id	string	Query items which were flagged by specific user
date_range	string	Query items created between specific date range. E.g., `{ daterange: "2020-04-20T00:00:00_2035-04-20T00:00:00" }`
recommended_action	remove, flag, shadow_block	Recommended Action by AI moderation

We support following sort parameters

key	type	description
id	string
created_at	string
updated_at	string

Response

key	type	description
items	array	List of review queue items
next	string	Next cursor for pagination purpose
stats	object	Counts of pending review queue items. E.g., `{ texts: 30, users: 5, media: 10 }`

Sample Response

{
  "items": [
    {
      "id": "a68277a4-a5c4-4f74-a039-ad92f93ac827",
      "created_at": 1725276192603929000,
      "updated_at": 1725276210096123000,
      "entity_type": "stream:feeds:v2:activity",
      "entity_id": "80f650aa-692e-11ef-8080-80016e37cdb0",
      "moderation_payload": {
        "texts": [
          "fuck you asshole. I really like her. We had an amazing date, but I am not sure if I feel attracted to her. I am confused. She is just not smart enough for me and also her face is pretty average. I don't want ugly babies. What should I do?"
        ],
        "images": [
          "https://github.com/user-attachments/assets/30a86186-6f76-4881-935a-2c547d904ed3"
        ],
        "custom": {
          "feed": "user:1725276189"
        }
      },
      "moderation_payload_hash": "3281929468518332009",
      "has_image": true,
      "has_video": false,
      "has_text": true,
      "status": "complete",
      "recommended_action": "flag",
      "completed_at": "2024-09-02T11:23:30.096683Z",
      "languages": [],
      "severity": 0,
      "reviewed_at": null,
      "reviewed_by": "",
      "entity_creator": {
        "id": "user-1725276189-5",
        "role": "user",
        "custom": {
          "name": "Lucius Young"
        },
        "created_at": 1725276189438843000,
        "updated_at": 1725276189438843000,
        "banned": false,
        "online": false,
        "ban_count": 0,
        "deleted_content_count": 0
      },
      "bans": [],
      "flags": [
        {
          "type": "block_list",
          "entity_type": "stream:feeds:v2:activity",
          "entity_id": "80f650aa-692e-11ef-8080-80016e37cdb0",
          "moderation_payload": {
            "texts": [
              "fuck you asshole. I really like her. We had an amazing date, but I am not sure if I feel attracted to her. I am confused. She is just not smart enough for me and also her face is pretty average. I don't want ugly babies. What should I do?"
            ],
            "images": [
              "https://github.com/user-attachments/assets/30a86186-6f76-4881-935a-2c547d904ed3"
            ],
            "custom": {
              "feed": "user:1725276189"
            }
          },
          "moderation_payload_hash": "3281929468518332009",
          "labels": ["profanity_en_2020_v1"],
          "result": [
            {
              "text": "fuck you asshole. I really like her. We had an amazing date, but I am not sure if I feel attracted to her. I am confused. She is just not smart enough for me and also her face is pretty average. I don't want ugly babies. What should I do?",
              "action": "flag",
              "labels": ["profanity_en_2020_v1"],
              "provider_name": "block_list"
            }
          ],
          "review_queue_item_id": "a68277a4-a5c4-4f74-a039-ad92f93ac827",
          "created_at": 1725276192742036000,
          "updated_at": 1725276192742036000
        },
        {
          "type": "automod_toxicity",
          "entity_type": "stream:feeds:v2:activity",
          "entity_id": "80f650aa-692e-11ef-8080-80016e37cdb0",
          "moderation_payload": {
            "texts": [
              "fuck you asshole. I really like her. We had an amazing date, but I am not sure if I feel attracted to her. I am confused. She is just not smart enough for me and also her face is pretty average. I don't want ugly babies. What should I do?"
            ],
            "images": [
              "https://github.com/user-attachments/assets/30a86186-6f76-4881-935a-2c547d904ed3"
            ],
            "custom": {
              "feed": "user:1725276189"
            }
          },
          "moderation_payload_hash": "3281929468518332009",
          "labels": ["obscene", "insult"],
          "result": [
            {
              "text": "fuck you asshole. I really like her. We had an amazing date, but I am not sure if I feel attracted to her. I am confused. She is just not smart enough for me and also her face is pretty average. I don't want ugly babies. What should I do?",
              "action": "flag",
              "labels": ["obscene", "insult"],
              "provider_name": "automod_toxicity"
            }
          ],
          "review_queue_item_id": "a68277a4-a5c4-4f74-a039-ad92f93ac827",
          "created_at": 1725276192742038000,
          "updated_at": 1725276192742038000
        },
        {
          "type": "ai_text",
          "entity_type": "stream:feeds:v2:activity",
          "entity_id": "80f650aa-692e-11ef-8080-80016e37cdb0",
          "moderation_payload": {
            "texts": [
              "fuck you asshole. I really like her. We had an amazing date, but I am not sure if I feel attracted to her. I am confused. She is just not smart enough for me and also her face is pretty average. I don't want ugly babies. What should I do?"
            ],
            "images": [
              "https://github.com/user-attachments/assets/30a86186-6f76-4881-935a-2c547d904ed3"
            ],
            "custom": {
              "feed": "user:1725276189"
            }
          },
          "moderation_payload_hash": "3281929468518332009",
          "labels": ["INSULT", "SUPPORTIVE", "VULGARITY"],
          "result": [
            {
              "provider_name": "ai_text",
              "text": "fuck you asshole. I really like her. We had an amazing date, but I am not sure if I feel attracted to her. I am confused. She is just not smart enough for me and also her face is pretty average. I don't want ugly babies. What should I do?",
              "action": "flag",
              "labels": ["INSULT", "SUPPORTIVE", "VULGARITY"]
            }
          ],
          "review_queue_item_id": "a68277a4-a5c4-4f74-a039-ad92f93ac827",
          "created_at": 1725276192742039000,
          "updated_at": 1725276192742039000
        }
      ],
      "actions": [
        {
          "id": "aed07d1f-e3e8-4237-bec0-f8d9bf591280",
          "created_at": 1725276192749455000,
          "type": "flag",
          "user": null,
          "reason": "",
          "custom": {
            "flag_type": "ai_text"
          },
          "target_user_id": "user-1725276189-5",
          "review_queue_item_id": "a68277a4-a5c4-4f74-a039-ad92f93ac827"
        },
        {
          "id": "139b7c59-70c0-4e1d-96f0-e057a93a4833",
          "created_at": 1725276192749455000,
          "type": "flag",
          "user": null,
          "reason": "",
          "custom": {
            "flag_type": "automod_toxicity"
          },
          "target_user_id": "user-1725276189-5",
          "review_queue_item_id": "a68277a4-a5c4-4f74-a039-ad92f93ac827"
        },
        {
          "id": "b4660069-5855-4b20-a0fe-d769f190a3b7",
          "created_at": 1725276192749455000,
          "type": "flag",
          "user": null,
          "reason": "",
          "custom": {
            "flag_type": "block_list"
          },
          "target_user_id": "user-1725276189-5",
          "review_queue_item_id": "a68277a4-a5c4-4f74-a039-ad92f93ac827"
        }
      ],
      "content_changed": false
    },
    {
      "id": "947712cf-fd61-4fd9-bdf3-f2c9064977a9",
      "created_at": 1725276191973725000,
      "updated_at": 1725276191973725000,
      "entity_type": "stream:chat:v1:message",
      "entity_id": "d10120bc-f3c6-4c47-a981-3b4a7b1cc748",
      "moderation_payload": {
        "texts": [
          "This is my phone number. +91 9958592028. Contact me for more information on whatsapp :)"
        ],
        "images": [
          "https://github.com/user-attachments/assets/06d6c129-6b6e-4be2-94fd-c0f1cdd77278"
        ],
        "custom": {
          "original_message_type": "regular"
        }
      },
      "moderation_payload_hash": "17739892842151117680",
      "has_image": true,
      "has_video": false,
      "has_text": true,
      "status": "complete",
      "recommended_action": "remove",
      "completed_at": null,
      "languages": [],
      "severity": 0,
      "reviewed_at": null,
      "reviewed_by": "",
      "message": {
        "id": "d10120bc-f3c6-4c47-a981-3b4a7b1cc748",
        "text": "This is my phone number. +91 9958592028. Contact me for more information on whatsapp :)",
        "html": "<p>This is my phone number. +91 9958592028. Contact me for more information on whatsapp :)</p>\n",
        "type": "error",
        "user": {
          "id": "sophia2-76",
          "role": "user",
          "custom": {
            "name": "Sophia Bailey",
            "image": "https://xsgames.co/randomusers/assets/avatars/female/14.jpg"
          },
          "created_at": 1725275473485946000,
          "updated_at": 1725275473485946000,
          "banned": false,
          "online": false
        },
        "attachments": [
          {
            "type": "image",
            "image_url": "https://github.com/user-attachments/assets/06d6c129-6b6e-4be2-94fd-c0f1cdd77278",
            "asset_url": "https://github.com/user-attachments/assets/06d6c129-6b6e-4be2-94fd-c0f1cdd77278",
            "custom": {}
          }
        ],
        "latest_reactions": [],
        "own_reactions": [],
        "reaction_counts": {},
        "reaction_scores": {},
        "reaction_groups": {},
        "reply_count": 0,
        "deleted_reply_count": 0,
        "cid": "messaging:sample-app-channel-2",
        "created_at": 1725276192098227000,
        "updated_at": 1725276192098227000,
        "custom": {},
        "shadowed": false,
        "mentioned_users": [],
        "silent": false,
        "pinned": false,
        "pinned_at": null,
        "pinned_by": null,
        "pin_expires": null
      },
      "entity_creator": {
        "id": "sophia2-76",
        "role": "user",
        "custom": {
          "name": "Sophia Bailey",
          "image": "https://xsgames.co/randomusers/assets/avatars/female/14.jpg"
        },
        "created_at": 1725275473485946000,
        "updated_at": 1725275473485946000,
        "banned": false,
        "online": false,
        "ban_count": 0,
        "deleted_content_count": 0
      },
      "bans": [],
      "flags": [
        {
          "type": "automod_platform_circumvention",
          "entity_type": "stream:chat:v1:message",
          "entity_id": "d10120bc-f3c6-4c47-a981-3b4a7b1cc748",
          "moderation_payload": {
            "texts": [
              "This is my phone number. +91 9958592028. Contact me for more information on whatsapp :)"
            ],
            "images": [
              "https://github.com/user-attachments/assets/06d6c129-6b6e-4be2-94fd-c0f1cdd77278"
            ],
            "custom": {
              "original_message_type": "regular"
            }
          },
          "moderation_payload_hash": "17739892842151117680",
          "labels": ["platform_circumvention"],
          "result": [
            {
              "provider_name": "automod_platform_circumvention",
              "text": "This is my phone number. +91 9958592028. Contact me for more information on whatsapp :)",
              "action": "flag",
              "labels": ["platform_circumvention"]
            }
          ],
          "review_queue_item_id": "947712cf-fd61-4fd9-bdf3-f2c9064977a9",
          "created_at": 1725276192082384000,
          "updated_at": 1725276192082384000
        },
        {
          "type": "ai_text",
          "entity_type": "stream:chat:v1:message",
          "entity_id": "d10120bc-f3c6-4c47-a981-3b4a7b1cc748",
          "moderation_payload": {
            "texts": [
              "This is my phone number. +91 9958592028. Contact me for more information on whatsapp :)"
            ],
            "images": [
              "https://github.com/user-attachments/assets/06d6c129-6b6e-4be2-94fd-c0f1cdd77278"
            ],
            "custom": {
              "original_message_type": "regular"
            }
          },
          "moderation_payload_hash": "17739892842151117680",
          "labels": ["SCAM", "PII"],
          "result": [
            {
              "text": "This is my phone number. +91 9958592028. Contact me for more information on whatsapp :)",
              "action": "remove",
              "labels": ["SCAM", "PII"],
              "provider_name": "ai_text"
            }
          ],
          "review_queue_item_id": "947712cf-fd61-4fd9-bdf3-f2c9064977a9",
          "created_at": 1725276192082387000,
          "updated_at": 1725276192082387000
        },
        {
          "type": "ai_image",
          "entity_type": "stream:chat:v1:message",
          "entity_id": "d10120bc-f3c6-4c47-a981-3b4a7b1cc748",
          "moderation_payload": {
            "texts": [
              "This is my phone number. +91 9958592028. Contact me for more information on whatsapp :)"
            ],
            "images": [
              "https://github.com/user-attachments/assets/06d6c129-6b6e-4be2-94fd-c0f1cdd77278"
            ],
            "custom": {
              "original_message_type": "regular"
            }
          },
          "moderation_payload_hash": "17739892842151117680",
          "labels": [
            "Weapons",
            "Violence",
            "Weapon Violence",
            "Graphic Violence"
          ],
          "result": [
            {
              "labels": [
                "Weapons",
                "Violence",
                "Weapon Violence",
                "Graphic Violence"
              ],
              "provider_name": "ai_image",
              "image": "https://github.com/user-attachments/assets/06d6c129-6b6e-4be2-94fd-c0f1cdd77278",
              "action": "flag"
            }
          ],
          "review_queue_item_id": "947712cf-fd61-4fd9-bdf3-f2c9064977a9",
          "created_at": 1725276222331622000,
          "updated_at": 1725276222331622000
        }
      ],
      "actions": [
        {
          "id": "8bfb5b9c-2570-43f7-b9e3-ab5298170f71",
          "created_at": 1725276222349316000,
          "type": "flag",
          "user": null,
          "reason": "",
          "custom": {
            "flag_type": "ai_image"
          },
          "target_user_id": "sophia2-76",
          "review_queue_item_id": "947712cf-fd61-4fd9-bdf3-f2c9064977a9"
        },
        {
          "id": "ffcfd5cf-0446-418d-852f-c4ec94f9994a",
          "created_at": 1725276192092815000,
          "type": "flag",
          "user": null,
          "reason": "",
          "custom": {
            "flag_type": "automod_platform_circumvention"
          },
          "target_user_id": "sophia2-76",
          "review_queue_item_id": "947712cf-fd61-4fd9-bdf3-f2c9064977a9"
        },
        {
          "id": "bff0c43e-0d36-421b-9012-91e681c175bc",
          "created_at": 1725276192092815000,
          "type": "flag",
          "user": null,
          "reason": "",
          "custom": {
            "flag_type": "ai_text"
          },
          "target_user_id": "sophia2-76",
          "review_queue_item_id": "947712cf-fd61-4fd9-bdf3-f2c9064977a9"
        }
      ],
      "content_changed": false
    }
  ],
  "stats": {
    "media": 90,
    "users": 51,
    "texts": 181
  },
  "next": "W3sibmFtZSI6ImNyZWF0ZWRfYXQiLCJ2YWx1ZSI6IjIwMjQtMDktMDJUMTE6MjM6MTEuOTczNzI1WiIsImRpcmVjdGlvbiI6LTF9XQ==",
  "duration": "141.27ms"
}

Submit Action

This endpoint allows you to perform specific actions on review queue items or flagged content. It’s the underlying API used on Stream’s moderation dashboard when a moderator deletes a flagged message or unblocks a blocked message.

This endpoint serves multiple purposes:

For Stream’s chat or activity feeds products: It enables you to delete messages or ban users from your custom moderation dashboard.
For in-house products: You can inform Stream about actions taken on review queue items, ensuring these actions are reflected in Stream’s moderation dashboard.

await client.moderation.submitAction("action_type", "item_id");

Request Params

key	required	type
action_type	true	string
item_id	true	string
options	false	object

If you are using Stream’s chat or feeds product, we have some set of predefined actions which you can use. These actions are used on Stream dashboard when you press one of the buttons highlighted in following screenshot.

Mark Item As Reviewed

Marks the review queue item as reviewed. Basically on dashboard this will simply move the item from Inbox tab to Reviewed tab. No specific action is taken on the item itself.

await client.moderation.submitAction("mark_reviewed", "item_id");

Ban a user

Bans the user entity or creator of flagged entity. You can globally ban the user or ban only for chat channel.

await client.moderation.submitAction("ban", "item_id", {
  ban: {
    reason: "spam",
    channel_ban_only: false, // default
  },
});

Delete Chat Message

await client.moderation.submitAction("delete_message", "item_id", {
  delete_message: { hard_delete: false },
});

Delete Feeds Activity

await client.moderation.submitAction("delete_activity", "item_id");

Delete User

await client.moderation.submitAction("delete_user", "item_id", {
  delete_user: {
    hard_delete: false,
    mark_messages_deleted: false,
    delete_conversation_channels: false,
  },
});

Delete Feeds Reaction

await client.moderation.submitAction("delete_reaction", "item_id", {
  delete_reaction: {
    hard_delete: false, // default
  },
});

Unban User

await client.moderation.submitAction("unban", "item_id", {
  unban: {
    channel_cid: "channel_cid", // to unban channel banned user
  },
});

Restore Chat Message or Activity or Reaction

await client.moderation.submitAction("restore", "item_id");

Unblock Message

await client.moderation.submitAction("unblock", "item_id");

Custom Check

Custom Check enables you to submit your own AI moderation results of a chat message/custom uploaded content for review by moderators on the Dashboard.

Add Custom Flags to a Chat Message

Custom Flags can be added to a Chat message. This enables you to submit your own AI moderation results of a chat message for review by moderators on the Dashboard.

await client.moderation.addCustomMessageFlags(messageId, [
  {
    type: "custom_check_image",
    reason: "Image was NSFW",
    labels: ["NSFW"],
  },
  {
    type: "custom_check_text",
    reason: "Text was harmful",
    labels: ["harmful"],
  },
  {
    type: "custom_check_video",
    reason: "Video contains copyright material",
    labels: ["copyright-violation"],
  },
]);

Add Custom Check Flags to the Custom Content

The Custom Check endpoint enables you to submit your own AI moderation results for the custom content for review by moderators on the Dashboard.

await client.moderation.addCustomFlags(
  "entity_type",
  "entity_id",
  "entity_creator_id",
  {
    texts: ["offensive"],
  },
  [
    {
      type: "custom_check_text",
      reason: "Text was offensive",
      custom: {
        explaination:
          "custom is a nullable field allowing you to attach a custom object to the flag for your specific use cases or requirements",
      },
    },
  ],
);

Request Params

key	required	type	description
entity_type	true	string	This is identifier for the type of the entity. This helps with categorizing Content.
entity_id	true	string	Unique identifier for entity.
entity_creator_id	true	string	Unique identifier for user who created this entity. Generally this is the user id of the app user.
moderation_payload	true	object	Entity or content to be moderated
flags	true	array	List of custom flags
flag.type	true	”custom_check_text”, “custom_check_image”, “custom_check_video”	Type of a custom flag

Response

key	type	description
status	“complete”, “partial”	Status of moderation. In case you have configured both synchronous and asynchronous (image moderation) moderations, then status will be “partial”. This is because async moderations are still running in background task.
id	string	id of the review queue item.
item	object	This is basically a json representation of the review queue item accessible on Stream dashboard. You can use this to compose your own dashboard UI.

Moderator Roles and Permissions

Webhooks