Moderation Tools

LAST EDIT Sep 27 2021

Flag

Copied!

Any user is allowed to flag a message. This triggers the message.flagged webhook event and adds the message to the inbox of your Stream Dashboard Chat Moderation view.

Query Message Flags

Copied!

If you prefer to build your own in app moderation dashboard, rather than use the Stream dashboard, then the query message flags endpoint lets you get flagged messages. Similar to other queries in Stream Chat, you can filter the flags using query operators.

Both server-authenticated and user-authenticated clients can use this method. For client-side requests, the user needs moderator or an admin permissions. This default behavior is controlled by the ReadMessageFlags permission and can be customized if needed. Read our permissions documentation to learn more.

name

description

example

operators

channel_cid

The channel CID of the flagged message. This parameter can only be omitted serverside.

{ channel_cid :{$in:["livestream:1","livestream:2"]}}

$eq, $in

user_id

The ID of the user that sent the message.

{"user_id": "12"}

$eq, $in

is_reviewed

If the message flag has been reviewed in the dashboard.

true

$eq

team

The team of the realted channel

{"team": "red"}

$eq, $in

Pagination for message flags can be done using the offset/limit options. Flags are returned in ascending order of creation, starting from the most recent flag.

Mutes

Copied!

Any user is allowed to mute another user. Mutes are stored at user level and returned with the rest of the user information when connectUser is called. A user will be be muted until the user is unmuted or the mute is expired.

nametypedescriptiondefaultoptional
timeoutnumberThe timeout in minutes until the mute is expired.no limit

After muting a user messages will still be delivered via web-socket. Implementing business logic such as hiding messages from muted users or display them differently is left to the developer to implement.

Messages from muted users are not delivered via push (APN/Firebase)

Ban

Copied!

Users can be banned from an app entirely or from a channel. When a user is banned, they will not be allowed to post messages until the ban is removed or expired but will be able to connect to Chat and to channels as before.

Users must be a member of a channel to be banned from that channel. Channel watchers cannot be banned.

It is also possible to ban the user's last known IP address to prevent creation of new "throw-away" accounts. This type of ban is only applicable on the app level. We do not recommend applying IP ban without reasonable timeout, however this is not restricted. The IP address will be unbanned either after reaching a timeout or with explicit user unban.

In most cases only admins or moderators are allowed to ban other users from a channel.

nametypedescriptiondefaultoptional
timeoutnumberThe timeout in minutes until the ban is automatically expired.no limit
reasonstringThe reason that the ban was created.
ip_banbooleanWhether or not to apply IP address banfalse
banned_by_idstringThe ID of the user who is performing the ban. This is required only when using API from the server-side
Banning a user from all channels can only be done using server-side auth.

Query Banned Users

Copied!

Banned users can be retrieved in different ways:

  1. Using the dedicated query bans endpoint

  2. User Search: you can add the banned:true condition to your search. Please note that this will only return users that were banned at the app-level and not the ones that were banned only on channels.

Query Bans Endpoint

Copied!

The query bans endpoint allows you to list bans for your application. Similar to other query endpoints, you can filter bans by different fields and control the ordering of results.

Globally banned users can only be retrieved using server-side auth

Available fields

Copied!

Name

Description

Example

Operators

channel_cid

The channel CID for the ban. When this parameter is not provided, both global and channel bans will be returned.

{ channel_cid :{$in:["livestream:1","livestream:2"]}}

$eq, $in

user_id

The ID of the banned user

{ user_id: "evil_user" }

$eq, $in, $neq, $nin

expired

The date (RFC339) of the ban expiration

{ expired: {$gt: "2020-10-02T15:00:00Z"} }

$eq, $gt, $gte, $lt, $lte

created_at

The date (RFC339) of the ban creation

{ created_at: {$gt: "2020-10-02T15:00:00Z"} }

$eq, $gt, $gte, $lt, $lte

banned_by_id

The ID of the user that created the ban

{ banned_by_id: "42"}

$eq, $in, $neq, $nin

Pagination for bans can be done in two ways: using offset/limit or using the "created_at" field. Bans are returned in ascending order by default so to get the second page you need to request bans with "created_at" less than the created_at of the last ban in the first page. Ordering can be reversed using the sort option.

Shadow Ban

Copied!

Instead of a default ban, you can shadow ban users from a channel, set of channels, or an entire App. When a user is shadow banned, they will still be allowed to post messages, but any message sent during the will only be visible to the messages author and invisible to other users of the App.

Shadow banning can delay a persistent bad actor from attempting ban evasion techniques by obfuscating the fact that a ban has occurred for them. Since a shadow ban's effectiveness relies on it not being discovered, shadow bans work best in chats with a high volume of messages and fast velocity where the offending user's messages not receiving engagement appears plausible; think livestreams.

Messages from a shadow banned user will include the shadowed: true flag on the message object. This flag is only visible to users other than the shadow banned user, so the offender will not be able to recognize that they are shadow banned even if they use developer tools to inspect responses from the API. You will need to implement UI logic for how your application will handle shadowed messages. Having the client hide these messages for everybody other than the user sending them is a common approach.

nametypedescriptiondefaultoptional
timeoutnumberThe timeout in minutes until the ban is automatically expired.no-limit
reasonstringThe reason that the ban was created.-
ip_banbooleanWhether or not to apply IP address banfalse
banned_by_idstringThe ID of the user who is performing the ban. This is required only when using API from the server-side-

Administrators can view shadow banned user status in queryChannels(), queryMembers() and queryUsers().

Block Lists

Copied!

A list of words you can define to moderate chat messages. A block list can be assigned to each channel type to either block or flag messages that contain these words. More information can be found here.

Advanced Chat Moderation

Copied!
The Advanced Chat Moderation beta is ending on October 4th. If you are currently participating in the beta and wish to continue using the paid feature, no action is required! If you are in the beta and wish to stop using the feature, visit your Dashboard to disable the beta access before the beta ends.

Advanced Chat Moderation uses an AI-based classification system to detect various types of bad content. The tool is powered by a machine learning model that provides a confidence interval (0-1) for a message, in each of three categories: Spam, Explicit and Toxic. The model is highly configurable for each channel type and removes the manual work of a human moderator. You can learn more here.

Advanced Chat Moderation is a paid addon for Stream chat and is billed at $650 (US) for the first million messages analyzed and $0.015/1000 messages after that.