{
"review_queue_item": {
--> See ReviewQueueItem shape in the table below
},
"flags": [
--> See ModerationEventFlag shape in the table below
],
"action": {
--> See ModerationEventActionLog shape in the table below
},
"type": "review_queue_item.new | review_queue_item.updated",
"created_at": "timestamp",
"received_at": "timestamp"
}
Webhooks
Webhooks provide a powerful way to implement custom logic for moderation.
Configure webhook URL
You can configure the webhook URL from the Stream dashboard.
- Go to the dashboard
- Select the app for which you want to receive webhook events
- Click on “Preferences” under the “Moderation” section in the left navigation
- Set the URL as shown in the following screenshot
Webhook events
The webhook url will receive following events
review_queue_item.new
- This event notifies you of new content available for review. The content could be a message, activity, reaction, or even a user profile.
- This event is triggered when content is flagged for the first time. Consequently, you’ll also receive the associated flags as part of the payload.
review_queue_item.updated
- This event notifies you when existing flagged content receives additional flags or when a moderator performs an action on flagged content.
- The payload includes the new flags or details of the action performed.
Webhook Events Payload Structure
The webhook event payload is structured as follows:
ReviewQueueItem Shape
This shape contains the following fields:
Key | Type | Description | Possible Values |
---|---|---|---|
id | string | Unique identifier of the review queue item | |
created_at | datetime | Timestamp when the review queue item was created | |
updated_at | datetime | Timestamp when the review queue item was last updated | |
entity_type | string | Describes the type of entity under review | Default types include:
entity_type can be any unique string. |
entity_id | string | The id of the message, activity or reaction | |
moderation_payload | object | exact content which was sent for auto moderation E.g., { texts: ["fuck you"], images: ["https://sampleimage.com/test.jpg"], videos: [] } | |
status | string | Possible values are as following:
| |
recommended_action | Action recommended by stream moderation engines for the entity/content. | Possible values are as following:
| |
completed_at | datetime | Timestamp when all the moderation engines finished assessing the content | |
languages | array | ||
severity | int | ||
ai_text_severity | string | Possible values are as following:
| |
reviewed_at | datetime | Time at which the entity was reviewed | |
reviewed_by | string | Id of the moderator who reviewed the item. This value is set when moderator takes an action on review queue item from dashboard | |
message (only applicable to Chat product) | object | Chat message object under review | |
call (only applicable to Video call moderation) | object | Call object from which moderated frames are taken | |
entity_creator | object | The user who created the entity under review. In case of chat, its the user who sent the message under review. In case of activity feeds, its the actor of the activity/reaction | |
entity_creator_id | string | The id of the user who created the entity under review. In case of chat, its the user who sent the message under review. In case of activity feeds, its the actor of the activity/reaction | |
flags | array | List of reasons the item was flagged for moderation. | |
flags_count | int | Count of flags returned | |
bans | array | List of bans administered to the user | |
config_key | string | Policy key for the moderation policy |
ModerationEventFlag Shape
This shape contains an array of flags associated with the moderation event. Each flag is represented as follows:
| Key | Type | Description | Possible Values |
| -------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --- |
| type
| string | Type of the flag. This represents the name of moderation provider which created the flag. |
ai_image
user_report
When user flags another userautomod_semantic_filters
automod_platform_circumvention
block_list
ai_text
automod
When user is flagged from Stream’s internal rules. Currently we have 1 rule setup: if more than 3 content from same user is flagged by moderation engines, then automatically flag the user as well. So user ends up in users list page on dashboardautomod_toxicity
reason
| string | Description of the reason for the flag | |
| created_at
| string | Timestamp when the flag was created | |
| updated_at
| string | Timestamp when the flag was updated | |
| labels
| array | Classification labels for the content under review from moderation engine (e.g., ai_text, ai_image etc) which created this flag | |
| result
| object | Complete result object from moderation engine. | |
| entity_type
| string | Describes the type of entity under review | Default types include: - chat message:
stream:chat:v1:message
- feeds activity:
stream:feeds:v2:activity
- feeds reaction:
stream:feeds:v2:reaction
- user:
stream:user
entity_type
can be any unique string. | |
| entity_id
| string | The id of the message, activity or reaction | | |
| moderation_payload
| object | exact content which was sent for auto moderation E.g., { texts: ["fuck you"], images: ["https://sampleimage.com/test.jpg"], videos: [] }
| |
| review_queue_id
| string | Unique identifier of the review queue item | |
| user_id
| string | Id of the user that reported the content | |ModerationEventActionLog Shape
This shape logs actions taken during the moderation event:
Key | Type | Description | Possible Values |
---|---|---|---|
id | string | uuid for action log | Unique identifier for the action log |
created_at | datetime | Timestamp when this action was performed | |
type | string | Type of action performed on the review queue item. |
|
user_id | string | Id of the user (or moderator) who performed the action | |
reason | string | Reason attached by moderator for the action. This can be any string value | |
custom | object | Additional data regarding the action. | In case of ban type action, custom object will contain following properties:
delete_user action, custom object will contain following properties:
delete_reaction or delete_message action, custom object will contain following properties:
|
target_user_id | string | This is same as entity creator id of the review queue item. |