Moderation

This cookbook shows how to handle moderation events in your video call application. These events are automatically emitted by Stream’s moderation system when content violates your configured moderation policies. You can configure moderation policies through the Stream Dashboard or via the moderation API.

The Stream API emits the following moderation events:

• call.moderation_warning - Emitted when a moderation warning is issued to the user
• call.moderation_blur - Emitted when a blur effect should be applied to the user’s camera
• call.ended - Emitted when a call ends, with reason: PolicyViolationModeration when the call is terminated due to a moderation policy violation

Moderation Notification

The following is an example of a custom ModerationNotification component that listens for moderation warning events and displays a notification to the user. It also handles cases where a call is ended due to policy violations.

import { useCall } from "@stream-io/video-react-native-sdk";
import { Alert } from "react-native";

export const ModerationNotification = () => {
  const call = useCall();
  useEffect(() => {
    if (!call) return;
    return call.on("call.moderation_warning", (e) => {
      Alert.alert("Call Moderation Warning", e.message);
    });
  }, [call]);
};

Moderation Blur

The useModeration hook is provided by @stream-io/video-react-native-sdk and handles moderation blur events automatically. When a call.moderation_blur event is emitted, the hook applies a blur effect to the user’s camera.

The SDK provides built-in video filters to support blurring of the video in a performant manner. Head over to this page for instructions to install and setup the @stream-io/video-filters-react-native library. Once you have wrapped your video components under the <BackgroundFiltersProvider /> context provider, the video filter to blur the video can be applied to your liking.

Behavior

The hook behaves differently depending on whether the @stream-io/video-filters-react-native package is available:

• If @stream-io/video-filters-react-native is available: A full-screen blur effect is applied to the user’s camera on the outgoing video track. The blur is automatically removed after the configured timeout.
• If @stream-io/video-filters-react-native is not available or an error occurs: The camera will be disabled as a fallback measure to ensure content moderation.

Usage

To enable automatic blurring moderation in your application, import the useModeration hook from @stream-io/video-react-native-sdk and call it in your custom call UI component.

import {
  StreamCall,
  StreamVideo,
  useModeration,
} from "@stream-io/video-react-native-sdk";

export const App = () => {
  return (
    <StreamVideo client={client}>
      <StreamCall call={call}>
        <CustomCallUI />
      </StreamCall>
    </StreamVideo>
  );
};

export const CustomCallUI = () => {
  // Enable moderation by plugging in the useModeration hook
  useModeration({ duration: 2000 }); // leave empty for default 5000ms

  // Render the actual component tree
};

Handling Moderation Policy Violation

After the applied moderation policies are violated multiple times, the call may be terminated by the moderation system. This is signaled through a call.ended event carrying PolicyViolationModeration as the reason.

Here is a minimalistic example:

import { useCall } from "@stream-io/video-react-native-sdk";
import { Alert } from "react-native";

export const MyComponent = () => {
  const call = useCall();
  useEffect(() => {
    if (!call) return;
    return call.on("call.ended", (event) => {
      if (event.reason === "PolicyViolationModeration") {
        Alert.alert(
          "Call Terminated",
          "The video call was terminated due to multiple policy violations.",
        );
      }
    });
  }, [call]);
};