Closed Captions

The Stream API supports adding real-time closed captioning (subtitles for participants) to your calls. This guide shows you how to build a closed captioning UI using our React SDK.

Prerequisites

Make sure that the closed captioning feature is enabled in your app’s Stream Dashboard. To enable this feature for a specific call type, go to the call type settings and set the Closed Captions Mode to one of the following:

Available: the feature is available for your call and can be turned on.
Auto-on: the feature is available and will be automatically turned on when the call starts.

Starting and stopping closed captions

If the Closed Caption Mode is set to Auto-on, closed caption will start as soon as the call starts. If it’s set to Available, you can start it manually:

await call.startClosedCaptions(); // start closed captions

In both cases, you can stop closed captions by calling:

await call.stopClosedCaptions(); // stop closed captions

The user must have permission to start or stop closed captioning.

Note that starting and stopping closed captioning affects all call participants. When closed captions are running, all call participants receive them.

If you want to enable or disable closed captioning for each participant individually, consider making it Auto-on for the call, but rendering captions conditionally based on each participant’s client-side preference.

Rendering closed captions

The next two sections show you how to render closed captions in your call UI and how to start and stop them.

Closed Captions UI

When running, closed captions are available to your application via a special useCallClosedCaptions hook. Here is how to use it:

import { StyleSheet, Text, View } from "react-native";
import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

export const ClosedCaptions = () => {
  const { useCallClosedCaptions } = useCallStateHooks();
  const closedCaptions = useCallClosedCaptions();
  return (
    <View style={styles.rootContainer}>
      {closedCaptions.map(({ user, start_time, text }) => (
        <View style={styles.closedCaptionItem} key={`${user.id}/${start_time}`}>
          <Text style={styles.speakerName}>{user.name}:</Text>
          <Text style={styles.closedCaption}>{text}</Text>
        </View>
      ))}
    </View>
  );
};

const styles = StyleSheet.create({}); // omitted for brevity

By default, this hook exposes two most recent captions. This behavior can be further tweaked.

Toggling closed captions

Only users who have permission to start or stop closed captions can toggle closed captions on and off. This can be set up in the Stream Dashboard in the Permissions section.

Here is how you can create a button that toggles closed captions depending on user permissions:

import {
  useCall,
  useCallStateHooks,
  OwnCapability,
} from "@stream-io/video-react-native-sdk";
import { Pressable, Text } from "react-native";

export const ToggleClosedCaptionsButton = () => {
  const call = useCall();
  const { useIsCallCaptioningInProgress, useHasPermissions } =
    useCallStateHooks();

  const isCaptioningInProgress = useIsCallCaptioningInProgress();
  const canToggle = useHasPermissions(
    OwnCapability.START_CLOSED_CAPTIONS_CALL,
    OwnCapability.STOP_CLOSED_CAPTIONS_CALL,
  );

  return (
    <Pressable
      disabled={!canToggle}
      onPress={() => {
        if (isCaptioningInProgress) {
          call.stopClosedCaptions();
        } else {
          call.startClosedCaptions();
        }
      }}
    >
      <Text>
        {isCaptioningInProgress ? "Disable" : "Enable"} closed captions
      </Text>
    </Pressable>
  );
};

Note that starting and stopping closed captioning affects all call participants. When closed captions are running, all call participants receive them.

And now we can add these newly created components anywhere in our call UI.

Advanced usage

If the default behavior is not enough, there are several ways to tweak it.

Override the default Close Caption Mode

It’s possible to override the default Close Caption Mode of the call type when creating a call:

await call.getOrCreate({
  data: {
    settings_override: {
      transcription: {
        mode: "available",
        closed_caption_mode: "available",
      },
    },
  },
});

Tweak visibility settings

By default, we keep a maximum of two captions visible, and each caption is visible for a maximum duration of 2.7 seconds (unless it is pushed out earlier by newer captions). This allows for a nice real-time experience while making sure there’s enough time to read each caption.

The settings can be tweaked further:

call.updateClosedCaptionSettings({
  visibilityDurationMs: 2700, // maximum duration a caption can stay visible
  maxVisibleCaptions: 2, // maximum number of captions visible at one time
});

Be careful when setting both visibilityDurationMs and maxVisibleCaptions to zero: this means that captions will be kept in call state indefinetely.

Build your own logic

Finally, if the behavior provided by the SDK isn’t enough, you can always subscribe to the caption events and build your own logic. Here is how to do it:

import { type CallClosedCaption } from "@stream-io/video-react-native-sdk";

const call = client.call(type, id);
const unsubscribe = call.on("call.closed_caption", (e: CallClosedCaption) => {
  console.log("Closed caption event:", e);
});

unsubscribe(); // remember to unsubscribe

Call Duration

Safe Area Insets