Closed Captions

The Stream API supports adding real-time closed captions (subtitles for participants) to your calls. This guide shows you how to implement this feature on the client side.

Prerequisites

Make sure that the closed caption feature is enabled in your app’s dashboard. The closed caption feature can be set on the call type level, and the available options are:

• available: the feature is available for your call and can be enabled.
• disabled: the feature is not available for your call. In this case, it’s a good idea to “hide” any UI element you have related to closed captions.
• auto-on: the feature is available and will be enabled automatically once the user is connected to the call.

It’s also possible to override the call type’s default when creating a call:

await call.getOrCreate(
  transcription: const StreamTranscriptionSettings(
    transcriptionMode: TranscriptionSettingsMode.available,
    closedCaptionMode: ClosedCaptionSettingsMode.available,
  ),
);

You can check the current value like this:

print(call.state.value.settings.transcription.closedCaptionMode);

Enabling, disabling and tweaking closed captions

If you set closedCaptionMode to available you need to enable closed caption events when you want to see captions:

await call.startClosedCaptions(); // enable closed captions
await call.stopClosedCaptions(); // disable closed captions

You can adjust the settings that control when closed captions appear by changing the Call Preferences.

streamVideo.makeCall(
  callType: StreamCallType.defaultType(),
  id: 'my-call-id',
  preferences: DefaultCallPreferences(
    closedCaptionsVisibleCaptions: 2, // number of captions that can be stored in the queue
    closedCaptionsVisibilityDurationMs: 2700, // the duration a caption can stay in the queue
  ),
);

Check if closed captions are enabled

final isCaptioningInProgress = call.state.value.isCaptioning;

Displaying the captions

You can access the most recent captions by accessing the closed captions stream in Call:

final subscription = call.closedCaptions.listen((captions) {
  updateDisplayedCaptions(captions);
});

void updateDisplayedCaptions(List<StreamClosedCaption> captions) {
  final captionsText = captions
      .map((caption) => '${caption.user.name}: ${caption.text}')
      .join('\n');
}

subscription.cancel();

This is how an example closed caption looks like:

{
  "text": "Thank you, guys, for listening.",
  "// When did the speaker start speaking": "",
  "start_time": "2024-09-25T12:22:21.310735726Z",
  "// When did the speaker finish saying the caption": "",
  "end_time": "2024-09-25T12:22:24.310735726Z",
  "speaker_id": "zitaszuperagetstreamio",
  "user": {
    "id": "zitaszuperagetstreamio",
    "name": "Zita",
    "role": "host",
    "// other user properties": ""
  }
}

See it in action

To see closed captions in action, check out our Flutter sample application on GitHub.