Camera & Microphone

Handling audio and video devices in your application means working with MediaStream, MediaDeviceInfo and other WebRTC API objects. To simplify this, we hide all the complexity inside the SDK and export utility functions and states. In this guide, we shall go over their usage.

Camera management

The SDK does its best to make working with the camera easy. We expose the following camera object on the call:

const call = useCall();
const camera = call.camera;

Call settings

The default state of the camera is determined by the settings in the call object.

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

const { useCallSettings } = useCallStateHooks();
const settings = useCallSettings();

console.log(settings?.video.camera_default_on);

Start-Stop Camera

We can use the functions camera.enable() and camera.disable() to control the publishing and unpublishing our video stream.

Alternatively, you can use camera.toggle().

import { useCall, useCallStateHooks } from "@stream-io/video-react-native-sdk";

const call = useCall();

const { useCameraState } = useCallStateHooks();
const { camera, isMute } = useCameraState();

console.log(`Camera is ${isMute ? "off" : "on"}`);
await camera.toggle();

// or, alternatively
await camera.enable();
await camera.disable();

It’s always best to await calls to enable(), disable(), and toggle(), however the SDK does its best to resolve potential race conditions: the last call always wins, so it’s safe to make these calls in an event handler.

The status is updated once the camera is actually enabled or disabled. Use optimisticIsMute for the “optimistic” status that is updated immediately after toggling the camera.

Manage Camera Facing Mode

We can toggle the camera face from front to back and vice versa using camera.flip().

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

const { useCameraState } = useCallStateHooks();
const { camera } = useCameraState();

console.log(direction); // direction returns 'front' or 'back'.
camera.flip();

We can get the facing mode state of the camera by:

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

const { useCameraState } = useCallStateHooks();
const { direction } = useCameraState(); // direction returns 'front' or 'back'.

Video mute status

We can get the mute state of our video stream by checking the status value returned from the useCameraState hook:

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

const { useCameraState } = useCallStateHooks();
const { status } = useCameraState(); // status returns enabled, disabled or undefined

Show Video Preview

We can get the video stream from the camera using the media stream from the call.camera object and show it using the RTCView component from @stream-io/react-native-webrtc library:

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";
import { RTCView } from "@stream-io/react-native-webrtc";
const { useCameraState } = useCallStateHooks();
const { camera } = useCameraState();

const localVideoStream = camera.state.mediaStream;

return <RTCView streamURL={localVideoStream?.toURL()} />;

Access to the Camera’s MediaStream

Our SDK exposes the current mediaStream instance that you can use for your needs (for example, local recording, etc…):

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

const { useCameraState } = useCallStateHooks();
const { mediaStream } = useCameraState();

const [videoTrack] = mediaStream.getVideoTracks();
console.log("Video track", videoTrack);

Microphone management

The SDK does its best to make working with the microphone easy. We expose the following microphone object on the call:

const call = useCall();
const microphone = call.microphone;

Call settings

The default state of the microphone is determined by the settings in the call object.

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

const { useCallSettings } = useCallStateHooks();
const settings = useCallSettings();

console.log(settings?.audio.mic_default_on);

Start-Stop Microphone

We can use the functions microphone.enable() and microphone.disable() to control the publishing and unpublishing our audio stream:

Alternatively, you can use microphone.toggle().

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

const { useMicrophoneState } = useCallStateHooks();
const { microphone, isMute } = useMicrophoneState();

console.log(`Microphone is ${isMute ? "off" : "on"}`);
await microphone.toggle();

// or, alternatively
await microphone.enable();
await microphone.disable();

It’s always best to await calls to enable(), disable(), and toggle(), however the SDK does its best to resolve potential race conditions: the last call always wins, so it’s safe to make these calls in an event handler.

The status is updated once the microphone is actually enabled or disabled. Use optimisticIsMute for the “optimistic” status that is updated immediately after toggling the microphone.

Audio mute status

We can get the mute state of our audio stream by checking the status value returned from the useMicrophoneState hook:

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

const { useMicrophoneState } = useCallStateHooks();
const { status } = useMicrophoneState(); // status returns enabled, disabled or undefined

Speaking while muted detection

Our SDK provides a mechanism that can detect whether the user started to speak while being muted. Through this mechanism, you can display a notification to the user, or apply any custom logic.

This feature is enabled by default unless the user doesn’t have the permission to send audio or explicitly disabled.

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

const { useMicrophoneState } = useCallStateHooks();
const { isSpeakingWhileMuted, microphone } = useMicrophoneState();

if (isSpeakingWhileMuted) {
  // your custom logic comes here
  console.log("You are speaking while muted!");
}

// to disable this feature completely:
await microphone.disableSpeakingWhileMutedNotification();

// to enable it back:
await microphone.enableSpeakingWhileMutedNotification();

Access to the Microphone’s MediaStream

Our SDK exposes the current mediaStream instance that you can use for your needs (for example, local recording, etc…):

import { useCallStateHooks } from "@stream-io/video-react-native-sdk";

const { useMicrophoneState } = useCallStateHooks();
const { mediaStream } = useMicrophoneState();

const [audioTrack] = mediaStream.getAudioTracks();
console.log("Audio track", audioTrack);

Speaker management

We can use the callManager module from the SDK for managing speaker audio output in our SDK. The start() and stop() methods are to be used for managing audio. Before joining a call or as soon as you join a call, call the start() method. Once you leave the call, use the stop() method.

import { callManager } from "@stream-io/video-react-native-sdk";

// To be called before joining a call or as soon as joining a call
callManager.start({
  audioRole: "communicator", // or "listener"
  deviceEndpointType: "speaker", // or "earpiece"
});

// To be called when a call is left
callManager.stop();

• audioRole: communicator (default) or listener. Use listener for users that won’t publish audio but only listen (typical for livestream audience). Use communicator otherwise.

• deviceEndpointType: speaker or earpiece. Available only when audioRole is set to communicator. speaker enables the loudspeaker when no bluetooth device or other wired headset is connected. earpiece routes the audio through the ear-speaker unless another external device is connected. earpiece should be passed only on a scenario of an audio call similar to a mobile cellular phone call.

The following device priority is used:

1. Bluetooth Headset or Wired Headset
2. Speakerphone or Earpiece.

Switching audio output device

The API for switching the audio output device is vastly different on iOS and Android.

iOSAndroid

In iOS, use the following method to open the popover from AVRoutePickerView that is presented by the system. The popover will show the audio devices that are available, and the user can tap on the list to choose the desired audio device.

import { callManager } from "@stream-io/video-react-native-sdk";

callManager.ios.showDeviceSelector();

Once this method is called, a popover similar to the on below will appear.

Force audio through the loudspeaker

A common use case is to allow switching the audio through the load speaker and ear speaker, e.g.: through a button toggle. To support this behavior, you can use the following method on both iOS and Android:

import { callManager } from "@stream-io/video-react-native-sdk";

// route audio through loud speaker immediately (audio outputs here until a new external device is connected)
callManager.speaker.setForceSpeakerphoneOn(true);

// revert back to default behaviour
callManager.speaker.setForceSpeakerphoneOn(false);

Audio volume control

The SDK supports both system-wide audio volume control and individual track level audio volume control.

System wide mute and unmute of audio volume

import { callManager } from "@stream-io/video-react-native-sdk";

// to mute audio
callManager.setMute(true);

// to unmute audio
callManager.setMute(false);

Participant volume control

We also support setting a participant audio volume. Here is a small snippet of how to set a participant volume to 50%:

import { type StreamVideoParticipant } from "@stream-io/video-react-native-sdk";

let participant: StreamVideoParticipant; // the intended participant
call.speaker.setParticipantVolume(participant.sessionId, 0.5);