Camera & Microphone

Handling audio and video devices in a your application means working with MediaStream, MediaDeviceInfo and other WebRTC API objects. To make this simpler, we hide all the complexity inside the SDK and export utility functions and states. In this guide we shall go over their usage.

Camera management

The SDK does its best to make working with the camera easy. We expose the following objects on the call:

Call settings

The default state of the camera is determined by the settings in the call object.

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useCallSettings } = useCallStateHooks();
const settings = useCallSettings();

console.log(settings?.video.camera_default_on);

note

Make sure, call.get() is called at least once in the application, after the call is created.

Start-Stop Camera

We can use the functions camera.enable() and camera.disable() to control the publishing and un-publishing our video stream.

Alternatively, you can use camera.toggle().

import { useCall, useCallStateHooks } from '@stream-io/video-react-native-sdk';

const call = useCall();

const { useCameraState } = useCallStateHooks();
const { camera, isMute } = useCameraState();

console.log(`Camera is ${isMute ? 'off' : 'on'}`);
await camera.toggle();

// or, alternatively
await camera.enable();
await camera.disable();

It's always best to await calls to enable(), disable(), and toggle(), however the SDK does its best to resolve potential race conditions: the last call always wins, so it's safe to make these calls in an event handler.

Status is updated once the camera is actually enabled or disabled. Use isOptimisticMute for the "optimistic" status that is updated immediately after toggling the camera.

Manage Camera Facing Mode

We can toggle the camera face from front to back and vice-versa using camera.flip().

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useCameraState } = useCallStateHooks();
const { camera } = useCameraState();

console.log(direction); // direction returns 'front' or 'back'.
camera.flip();

We can get the facing mode state of the camera by:

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useCameraState } = useCallStateHooks();
const { direction } = useCameraState(); // direction returns 'front' or 'back'.

Video mute status

We can get the mute state of our video stream by checking the status value returned from the useCameraState hook:

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useCameraState } = useCallStateHooks();
const { status } = useCameraState(); // status returns enabled, disabled or undefined

Show Video Preview

We can get the video stream from the camera using the media stream from the call.camera object and show it using the RTCView component from @stream-io/react-native-webrtc library:

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';
import { RTCView } from '@stream-io/react-native-webrtc';
const { useCameraState } = useCallStateHooks();
const { camera } = useCameraState();

const localVideoStream = camera.state.mediaStream;

return <RTCView streamURL={localVideoStream?.toURL()} />;

Access to the Camera's MediaStream

Our SDK exposes the current mediaStream instance that you can use for your needs (for example, local recording, etc...):

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useCameraState } = useCallStateHooks();
const { mediaStream } = useCameraState();

const [videoTrack] = mediaStream.getVideoTracks();
console.log('Video track', videoTrack);

Microphone management

The SDK does its best to make working with the microphone easy. We expose the following objects on the call:

Call settings

The default state of the microphone is determined by the settings in the call object.

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useCallSettings } = useCallStateHooks();
const settings = useCallSettings();

console.log(settings?.audio.mic_default_on);

note

Make sure, call.get() is called at least once in the application, after the call is created.

Start-Stop Microphone

We can use the functions microphone.enable() and microphone.disable() to control the publishing and un-publishing our audio stream:

Alternatively, you can use microphone.toggle().

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useMicrophoneState } = useCallStateHooks();
const { microphone, isMute } = useMicrophoneState();

console.log(`Microphone is ${isMute ? 'off' : 'on'}`);
await microphone.toggle();

// or, alternatively
await microphone.enable();
await microphone.disable();

It's always best to await calls to enable(), disable(), and toggle(), however the SDK does its best to resolve potential race conditions: the last call always wins, so it's safe to make these calls in an event handler.

Status is updated once the microphone is actually enabled or disabled. Use isOptimisticMute for the "optimistic" status that is updated immediately after toggling the microphone.

Audio mute status

We can get the mute state of our audio stream by checking the status value returned from the useMicrophoneState hook:

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useMicrophoneState } = useCallStateHooks();
const { status } = useMicrophoneState(); // status returns enabled, disabled or undefined

Speaking while muted detection

Our SDK provides a mechanism that can detect whether the user started to speak while being muted. Through this mechanism, you can display a notification to the user, or apply any custom logic.

This feature is enabled by default unless the user doesn't have the permission to send audio or explicitly disabled.

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useMicrophoneState } = useCallStateHooks();
const { isSpeakingWhileMuted, microphone } = useMicrophoneState();

if (isSpeakingWhileMuted) {
  // your custom logic comes here
  console.log('You are speaking while muted!');
}

// to disable this feature completely:
await microphone.disableSpeakingWhileMutedNotification();

// to enable it back:
await microphone.enableSpeakingWhileMutedNotification();

Access to the Microphone's MediaStream

Our SDK exposes the current mediaStream instance that you can use for your needs (for example, local recording, etc...):

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useMicrophoneState } = useCallStateHooks();
const { mediaStream } = useMicrophoneState();

const [audioTrack] = mediaStream.getAudioTracks();
console.log('Audio track', audioTrack);

Speaker management

danger

We do not support using hooks to change audio output source for React Native SDK as React Native WebRTC doesn't support device switching in RN. This means the following would not work with React Native SDK.

import { useCallStateHooks } from '@stream-io/video-react-native-sdk';

const { useMicrophoneState, useSpeakerState } = useCallStateHooks();

// This will give error on React Native.
const { selectedDevice } = useMicrophoneState();
// Also, This will give error on React Native.
const { speaker, selectedDevice, devices } = useSpeakerState();

We primarily utilize react-native-incall-manager for managing speaker audio output in our SDK.

This functionality is seamlessly integrated into our video calling and livestream components.

In our SDK, we set the media type to video for video calls and livestream use-case, and audio for audio-only calls, in the start method of the InCallManager.

Passing video to media, routes the audio to device speaker by default unless an external device is connected.

Passing audio to media, enables the proximity sensor, routes the audio through earpiece by default unless an external device is connected.

When an external device is connected in between or before the call, the audio is always routed through it.

For Audio Rooms, you have to control the behaviour of InCallManager. It allows you to start/stop the audio output on the device speaker through:

import InCallManager from 'react-native-incall-manager';

// Called on call join
InCallManager.start({ media: 'video' }); // `media` values - audio/video, default: audio

// Called when call is left
InCallManager.stop();

To force route the audio through speaker, use the following method:

import InCallManager from 'react-native-incall-manager';

InCallManager.setForceSpeakerphoneOn(true); // Pass true for speaker on, and false for off. Once off audio is always routed through earpiece.

For more information and customization please visit the official docs of InCallManager.

Client-side settings

Before joining a call, user may need to preview their streams and decide their mute status.

You can set the same on the dashboard for your app. This can be done under the Call Types and then selecting your call type.

.

We use the useApplyDefaultMediaStreamSettings hook to apply the backend settings. You can use the same, if needed.

note

Generally, the backend settings are applied by default when you join the call(call.join()). To apply the backend settings somewhere else like custom Lobby, OutgoingCall components, etc., you can use the hook useApplyDefaultMediaStreamSettings above.

If you have use-cases, where you want to voluntarily have a different behaviour, you can design your own hook and apply your logic on top of the default one.