This tutorial teaches you how to quickly build a production-ready voice AI agent with OpenAI realtime using Stream's video edge network, Kotlin, and Node.

The instructions to the agent are sent server-side (Node) so you can do function calling or RAG.
The integration uses Stream's video edge network (for low latency) and WebRTC (so it works under slow/unreliable network conditions).
You have full control over the AI setup and visualization.

The result will look something like this:

This page works for both humans and AI coding agents: every step is a file operation, shell command, or a clearly-marked manual action, and each step has a verification checkpoint.

This is a full-stack tutorial. It has a Node backend (mints tokens, connects the OpenAI agent — this is where your OpenAI key lives, never on the client) and an Android app (joins the call, visualizes audio). You need an OpenAI account and a Stream account. While this uses Node + Kotlin, you could use any backend language + Stream SDK.

Choose your path

Path A — Let your AI agent build it (recommended)

Install the Stream CLI once, then add the skills, so Claude Code, Cursor, or Codex build the Android code against current SDK APIs:

bash

1
2
3
4
5
6
7
# Install the getstream CLI
curl -fsSL https://getstream.io/cli.sh | bash

# Install the skills (router, docs, and builder) for your agent.
# --universal for Cursor, Codex, and other AGENTS-ecosystem tools; --claude for Claude Code.
getstream skills --universal
getstream skills stream-android --universal

Then ask your agent:

plaintext

1
2
3
4
/stream-android Build the Android client for a voice AI assistant: fetch credentials
from my Node /credentials endpoint, join the Stream call, POST to /connect to attach
the OpenAI agent, and visualize participant audio levels. The Node backend and OpenAI
key are provided separately.

Where you come in. Your coding agent can scaffold the Node server and the Android app, but you provide the OpenAI API key and your Stream API secret (secrets never go in client code), and run the app granting microphone access. The /stream-android skill covers the Android SDK; the Node backend uses Stream's Node SDK.

Path B — Build it manually

Follow the steps below: Part 1 builds the Node backend, Part 2 builds the Android app.

For AI assistants reading this page: append .md to any Stream docs URL for a clean Markdown version. A condensed index for the Android Video SDK is at https://getstream.io/video/docs/android/llms.txt. Prefer these over parsing HTML.

Important Building Blocks

Node backend — mints Stream user tokens (/credentials) and connects the OpenAI agent to a call (/connect) via streamClient.video.connectOpenAi(...). Your OpenAI key and Stream secret live here only.
connectOpenAi — returns OpenAI's official realtime client, so you have full control: updateSession for instructions, addTool for function calling, event listeners for transcripts.
Android StreamVideo client — fetches credentials from your backend, joins the call, and reads call.state.activeSpeakers + participant.audioLevels to drive the audio visualization.

Part 1 — The Node backend

Step 1 — Connect an AI agent to Stream from your backend

First we'll get an agent app that adds an AI agent to a call, introducing the basic concepts.

Step 1.1 — OpenAI and Stream credentials

You need an OpenAI account and API key. The OpenAI credentials are never shared client-side — they're only exchanged between your server and Stream's servers. You also need a Stream account, and the API key + secret from the Stream dashboard.

🧑 Human checkpoint: creating an OpenAI account and API key is a manual step outside any CLI. Agents: ask the human for the OpenAI API key and the Stream API key + secret before continuing. The Stream key/secret come from the dashboard (or getstream env for the key — but the secret is only shown in the dashboard).

Step 1.2 — Create the Node.js project

Use Node.js 22 or later (node -v). Create a folder openai-audio-tutorial, cd into it, and run:

bash

1
npm init -y

Step 1.3 — Install the dependencies

Replace the generated package.json with:

package.json (json)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
  "name": "@stream-io/video-ai-demo-server",
  "type": "module",
  "dependencies": {
    "@hono/node-server": "^1.13.8",
    "@stream-io/node-sdk": "^0.7.63",
    "@stream-io/openai-realtime-api": "^0.4.2",
    "dotenv": "^16.3.1",
    "hono": "^4.7.4",
    "open": "^10.1.0"
  },
  "scripts": {
    "server": "node ./server.mjs",
    "standalone-ui": "node ./standalone.mjs"
  }
}

Then install:

bash

1
npm install

Step 1.4 — Set up the credentials

Create a .env file in the project root:

.env (text)

1
2
3
4
5
6
# Stream API credentials
STREAM_API_KEY=your_stream_api_key
STREAM_API_SECRET=your_stream_api_secret

# OpenAI API key
OPENAI_API_KEY=your_openai_api_key

Fill in your actual keys from Step 1.1. You can find the Stream keys on your dashboard:

API Keys

✅ Checkpoint: .env has all three values filled in (no your_... placeholders left).

Step 1.5 — Implement the standalone-ui script

Before the Android integration, we'll build a simple server integration that connects the AI agent to a call and joins it from a webapp. Create standalone.mjs:

standalone.mjs (js)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
import { config } from "dotenv";
import { StreamClient } from "@stream-io/node-sdk";
import open from "open";
import crypto from "crypto";

// load config from dotenv
config();

async function main() {
  // Get environment variables
  const streamApiKey = process.env.STREAM_API_KEY;
  const streamApiSecret = process.env.STREAM_API_SECRET;
  const openAiApiKey = process.env.OPENAI_API_KEY;

  // Check if all required environment variables are set
  if (!streamApiKey || !streamApiSecret || !openAiApiKey) {
    console.error(
      "Error: Missing required environment variables, make sure to have a .env file in the project root, check .env.example for reference",
    );
    process.exit(1);
  }

  const streamClient = new StreamClient(streamApiKey, streamApiSecret);
  const call = streamClient.video.call("default", crypto.randomUUID());

  // realtimeClient is https://github.com/openai/openai-realtime-api-beta openai/openai-realtime-api-beta
  const realtimeClient = await streamClient.video.connectOpenAi({
    call,
    openAiApiKey,
    agentUserId: "lucy",
  });

  // Set up event handling, all events from openai realtime api are available here see: https://platform.openai.com/docs/api-reference/realtime-server-events
  realtimeClient.on("realtime.event", ({ time, source, event }) => {
    console.log(`got an event from OpenAI ${event.type}`);
    if (event.type === "response.audio_transcript.done") {
      console.log(`got a transcript from OpenAI ${event.transcript}`);
    }
  });

  realtimeClient.updateSession({
    instructions:
      "You are a helpful assistant that can answer questions and help with tasks.",
  });

  // Get token for the call
  const token = streamClient.generateUserToken({ user_id: "theodore" });

  // Construct the URL, TODO: replace this with
  const callUrl = `https://pronto.getstream.io/join/${call.id}?type=default&api_key=${streamClient.apiKey}&token=${token}`;

  // Open the browser
  console.log(`Opening browser to join the call... ${callUrl}`);
  await open(callUrl);
}

main().catch((error) => {
  console.error("Error:", error);
  process.exit(1);
});

Step 1.6 — Running the sample

Run the script:

bash

1
npm run standalone-ui

This opens your browser and connects you to a call where you can talk to the OpenAI agent. As you talk, your shell logs each event OpenAI sends.

🧑 Human checkpoint: this opens a browser and uses your microphone. Agents: run it, then ask the human to talk to the voice agent and confirm transcripts appear in the shell.

✅ Checkpoint: the browser call connects, and the shell logs got an event from OpenAI ... lines (including a transcript when you speak).

Let's review the server-side code:

We instantiate the Stream Node SDK and create a call to host the conversation:

1
2
const streamClient = new StreamClient(streamApiKey, streamApiSecret);
const call = streamClient.video.call("default", crypto.randomUUID());

connectOpenAi instantiates the Realtime API client and connects the agent to the call as user "lucy":

1
2
3
4
5
const realtimeClient = await streamClient.video.connectOpenAi({
  call,
  openAiApiKey,
  agentUserId: "lucy",
});

realtimeClient is OpenAI's official client — pass instructions and listen to events:

1
2
3
4
5
6
7
8
9
10
11
realtimeClient.on("realtime.event", ({ time, source, event }) => {
  console.log(`got an event from OpenAI ${event.type}`);
  if (event.type === "response.audio_transcript.done") {
    console.log(`got a transcript from OpenAI ${event.transcript}`);
  }
});

realtimeClient.updateSession({
  instructions:
    "You are a helpful assistant that can answer questions and help with tasks.",
});

Step 2 — Set up your server-side integration

For a real app the backend must handle client authentication and send instructions to OpenAI (RAG, function calling). Our backend will:

Generate a valid token for the Android app to join the call.
Join the same call with the AI agent and set it up with instructions.

Step 2.1 — Implement `server.mjs`

Create server.mjs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
import { serve } from "@hono/node-server";
import { StreamClient } from "@stream-io/node-sdk";
import { Hono } from "hono";
import crypto from "crypto";
import { config } from "dotenv";

// load config from dotenv
config();

// Get environment variables
const streamApiKey = process.env.STREAM_API_KEY;
const streamApiSecret = process.env.STREAM_API_SECRET;
const openAiApiKey = process.env.OPENAI_API_KEY;

// Check if all required environment variables are set
if (!streamApiKey || !streamApiSecret || !openAiApiKey) {
  console.error(
    "Error: Missing required environment variables, make sure to have a .env file in the project root, check .env.example for reference",
  );
  process.exit(1);
}

const app = new Hono();
const streamClient = new StreamClient(streamApiKey, streamApiSecret);

/**
 * Endpoint to generate credentials for a new video call.
 * Creates a unique call ID, generates a token, and returns necessary connection details.
 */
app.get("/credentials", (c) => {
  console.log("got a request for credentials");
  // Generate a shorter UUID for callId (first 12 chars)
  const callId = crypto.randomUUID().replace(/-/g, "").substring(0, 12);
  // Generate a shorter UUID for userId (first 8 chars with prefix)
  const userId = `user-${crypto.randomUUID().replace(/-/g, "").substring(0, 8)}`;
  const callType = "default";
  const token = streamClient.generateUserToken({
    user_id: userId,
  });
  return c.json({
    apiKey: streamApiKey,
    token,
    callType,
    callId,
    userId,
  });
});

/**
 * Endpoint to connect an AI agent to an existing video call.
 * Takes call type and ID parameters, connects the OpenAI agent to the call,
 * sets up the realtime client with event handlers and tools,
 * and returns a success response when complete.
 */
app.post("/:callType/:callId/connect", async (c) => {
  console.log("got a request for connect");
  const callType = c.req.param("callType");
  const callId = c.req.param("callId");

  const call = streamClient.video.call(callType, callId);
  const realtimeClient = await streamClient.video.connectOpenAi({
    call,
    openAiApiKey,
    agentUserId: "lucy",
  });
  await setupRealtimeClient(realtimeClient);
  console.log("agent is connected now");
  return c.json({ ok: true });
});

async function setupRealtimeClient(realtimeClient) {
  realtimeClient.on("error", (event) => {
    console.error("Error:", event);
  });

  realtimeClient.on("session.update", (event) => {
    console.log("Realtime session update:", event);
  });

  realtimeClient.updateSession({
    instructions:
      "You are a helpful assistant that can answer questions and help with tasks.",
    turn_detection: { type: "semantic_vad" },
    input_audio_transcription: { model: "gpt-4o-transcribe" },
    input_audio_noise_reduction: { type: "near_field" },
  });

  realtimeClient.addTool(
    {
      name: "get_weather",
      description:
        "Call this function to retrieve current weather information for a specific location. Provide the city name.",
      parameters: {
        type: "object",
        properties: {
          city: {
            type: "string",
            description: "The name of the city to get weather information for",
          },
        },
        required: ["city"],
      },
    },
    async ({ city, country, units = "metric" }) => {
      console.log("get_weather request", { city, country, units });
      try {
        // This is a placeholder for actual weather API implementation
        // In a real implementation, you would call a weather API service here
        const weatherData = {
          location: country ? `${city}, ${country}` : city,
          temperature: 22,
          units: units === "imperial" ? "°F" : "°C",
          condition: "Partly Cloudy",
          humidity: 65,
          windSpeed: 10,
        };

        return weatherData;
      } catch (error) {
        console.error("Error fetching weather data:", error);
        return { error: "Failed to retrieve weather information" };
      }
    },
  );

  return realtimeClient;
}

// Start the server
serve({
  fetch: app.fetch,
  hostname: "0.0.0.0",
  port: 3000,
});

console.log(`Server started on :3000`);

Two endpoints: /credentials generates a call ID and token; /:callType/:callId/connect connects the AI agent ("lucy") to a call. The updateSession call sets the instructions, semantic voice-activity detection, a GPT-4o transcription model, and near-field noise reduction. We also show a get_weather function-call tool.

Step 2.2 — Running the server

bash

1
npm run server

Check it with a curl:

bash

1
curl -X GET http://localhost:3000/credentials

✅ Checkpoint: the curl returns JSON with apiKey, token, callType, callId, and userId. Keep the server running — the Android app calls it. (Agents: leave the server process running in the background for the rest of the tutorial.)

Part 2 — The Android app

Step 3 — Setting up the Android project

Now the Android app, which connects to this API and visualizes the AI's audio levels.

Step 3.1 — Create the project & add the Stream Video dependency

Create a new project such as AIVideoDemo.

🧑 Human checkpoint: the project is created in Android Studio's New Project wizard. Agents: detect the project shape first (find . -maxdepth 3 -name "settings.gradle*" -o -name "build.gradle*"); if there's no Gradle project, ask the human to create it.

Add the Stream Video Android SDK by following the installation guide.

Step 3.2 — Add the other dependencies

You need microphone permission (via accompanist) and network calls (via retrofit). Copy the following libs.versions.toml and build.gradle, adjusting as needed. Replace <latest_version> for stream with the latest release:

libs.versions.toml (kotlin)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
[versions]
agp = "8.8.1"
kotlin = "2.0.0"
coreKtx = "1.15.0"
lifecycleRuntimeKtx = "2.8.7"
activityCompose = "1.10.0"
composeBom = "2024.04.01"
stream = "<latest_version>"
retrofit = "2.11.0"
accompanist = "0.33.1-alpha"
kotlinxSerializationJson="1.6.0"

[libraries]
androidx-core-ktx = { group = "androidx.core", name = "core-ktx", version.ref = "coreKtx" }
androidx-lifecycle-runtime-ktx = { group = "androidx.lifecycle", name = "lifecycle-runtime-ktx", version.ref = "lifecycleRuntimeKtx" }
androidx-lifecycle-runtime-compose = {group = "androidx.lifecycle", name = "lifecycle-runtime-compose", version.ref = "lifecycleRuntimeKtx"}
androidx-lifecycle-viewmodel-compose = {group = "androidx.lifecycle", name = "lifecycle-viewmodel-compose", version.ref = "lifecycleRuntimeKtx"}

androidx-activity-compose = { group = "androidx.activity", name = "activity-compose", version.ref = "activityCompose" }
androidx-activity-ktx = { group = "androidx.activity", name = "activity-ktx", version.ref = "activityCompose" }
androidx-compose-bom = { group = "androidx.compose", name = "compose-bom", version.ref = "composeBom" }

androidx-ui = { group = "androidx.compose.ui", name = "ui" }
androidx-ui-graphics = { group = "androidx.compose.ui", name = "ui-graphics" }
androidx-ui-tooling-preview = { group = "androidx.compose.ui", name = "ui-tooling-preview" }
androidx-material3 = { group = "androidx.compose.material3", name = "material3" }

getstream-video-android-ui-core = { group = "io.getstream", name = "stream-video-android-ui-core", version.ref = "stream"}
retrofit = {group = "com.squareup.retrofit2", name = "retrofit", version.ref = "retrofit" }
retrofit-gson = {group = "com.squareup.retrofit2", name = "converter-gson", version.ref = "retrofit" }
accompanist-permissions = {group = "com.google.accompanist", name = "accompanist-permissions", version.ref = "accompanist" }
kotlinx-serialization-json = { group = "org.jetbrains.kotlinx", name="kotlinx-serialization-json", version.ref = "kotlinxSerializationJson" }

build.gradle (gradle)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
dependencies {
	
    implementation(libs.androidx.core.ktx)
    implementation(libs.androidx.lifecycle.runtime.ktx)
    implementation(libs.androidx.lifecycle.runtime.compose)
    implementation(libs.androidx.lifecycle.viewmodel.compose)
    implementation(libs.androidx.activity.compose)
    implementation(libs.androidx.activity.ktx)
    implementation(platform(libs.androidx.compose.bom))
    implementation(libs.androidx.ui)
    implementation(libs.androidx.ui.graphics)
    implementation(libs.androidx.ui.tooling.preview)
    implementation(libs.androidx.material3)

    implementation(libs.getstream.video.android.ui.core)
    implementation(libs.retrofit)
    implementation(libs.retrofit.gson)
    implementation(libs.kotlinx.serialization.json)
    implementation(libs.accompanist.permissions)

}

✅ Checkpoint: ./gradlew :app:assembleDebug succeeds and import io.getstream.video.android.core.StreamVideoBuilder resolves.

Step 4 — Stream Video Setup

Step 4.1 — Declaring the required properties

Create ApiService.kt — the network layer:

ApiService.kt (kotlin)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
package io.getstream.ai.audiodemo

import io.getstream.ai.audiodemo.Credentials
import okhttp3.ResponseBody
import retrofit2.Retrofit
import retrofit2.converter.gson.GsonConverterFactory
import retrofit2.http.GET
import retrofit2.http.POST
import retrofit2.http.Path

interface ApiService {
    @GET("credentials")
    suspend fun getCredentials(): Credentials

    @POST("{callType}/{channelId}/connect")
    suspend fun connectAI(
        @Path("callType", encoded = true) callType: String,
        @Path("channelId", encoded = true) channelId: String): ResponseBody
}

object RetrofitInstance {
    private val BASE_URL = "http://10.0.2.2:3000/"

    val api: ApiService by lazy {
        Retrofit.Builder()
            .baseUrl(BASE_URL)
            .addConverterFactory(GsonConverterFactory.create())
            .build()
            .create(ApiService::class.java)
    }
}

Create MainViewModel.kt — makes network calls, holds UI state, and holds the StreamVideo and Call objects:

MainViewModel.kt (kotlin)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
class MainViewModel(val app: Application) : AndroidViewModel(app) {

    var callUiState = MutableStateFlow(CallUiState.IDLE)
    val credentials = MutableStateFlow<DataLayerResponse<Credentials>>(DataLayerResponse.Initial("Initial State"))
    var client: StreamVideo? = null
    var call: Call? = null

    fun initCredentials() {
        viewModelScope.launch {
            try {
                credentials.emit(DataLayerResponse.Initial("Getting Credentials"))
                val response = RetrofitInstance.api.getCredentials()
                createStreamClient(response)
                credentials.emit(DataLayerResponse.Success(response))
            } catch (ex: Exception) {
                credentials.emit(DataLayerResponse.Error(ex.message))
            }
        }
    }

    private fun createStreamClient(credentials: Credentials) {
        val userId = credentials.userId

        val user = User(
            id = userId, 
            name = "Tutorial", 
            image = "https://bit.ly/2TIt8NR",
        )

        client = StreamVideoBuilder(
            context = app.applicationContext,
            apiKey = credentials.apiKey,
            geo = GEO.GlobalEdgeNetwork,
            user = user,
            token = credentials.token,
        ).build()
    }

    fun joinCall() {

        client?.let { client->
            val credentials = (credentials.value as DataLayerResponse.Success).data
            call = client.call(credentials.callType, credentials.callId)
            connectAi(call!!, credentials.callId, credentials.callType)
        }

    }

    private fun connectAi(call: Call, channelId: String, callType: String) {
        viewModelScope.launch {
            try {
                callUiState.emit(CallUiState.JOINING)
                val encodedChannelId = URLEncoder.encode(channelId, StandardCharsets.UTF_8.toString())
                val encodedCallType = URLEncoder.encode(callType, StandardCharsets.UTF_8.toString())
                val response = RetrofitInstance.api.connectAI(encodedCallType, encodedChannelId)
                call.join(create = true)
                callUiState.emit(CallUiState.ACTIVE)
            }catch (ex: Exception) {
                ex.printStackTrace()
                callUiState.emit(CallUiState.ERROR)
            }

        }
    }

    fun disconnect(){
        viewModelScope.launch {
            call?.end()
            callUiState.emit(CallUiState.IDLE)
        }
    }
}

sealed class DataLayerResponse<T> {
    class Success<T>(val data: T): DataLayerResponse<T>()
    class Error<T>(val message: String?): DataLayerResponse<T>()
    class Initial<T>(val message: String): DataLayerResponse<T>()
}

Create MicrophonePermissionScreen.kt for the microphone permission boilerplate:

MicrophonePermissionScreen.kt (kotlin)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
package io.getstream.ai.audiodemo

@OptIn(ExperimentalPermissionsApi::class)
@Composable
fun MicrophonePermissionScreen(onPermissionGranted: () -> Unit) {
    val microphonePermissionState =
        rememberPermissionState(android.Manifest.permission.RECORD_AUDIO)

    LaunchedEffect(Unit) {
        if (!microphonePermissionState.status.isGranted) {
            microphonePermissionState.launchPermissionRequest()
        }
    }

    LaunchedEffect(microphonePermissionState.status.isGranted) {
        if (microphonePermissionState.status.isGranted) {
            onPermissionGranted()
        }
    }

    Column(
        modifier = Modifier
            .fillMaxSize()
            .padding(16.dp),
        horizontalAlignment = Alignment.CenterHorizontally,
        verticalArrangement = Arrangement.Center
    ) {
        when {
            microphonePermissionState.status.isGranted -> {}
            microphonePermissionState.status.shouldShowRationale -> {
                Column(horizontalAlignment = Alignment.CenterHorizontally) {
                    Text(
                        text = "Microphone access is required to proceed.",
                        fontSize = 18.sp,
                        color = Color.White
                    )
                    Spacer(modifier = Modifier.height(8.dp))
                    Button(onClick = { microphonePermissionState.launchPermissionRequest() }) {
                        Text(text = "Grant Permission", color = Color.White)
                    }
                }
            }

            else -> {
                Column(horizontalAlignment = Alignment.CenterHorizontally) {
                    Text(
                        text = "Microphone permission is needed to proceed.",
                        fontSize = 18.sp,
                        color = Color.White
                    )
                    Spacer(modifier = Modifier.height(8.dp))
                    Button(onClick = { microphonePermissionState.launchPermissionRequest() }) {
                        Text(text = "Request Permission", color = Color.White)
                    }
                }
            }
        }
    }
}

Create themes.xml under res/values so the background is black:

themes.xml (xml)

1
2
3
4
5
6
<?xml version="1.0" encoding="utf-8"?>
<resources>
    <style name="Theme.AiVoiceDemo" parent="android:Theme.Material.Light.NoActionBar" >
        <item name="android:windowBackground">@color/black</item>
    </style>
</resources>

Create call_end.xml under res/drawable — the disconnect icon:

call_end.xml (xml)

1
2
3
<vector xmlns:android="http://schemas.android.com/apk/res/android" android:height="24dp" android:tint="#000000" android:viewportHeight="24" android:viewportWidth="24" android:width="24dp">
  <path android:fillColor="@android:color/white" android:pathData="M12,9c-1.6,0 -3.15,0.25 -4.6,0.72v3.1c0,0.39 -0.23,0.74 -0.56,0.9 -0.98,0.49 -1.87,1.12 -2.66,1.85 -0.18,0.18 -0.43,0.28 -0.7,0.28 -0.28,0 -0.53,-0.11 -0.71,-0.29L0.29,13.08c-0.18,-0.17 -0.29,-0.42 -0.29,-0.7 0,-0.28 0.11,-0.53 0.29,-0.71C3.34,8.78 7.46,7 12,7s8.66,1.78 11.71,4.67c0.18,0.18 0.29,0.43 0.29,0.71 0,0.28 -0.11,0.53 -0.29,0.71l-2.48,2.48c-0.18,0.18 -0.43,0.29 -0.71,0.29 -0.27,0 -0.52,-0.11 -0.7,-0.28 -0.79,-0.74 -1.69,-1.36 -2.67,-1.85 -0.33,-0.16 -0.56,-0.5 -0.56,-0.9v-3.1C15.15,9.25 13.6,9 12,9z"/>  
</vector>

Because we test against localhost, add network_security_config.xml under res/xml:

network_security_config.xml (xml)

1
2
3
4
5
6
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
  <domain-config cleartextTrafficPermitted="true">
    <domain includeSubdomains="true">10.0.2.2</domain>
  </domain-config>
</network-security-config>

Update AndroidManifest.xml:

AndroidManifest.xml (xml)

1
2
3
<uses-permission android:name="android.permission.INTERNET" />

<application android:networkSecurityConfig="@xml/network_security_config" .../>

Add the Credentials model that reflects the server response:

kotlin

1
2
3
4
5
6
7
8
9
package io.getstream.ai.audiodemo

data class Credentials(
     val apiKey: String,
     val token: String,
     val callId: String,
     val callType: String,
     val userId: String
)

And a CallUiState enum for the different UI states:

kotlin

1
2
3
enum class CallUiState {
    IDLE, JOINING, ACTIVE, ERROR
}

Step 4.2 — Fetching the credentials

initCredentials() (already in MainViewModel.kt) fetches credentials from the server:

MainViewModel.kt (kotlin)

1
2
3
4
5
6
7
8
9
10
11
fun initCredentials() {
    viewModelScope.launch {
        try {
            credentials.emit(DataLayerResponse.Initial("Getting Credentials"))
            val response = RetrofitInstance.api.getCredentials()
            createStreamClient(response)
            credentials.emit(DataLayerResponse.Success(response))
        } catch (ex: Exception) {
            credentials.emit(DataLayerResponse.Error(ex.message))
        }
  }

Note: BASE_URL uses 10.0.2.2 (the emulator's alias for your host machine's localhost), so the emulator is the simplest way to test. On a real device, set BASE_URL to your computer's LAN IP, put both on the same WiFi, and allow cleartext to that host.

Step 4.3 — Connecting to Stream Video

createStreamClient builds the StreamVideo client from the fetched credentials:

kotlin

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
private fun createStreamClient(credentials: Credentials) {
      val userId = credentials.userId
      
      val user = User(
          id = userId, 
          name = "Tutorial", 
          image = "https://bit.ly/2TIt8NR",
      )

      client = StreamVideoBuilder(
          context = app.applicationContext,
          apiKey = credentials.apiKey,
          geo = GEO.GlobalEdgeNetwork,
          user = user,
          token = credentials.token,
      ).build()
}

We call this on the appearance of the root view in the next step.

Step 5 — Building the UI

Replace MainActivity.kt with:

MainActivity.kt (kotlin)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
class MainActivity : ComponentActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        enableEdgeToEdge()

        setContent {
            var isPermissionGranted by remember { mutableStateOf(false) }
            if (isPermissionGranted) {
                Body()
            } else {
                MicrophonePermissionScreen(onPermissionGranted = {
                    isPermissionGranted = true
                })
            }
        }
    }
}

@Composable
fun Body() {
    val viewModel: MainViewModel = viewModel(key = "MainViewModel")
    LaunchedEffect(Unit) {
        viewModel.initCredentials()
    }
    val creds = viewModel.credentials.collectAsStateWithLifecycle()
    when (creds.value) {
        is DataLayerResponse.Initial -> {
            val message = (creds.value as DataLayerResponse.Initial<Credentials>).message
            Box(
                Modifier
                    .fillMaxSize()
                    .background(Color.Black),
                contentAlignment = Alignment.Center
            ) {
                Text(message, color = Color.White)
            }
        }

        is DataLayerResponse.Success -> {
            Box(
                modifier = Modifier
                    .fillMaxSize()
                    .background(Color.Black)
            ) {
                AiContentUi()
            }

        }

        is DataLayerResponse.Error -> {
            Box(
                Modifier
                    .fillMaxSize()
                    .background(Color.Black),
                contentAlignment = Alignment.Center
            ) {
                Text("Getting Credentials Failed", color = Color.White)
            }
        }
    }
}

@Composable
fun AiContentUi() {
    val viewModel: MainViewModel = viewModel(key = "MainViewModel")
    val callUiState by viewModel.callUiState.collectAsStateWithLifecycle()

    when (callUiState) {
        CallUiState.IDLE -> {
            Box(Modifier.fillMaxSize(), contentAlignment = Alignment.Center) {
                Button(colors = ButtonDefaults.buttonColors()
                    .copy(contentColor = Color.White, containerColor = Color.Black),
                    onClick = {
                        viewModel.joinCall()
                    }) {
                    Text("Click to talk to Ai", color = Color.White)
                }
            }

        }

        CallUiState.JOINING -> {
            Box(Modifier.fillMaxSize(), contentAlignment = Alignment.Center) {
                Row(
                    horizontalArrangement = Arrangement.Center,
                    verticalAlignment = Alignment.CenterVertically
                ) {
                    Text(
                        "Waiting for AI Agent to Join...",
                        color = Color.White,
                        modifier = Modifier.padding(bottom = 12.dp)
                    )
                    Spacer(Modifier.width(8.dp))
                    CircularProgressIndicator(
                        modifier = Modifier.width(24.dp),
                        color = MaterialTheme.colorScheme.secondary,
                        trackColor = MaterialTheme.colorScheme.surfaceVariant,
                    )
                }
            }
        }

        CallUiState.ACTIVE -> {
            Box(Modifier.fillMaxSize()) {
                AISpeakingView(viewModel.call?.state!!)
                CallEndButton(modifier = Modifier
                    .align(Alignment.BottomEnd)
                    .padding(12.dp), onClick = {
                    viewModel.disconnect()
                })
            }
        }

        CallUiState.ERROR -> {
            Box(Modifier.fillMaxSize(), contentAlignment = Alignment.Center) {
                Column(horizontalAlignment = Alignment.CenterHorizontally) {
                    Text("Something went wrong, please re-try", color = Color.White)
                    Button(colors = ButtonDefaults.buttonColors()
                        .copy(contentColor = Color.White, containerColor = Color.Black),
                        onClick = {
                            viewModel.joinCall()
                        }) {
                        Text("Connect to Ai Agent", color = Color.White)
                    }
                }
            }
        }
    }
}

@Composable
fun CallEndButton(modifier: Modifier, onClick: () -> Unit) {
    IconButton(
        onClick = onClick,
        modifier = modifier
            .size(56.dp)
            .clip(CircleShape)
            .background(Color.Red)
    ) {
        Icon(
            painter = painterResource(id = R.drawable.call_end),
            contentDescription = "End call",
            tint = Color.White
        )
    }
}

enum class CallUiState {
    IDLE, JOINING, ACTIVE, ERROR
}

We connect to StreamVideo via viewModel.initCredentials(), then render UI based on callUiState:

IDLE — a "Click to talk to AI" button that calls joinCall().
JOINING — a progress spinner.
ACTIVE — the AISpeakingView audio visualization plus a CallEndButton overlay.
ERROR — a retry button.

Declare AISpeakingView with a TODO for now:

kotlin

1
2
3
4
@Composable
fun AISpeakingView(callState: CallState) {
	//TODO
}

🧑 Human checkpoint: the microphone prompt is an Android system dialog. Agents: install the app (with the Node server from Step 2 running), then ask the human to launch it and allow microphone access.

✅ Checkpoint: run the app, tap Click to talk to AI, and after "Waiting for AI Agent to Join..." you can converse with the agent. Next we add the audio visualization.

Step 6 — Visualizing the audio levels

AISpeakingView listens to each participant's audio levels from the call state and visualizes them with a glowing animation that expands/contracts with voice amplitude.

Step 6.1 — AISpeakingView

Replace the TODO with:

kotlin

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
enum class SpeakerState {
    AI_SPEAKING,
    USER_SPEAKING,
    IDLE
}

fun SpeakerState.gradientColors(): List<Color> {
    return when (this) {
        SpeakerState.USER_SPEAKING -> listOf(
            Color.Red,
            Color.Red.copy(alpha = 0f)
        )
        else -> listOf(
            Color(0f, 0.976f, 1f),
            Color(0f, 0.227f, 1f, 0f)
        )
    }
}

@Composable
fun AISpeakingView(callState: CallState) {
    val agentId = "lucy"

    var amplitude by remember { mutableFloatStateOf(0f) }
    var audioLevels by remember { mutableStateOf(listOf<Float>()) }
    var speakerState by remember { mutableStateOf(SpeakerState.IDLE) }

    GlowView(amplitude, speakerState)

    LaunchedEffect(Unit) {

        callState.activeSpeakers.collectLatest { speakers->
            speakers.forEach { speaker->
                speaker.audioLevels.collectLatest { audioLevel->
                    if(speaker.userId.value.contains("lucy")) {
                        if(speakerState!=SpeakerState.AI_SPEAKING) {
                            speakerState = SpeakerState.AI_SPEAKING
                        }
                    } else {
                        if(speakerState!=SpeakerState.USER_SPEAKING) {
                            speakerState = SpeakerState.USER_SPEAKING
                        }
                    }
                    audioLevels = audioLevel
                    amplitude =
                        computeSingleAmplitude(audioLevels) * getRandomFloatInRange(1f, 2f)
                }
            }
        }
    }

    LaunchedEffect(Unit) {

        callState.activeSpeakers.collectLatest { speaker->
            val aiSpeaker = speaker.find { it.userId.value.contains(agentId) }

            // Find the local user speaking
            val localSpeaker = speaker.find { it.userId.value == callState.me.value!!.userId.value }
            if(aiSpeaker == null && localSpeaker == null){
                speakerState = SpeakerState.IDLE
                audioLevels = emptyList()
                amplitude = 0f
            }
        }
    }
}

fun computeSingleAmplitude(levels: List<Float>): Float {
    val normalized = normalizePeak(levels)
    if (normalized.isEmpty()) return 0f

    return normalized.average().toFloat()
}

fun normalizePeak(levels: List<Float>): List<Float> {
    val maxLevel = levels.maxOfOrNull { abs(it) } ?: return levels
    return if (maxLevel > 0) levels.map { it / maxLevel } else levels
}

We call the agent "lucy" and filter it out from the current user, showing blue when the AI speaks and red when the user speaks (via SpeakerState). The audioLevels array drives the visualization and computes the amplitude passed to the glow view.

Step 6.2 — GlowView

GlowView draws three animated glow layers. Each has min/max radius, brightness, blur, opacity, and wave length. Adjust to taste.

kotlin

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
@Composable
fun GlowView(
    amplitude: Float,
    speakerState: SpeakerState = SpeakerState.AI_SPEAKING
) {
    // Animated amplitude for smooth transitions
    val animatedAmplitude = remember { Animatable(0f) }

    // Update animated amplitude when input amplitude changes
    LaunchedEffect(amplitude) {
        animatedAmplitude.animateTo(
            targetValue = amplitude,
            animationSpec = tween(600, easing = EaseInOut)
        )
    }

    // Continuous time value for wave animation
    var time by remember { mutableStateOf(0f) }

    // Rotation animation
    val infiniteRotation = rememberInfiniteTransition()
    val rotationAngle by infiniteRotation.animateFloat(
        initialValue = 0f,
        targetValue = 360f,
        animationSpec = infiniteRepeatable(
            animation = tween(10000, easing = LinearEasing),
            repeatMode = RepeatMode.Restart
        )
    )

    // Time progression for wave effect
    LaunchedEffect(Unit) {
        while (true) {
            time = (time + 0.005f) % 1.0f
            delay(16L)
        }
    }

    val gradientColors = speakerState.gradientColors()

    Box(
        modifier = Modifier
            .fillMaxSize()
            .background(Color.Black)
    ) {
        Canvas(modifier = Modifier.fillMaxSize()) {
            val center = Offset(size.width / 2, size.height / 2)

            // Outer Layer
            drawGlowLayer(
                center = center,
                baseRadiusMin = size.minDimension * 0.30f,
                baseRadiusMax = size.minDimension * 0.50f,
                blurRadius = 60f,
                baseOpacity = 0.35f,
                scaleRange = 0.3f,
                waveRangeMin = 0.2f,
                waveRangeMax = 0.02f,
                time = time,
                amplitude = animatedAmplitude.value,
                rotationAngle = rotationAngle,
                gradientColors = gradientColors
            )

            // Middle Layer
            drawGlowLayer(
                center = center,
                baseRadiusMin = size.minDimension * 0.20f,
                baseRadiusMax = size.minDimension * 0.30f,
                blurRadius = 40f,
                baseOpacity = 0.55f,
                scaleRange = 0.3f,
                waveRangeMin = 0.15f,
                waveRangeMax = 0.03f,
                time = time,
                amplitude = animatedAmplitude.value,
                rotationAngle = rotationAngle,
                gradientColors = gradientColors
            )

            // Inner Core Layer
            drawGlowLayer(
                center = center,
                baseRadiusMin = size.minDimension * 0.10f,
                baseRadiusMax = size.minDimension * 0.20f,
                blurRadius = 20f,
                baseOpacity = 0.9f,
                scaleRange = 0.5f,
                waveRangeMin = 0.35f,
                waveRangeMax = 0.05f,
                time = time,
                amplitude = animatedAmplitude.value,
                rotationAngle = rotationAngle,
                gradientColors = gradientColors
            )
        }
    }
}

private fun androidx.compose.ui.graphics.drawscope.DrawScope.drawGlowLayer(
    center: Offset,
    baseRadiusMin: Float,
    baseRadiusMax: Float,
    blurRadius: Float,
    baseOpacity: Float,
    scaleRange: Float,
    waveRangeMin: Float,
    waveRangeMax: Float,
    time: Float,
    amplitude: Float,
    rotationAngle: Float,
    gradientColors: List<Color>
) {
    // Calculate the actual radius based on amplitude
    val baseRadius = lerp(baseRadiusMin, baseRadiusMax, amplitude)

    // Calculate wave range (inverse relationship to amplitude)
    val waveRange = lerp(waveRangeMax, waveRangeMin, 1 - amplitude)

    // Calculate the scale factors
    val shapeWaveSin = sin(2 * PI * time).toFloat()
    val shapeWaveCos = cos(2 * PI * time).toFloat()

    // Scale from amplitude
    val amplitudeScale = 1.0f + scaleRange * amplitude

    // Final x/y scale = amplitude scale + wave
    val xScale = (amplitudeScale + waveRange * shapeWaveSin)
    val yScale = (amplitudeScale + waveRange * shapeWaveCos)

    // Draw the oval with gradient
    drawIntoCanvas { canvas ->
        val paint = androidx.compose.ui.graphics.Paint().asFrameworkPaint().apply {
            shader = RadialGradient(
                center.x, center.y, baseRadius,
                intArrayOf(
                    gradientColors[0].copy(alpha = 0.9f).toArgb(),
                    gradientColors[1].toArgb()
                ),
                floatArrayOf(0f, 1f),
                Shader.TileMode.CLAMP
            )
            alpha = (baseOpacity * 255).toInt()
        }

        // Apply blur
        paint.maskFilter = BlurMaskFilter(blurRadius, BlurMaskFilter.Blur.NORMAL)

        // Save the current state, rotate, draw, and restore
        canvas.nativeCanvas.save()
        canvas.nativeCanvas.rotate(rotationAngle, center.x, center.y)

        // Draw the oval with calculated dimensions
        canvas.nativeCanvas.drawOval(
            center.x - baseRadius * xScale,
            center.y - baseRadius * yScale,
            center.x + baseRadius * xScale,
            center.y + baseRadius * yScale,
            paint
        )

        canvas.nativeCanvas.restore()
    }
}

fun getRandomFloatInRange(min: Float, max: Float): Float {
    return Random.nextFloat() * (max - min) + min
}

✅ Checkpoint: run the app and talk to the AI — the glow expands with voice amplitude, turning blue when the agent speaks and red when you do.

You can find the Node.js backend source and the completed Android app on GitHub.

Verify the whole build

Backend: npm run server is running, and curl http://localhost:3000/credentials returns JSON.
App: ./gradlew :app:installDebug, launch, grant microphone access.
End to end: tap Click to talk to AI → "Waiting for AI Agent to Join..." → speak → the agent replies and the glow reacts (blue for the agent, red for you).

Troubleshooting

App can't reach the server — the Node server isn't running, or BASE_URL is wrong. On the emulator use http://10.0.2.2:3000/; on a device use your host's LAN IP and same WiFi.
CLEARTEXT communication ... not permitted — missing network_security_config.xml or the networkSecurityConfig attribute on <application> (Step 4.1).
Agent never joins — the /connect call failed. Check the server logs; confirm OPENAI_API_KEY and the Stream key/secret in .env are valid.
No microphone prompt — permission denied. Grant microphone in system settings; MicrophonePermissionScreen re-requests it.
stream project is not initialized (if using the CLI for keys) — run getstream init first.
Build errors on stream version — you left <latest_version> in libs.versions.toml. Set it to a real release from the Releases page.

Recap

You built an app that lets you talk with an AI bot in an audio call. Stream's wrapper handles the WebSocket communication with OpenAI's realtime API, so you connect an agent in a few steps, join the call from Android, and visualize audio levels.

Both the Video SDK for Android and the API have much more for advanced use cases.

Next Steps

Other platforms: React, iOS, Flutter, React Native.
Backend documentation with JS and Python examples.
Android SDK documentation for additional features.

Machine-readable resources

For AI agents and coding assistants working with this SDK:

CLI + skills: curl -fsSL https://getstream.io/cli.sh | bash, then getstream skills --universal and getstream skills stream-android --universal for the Android pack (use --claude instead for Claude Code). (Alternative: npx skills add GetStream/agent-skills -s stream.)
Backend: Stream Node SDK; streamClient.video.connectOpenAi(...) returns OpenAI's official realtime client
Secrets: OpenAI key and Stream secret live only on the Node backend, never in the Android app
Android integration skill: invoke /stream-android in your agent; /stream-docs searches live SDK docs with citations
Docs index for LLMs: https://getstream.io/video/docs/android/llms.txt
Markdown endpoints: append .md to any docs URL for a clean, token-efficient version
Source of truth for APIs: the Android SDK repository and the OpenAI realtime backend docs

Android AI Voice Assistant Tutorial

Choose your path

Path A — Let your AI agent build it (recommended)

Path B — Build it manually

Important Building Blocks

Part 1 — The Node backend

Step 1 — Connect an AI agent to Stream from your backend

Step 1.1 — OpenAI and Stream credentials

Step 1.2 — Create the Node.js project

Step 1.3 — Install the dependencies

Step 1.4 — Set up the credentials

Step 1.5 — Implement the standalone-ui script

Step 1.6 — Running the sample

Step 2 — Set up your server-side integration

Step 2.1 — Implement `server.mjs`

Step 2.2 — Running the server

Part 2 — The Android app

Step 3 — Setting up the Android project

Step 3.1 — Create the project & add the Stream Video dependency

Step 3.2 — Add the other dependencies

Step 4 — Stream Video Setup

Step 4.1 — Declaring the required properties

Step 4.2 — Fetching the credentials

Step 4.3 — Connecting to Stream Video

Step 5 — Building the UI

Step 6 — Visualizing the audio levels

Step 6.1 — AISpeakingView

Step 6.2 — GlowView

Verify the whole build

Troubleshooting

Recap

Next Steps

Machine-readable resources

Give us feedback!

Start coding for free

Android AI Voice Assistant Tutorial

Choose your path

Path A — Let your AI agent build it (recommended)

Path B — Build it manually

Important Building Blocks

Part 1 — The Node backend

Step 1 — Connect an AI agent to Stream from your backend

Step 1.1 — OpenAI and Stream credentials

Step 1.2 — Create the Node.js project

Step 1.3 — Install the dependencies

Step 1.4 — Set up the credentials

Step 1.5 — Implement the standalone-ui script

Step 1.6 — Running the sample

Step 2 — Set up your server-side integration

Step 2.1 — Implement server.mjs

Step 2.2 — Running the server

Part 2 — The Android app

Step 3 — Setting up the Android project

Step 3.1 — Create the project & add the Stream Video dependency

Step 3.2 — Add the other dependencies

Step 4 — Stream Video Setup

Step 4.1 — Declaring the required properties

Step 4.2 — Fetching the credentials

Step 4.3 — Connecting to Stream Video

Step 5 — Building the UI

Step 6 — Visualizing the audio levels

Step 6.1 — AISpeakingView

Step 6.2 — GlowView

Verify the whole build

Troubleshooting

Recap

Next Steps

Machine-readable resources

Give us feedback!

Start coding for free

Step 2.1 — Implement `server.mjs`