# Personal Assistant

A personal assistant knows what's on the user's plate. It reads their calendar, adds their reminders, searches their email, keeps every conversation they've had with it, and listens when they'd rather talk than type.

>
> **Note:** **What you'll build**
>
> - A side menu with the user's conversations, titled and searchable
> - The user's calendar and reminders, as tools the agent runs on the phone
> - A question before the agent adds a reminder
> - The user's Gmail, which they connect from the conversation
> - Photos the user attaches to a question
> - A voice mode in the same conversation
>

**Starts from:** the [Quickstart](https://getstream.io/chat/docs/sdk/flutter/ai-components/), with `agents`, `AgentMessage` and `AgentTransport` in your project, and the `VoiceCallScreen` of the [Voice agent](https://getstream.io/chat/docs/sdk/flutter/ai-components/use-cases/voice-agent/) use case. `agents` is the `StreamAgents` its `signIn` returns, kept where your screens reach it, such as `final agents = signIn('jane');` in `lib/agents.dart`. The code below imports the others from `lib/agent_transport.dart`. The calendar tools reach EventKit on iOS and the calendar provider on Android through a platform channel, in an app created with `flutter create --org com.example assistant`, whose id `com.example.assistant` the sign-ins come back to.

## Best practices

- Run on the phone what only the phone has, such as the calendar, reminders or location. Only a tool's answer leaves the phone.
- Ask for each permission when a tool first needs it, not when the app starts.
- Ask before a tool that changes something, such as adding a reminder.
- Let users connect their own accounts, such as Gmail, from the conversation. The agent never sees their password.

## Define the assistant

Make the agent from the voice template, so it can talk as well as write, and give it each user's Gmail as a connector:

```sh
getstream agents init assistant --template voice
```

```yaml title="agents/assistant/agent.yaml"
name: assistant
mode: voice
llm: llm-fast
connectors:
  - name: gmail # each user signs in to their own Gmail
    tools: [search_threads, get_thread, get_message]
```

Write what the assistant is for in `instructions.md`, then sync it with `getstream agents sync assistant`. [User connection](https://getstream.io/agents/docs/connectors/each-user-account/?client=flutter) lists the other connectors, such as Google Calendar and Notion.

## Give the agent the calendar and reminders

A tool the app gives the agent runs on the phone. When the model calls it, its `run` function runs and returns the answer the model sees. On iOS, `add_reminder` adds a reminder to the user's Reminders app. Android has no reminders app of its own for an app to add to, so there it adds an event to the user's calendar, with an alert when it's due. It declares an approval, so the user is asked first, with the reminder's title.

Dart can't read the calendar on its own, so the tools ask the app's iOS and Android code over a `MethodChannel`. On Android, a tool asks for the calendar the first time it needs it, with `permission_handler`; on iOS, EventKit asks by itself. Add `permission_handler` to your `pubspec.yaml`:

```yaml title="pubspec.yaml"
dependencies:
  # Asks for the calendar, the microphone and the camera on Android.
  permission_handler: ^12.0.3
```

Then the tools:

```dart title="lib/assistant_tools.dart"
import 'dart:io' show Platform;

import 'package:flutter/services.dart';
import 'package:permission_handler/permission_handler.dart';
import 'package:stream_agents_core/stream_agents_core.dart';

/// The calendar and reminders, which the app's iOS and Android code read and write.
const _calendar = MethodChannel('com.example.assistant/calendar');

/// Whether the app may use the calendar, asking the first time on Android.
Future<bool> _calendarAllowed() async =>
    !Platform.isAndroid || await Permission.calendarFullAccess.request().isGranted;

/// Calls the app's calendar code. What it couldn't do comes back as words for the model.
Future<T?> _ask<T>(String method, Map<String, Object?> arguments) async {
  try {
    return await _calendar.invokeMethod<T>(method, arguments);
  } on PlatformException catch (error) {
    throw ToolFailure(error.message ?? 'The phone could not do that just now.');
  }
}

final listEvents = AgentTool(
  name: 'list_events',
  description: "The events on the user's calendar for one day.",
  parameters: AgentTool.strings({'day': 'the day, as YYYY-MM-DD'}, required: ['day']),
  run: (arguments) async {
    if (!await _calendarAllowed()) return 'No access to the calendar.';
    final lines = await _ask<List<Object?>>('listEvents', {'day': '${arguments['day'] ?? ''}'});
    return (lines ?? const []).join('\n');
  },
);

// Android has no reminders app of its own for an app to add to, so there a reminder is an
// event in the user's calendar, with an alert when it's due.
final addReminder = AgentTool(
  name: 'add_reminder',
  description: Platform.isIOS
      ? "Adds a reminder to the user's Reminders app."
      : "Adds a reminder to the user's calendar, with an alert when it's due.",
  parameters: AgentTool.strings(
    {'title': 'what to remind them of', 'due': 'when, in ISO 8601 with the UTC offset'},
    required: ['title', 'due'],
  ),
  approval: const ToolApproval(title: 'Add this reminder?', reasonArgument: 'title'),
  run: (arguments) async {
    if (!await _calendarAllowed()) return 'No access to the calendar.';
    final due = DateTime.tryParse('${arguments['due'] ?? ''}');
    if (due == null) {
      throw const ToolFailure('The reminder needs a due time, such as 2026-10-16T18:00:00+02:00.');
    }
    await _ask<void>('addReminder', {
      'title': '${arguments['title'] ?? ''}',
      'due': due.millisecondsSinceEpoch,
    });
    return 'Added the reminder.';
  },
);

/// The tools the assistant may call, which run on this phone.
final assistantTools = [listEvents, addReminder];

/// Why a tool couldn't answer, in words the model passes on.
class ToolFailure implements Exception {
  const ToolFailure(this.message);

  final String message;

  @override
  String toString() => message;
}
```

A tool that throws, such as on a day it can't read, tells the model it failed and why. The app's own code answers the channel. Replace the `AppDelegate.swift` and `MainActivity.kt` that `flutter create` made:

```swift label="iOS" title="ios/Runner/AppDelegate.swift"
import EventKit
import Flutter
import UIKit

@main
@objc class AppDelegate: FlutterAppDelegate, FlutterImplicitEngineDelegate {
  private let calendar = CalendarChannel()

  override func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
  ) -> Bool {
    return super.application(application, didFinishLaunchingWithOptions: launchOptions)
  }

  func didInitializeImplicitFlutterEngine(_ engineBridge: FlutterImplicitEngineBridge) {
    GeneratedPluginRegistrant.register(with: engineBridge.pluginRegistry)
    if let registrar = engineBridge.pluginRegistry.registrar(forPlugin: "CalendarChannel") {
      calendar.register(with: registrar.messenger())
    }
  }
}

/// The calendar and the reminders, for the agent's list_events and add_reminder tools.
final class CalendarChannel {
  private let store = EKEventStore()
  private var channel: FlutterMethodChannel?

  func register(with messenger: FlutterBinaryMessenger) {
    let channel = FlutterMethodChannel(name: "com.example.assistant/calendar", binaryMessenger: messenger)
    channel.setMethodCallHandler { [weak self] call, result in
      self?.handle(call, result: result)
    }
    self.channel = channel
  }

  private func handle(_ call: FlutterMethodCall, result: @escaping FlutterResult) {
    let arguments = call.arguments as? [String: Any] ?? [:]
    Task { @MainActor in
      do {
        switch call.method {
        case "listEvents":
          result(try await events(on: arguments["day"] as? String ?? ""))
        case "addReminder":
          let due = (arguments["due"] as? NSNumber).map { Date(timeIntervalSince1970: $0.doubleValue / 1000) }
          try await addReminder(title: arguments["title"] as? String ?? "", due: due)
          result(nil)
        default:
          result(FlutterMethodNotImplemented)
        }
      } catch let failure as ToolFailure {
        result(FlutterError(code: "tool_failure", message: failure.message, details: nil))
      } catch {
        result(FlutterError(code: "tool_failure", message: error.localizedDescription, details: nil))
      }
    }
  }

  /// What is on the calendar for one day, each as a line: its time and its title.
  private func events(on day: String) async throws -> [String] {
    let isoDay = DateFormatter()
    isoDay.locale = Locale(identifier: "en_US_POSIX")
    isoDay.dateFormat = "yyyy-MM-dd"
    guard let start = isoDay.date(from: day) else {
      throw ToolFailure("The day has to be a date like 2026-10-16.")
    }
    let allowed: Bool
    if #available(iOS 17, *) {
      allowed = try await store.requestFullAccessToEvents()
    } else {
      allowed = try await store.requestAccess(to: .event)
    }
    guard allowed else { throw ToolFailure("No access to the calendar.") }
    let end = Calendar.current.date(byAdding: .day, value: 1, to: start) ?? start
    let found = store.events(matching: store.predicateForEvents(withStart: start, end: end, calendars: nil))
    let time = DateFormatter()
    time.dateStyle = .none
    time.timeStyle = .short
    return found.map { event in
      let when = event.isAllDay ? "All day" : time.string(from: event.startDate)
      return "\(when) \(event.title ?? "")"
    }
  }

  /// Adds a reminder to the user's Reminders, with an alert when it is due.
  private func addReminder(title: String, due: Date?) async throws {
    let allowed: Bool
    if #available(iOS 17, *) {
      allowed = try await store.requestFullAccessToReminders()
    } else {
      allowed = try await store.requestAccess(to: .reminder)
    }
    guard allowed else { throw ToolFailure("No access to Reminders.") }
    let reminder = EKReminder(eventStore: store)
    reminder.title = title
    reminder.calendar = store.defaultCalendarForNewReminders()
    if let due {
      reminder.dueDateComponents = Calendar.current.dateComponents([.year, .month, .day, .hour, .minute], from: due)
      reminder.addAlarm(EKAlarm(absoluteDate: due))
    }
    try store.save(reminder, commit: true)
  }
}

/// Why a tool couldn't answer, in words the model passes on.
struct ToolFailure: Error {
  let message: String

  init(_ message: String) {
    self.message = message
  }
}
```

```kotlin label="Android" title="android/app/src/main/kotlin/com/example/assistant/MainActivity.kt"
package com.example.assistant

import android.content.ContentUris
import android.content.ContentValues
import android.content.Context
import android.os.Handler
import android.os.Looper
import android.provider.CalendarContract
import io.flutter.embedding.android.FlutterActivity
import io.flutter.embedding.engine.FlutterEngine
import io.flutter.plugin.common.MethodCall
import io.flutter.plugin.common.MethodChannel
import java.text.DateFormat
import java.text.ParseException
import java.text.SimpleDateFormat
import java.util.Calendar
import java.util.Date
import java.util.Locale
import java.util.TimeZone
import java.util.concurrent.Executors

class MainActivity : FlutterActivity() {
    override fun configureFlutterEngine(flutterEngine: FlutterEngine) {
        super.configureFlutterEngine(flutterEngine)
        val calendar = CalendarChannel(applicationContext)
        MethodChannel(flutterEngine.dartExecutor.binaryMessenger, CalendarChannel.NAME)
            .setMethodCallHandler(calendar::handle)
    }
}

/** The calendar, for the agent's list_events and add_reminder tools. */
class CalendarChannel(context: Context) {
    private val context = context.applicationContext
    private val main = Handler(Looper.getMainLooper())

    fun handle(call: MethodCall, result: MethodChannel.Result) {
        executor.execute {
            try {
                val answer: Any? = when (call.method) {
                    "listEvents" -> eventsOn(call.argument<String>("day").orEmpty())
                    "addReminder" -> {
                        val due = call.argument<Number>("due")?.toLong()
                            ?: throw ToolFailure("A reminder needs a due time.")
                        addReminder(call.argument<String>("title").orEmpty(), Date(due))
                        null
                    }
                    else -> {
                        main.post { result.notImplemented() }
                        return@execute
                    }
                }
                main.post { result.success(answer) }
            } catch (failure: ToolFailure) {
                main.post { result.error("tool_failure", failure.message, null) }
            } catch (failure: SecurityException) {
                main.post { result.error("tool_failure", "No access to the calendar.", null) }
            } catch (failure: Exception) {
                main.post { result.error("tool_failure", "The phone could not use its calendar just now.", null) }
            }
        }
    }

    /** The events on [day], each as a line: its time and its title. */
    private fun eventsOn(day: String): List<String> {
        val start = try {
            SimpleDateFormat("yyyy-MM-dd", Locale.US).apply { isLenient = false }.parse(day)
        } catch (_: ParseException) {
            null
        } ?: throw ToolFailure("The day has to be a date like 2026-10-16.")
        val end = Calendar.getInstance().apply {
            time = start
            add(Calendar.DAY_OF_MONTH, 1)
        }.timeInMillis
        val time = DateFormat.getTimeInstance(DateFormat.SHORT)
        val columns = arrayOf(CalendarContract.Instances.BEGIN, CalendarContract.Instances.TITLE)
        val rows = CalendarContract.Instances.query(context.contentResolver, columns, start.time, end)
            ?: throw ToolFailure("The phone could not read its calendar just now.")
        return rows.use {
            buildList {
                while (it.moveToNext()) add("${time.format(Date(it.getLong(0)))} ${it.getString(1).orEmpty()}")
            }
        }
    }

    /** Puts the reminder in the primary calendar the user can write to, with an alert as it starts. */
    private fun addReminder(title: String, due: Date) {
        val resolver = context.contentResolver
        val calendar = resolver.query(
            CalendarContract.Calendars.CONTENT_URI,
            arrayOf(CalendarContract.Calendars._ID),
            "${CalendarContract.Calendars.CALENDAR_ACCESS_LEVEL} >= ${CalendarContract.Calendars.CAL_ACCESS_CONTRIBUTOR}",
            null,
            "${CalendarContract.Calendars.IS_PRIMARY} DESC",
        )?.use { rows -> if (rows.moveToFirst()) rows.getLong(0) else null }
            ?: throw ToolFailure("There's no calendar to add the reminder to.")
        val event = resolver.insert(
            CalendarContract.Events.CONTENT_URI,
            ContentValues().apply {
                put(CalendarContract.Events.CALENDAR_ID, calendar)
                put(CalendarContract.Events.TITLE, title)
                put(CalendarContract.Events.DTSTART, due.time)
                put(CalendarContract.Events.DTEND, due.time + 15 * 60_000L)
                put(CalendarContract.Events.EVENT_TIMEZONE, TimeZone.getDefault().id)
                put(CalendarContract.Events.HAS_ALARM, 1)
            },
        ) ?: throw ToolFailure("The calendar didn't take the reminder.")
        resolver.insert(
            CalendarContract.Reminders.CONTENT_URI,
            ContentValues().apply {
                put(CalendarContract.Reminders.EVENT_ID, ContentUris.parseId(event))
                put(CalendarContract.Reminders.MINUTES, 0)
                put(CalendarContract.Reminders.METHOD, CalendarContract.Reminders.METHOD_ALERT)
            },
        )
    }

    companion object {
        const val NAME = "com.example.assistant/calendar"

        /** Where the calendar is read and written, off the main thread. */
        private val executor = Executors.newSingleThreadExecutor()
    }
}

/** Why a tool couldn't answer, in words the model passes on. */
class ToolFailure(message: String) : Exception(message)
```

Add `NSCalendarsFullAccessUsageDescription`, `NSCalendarsUsageDescription`, `NSRemindersFullAccessUsageDescription` and `NSRemindersUsageDescription` to your `Info.plist`, and `READ_CALENDAR` and `WRITE_CALENDAR` to your `AndroidManifest.xml`.

## Open a conversation

Pass the tools every time a conversation opens, new or carried on. A new one also names where a connector's sign-in comes back to, in your app's own URL scheme:

```dart title="lib/open_conversation.dart"
import 'package:stream_agents_core/stream_agents_core.dart';

import 'agent_transport.dart';
import 'agents.dart';
import 'assistant_tools.dart';

Future<AgentTransport> openConversation(Session? session) async {
  if (session != null) {
    return AgentTransport.open(await agents.attach(session.id, tools: assistantTools));
  }
  final options = SessionOptions(tools: assistantTools, returnUrl: 'com.example.assistant:/connected');
  return AgentTransport.open(await agents.agent('assistant').chat(options));
}
```

## List the user's conversations

Each conversation is a session, which the router titles after its first answers. `ConversationList` lists the user's sessions with the assistant, most recent first, and searches them:

```dart title="lib/conversation_list.dart"
import 'package:flutter/material.dart';
import 'package:stream_agents_core/stream_agents_core.dart';

import 'agents.dart';

class ConversationList extends StatefulWidget {
  const ConversationList({super.key, required this.onSelect});

  /// Called with the conversation picked, or null for a new one.
  final ValueChanged<Session?> onSelect;

  @override
  State<ConversationList> createState() => _ConversationListState();
}

class _ConversationListState extends State<ConversationList> {
  List<Session> _sessions = const [];
  String _search = '';

  /// How many reads there have been, so only the latest one's answer is shown.
  var _reads = 0;

  @override
  void initState() {
    super.initState();
    _load();
  }

  Future<void> _load() async {
    final read = ++_reads;
    final all = agents.agent('assistant').sessions;
    try {
      final page = _search.isEmpty ? await all.query() : await all.search(_search);
      if (!mounted || read != _reads) return;
      // A session that was only a call has no written conversation to open.
      setState(() => _sessions = [for (final session in page.items) if (session.conversationId.isNotEmpty) session]);
    } on Object {
      if (mounted && read == _reads) setState(() => _sessions = const []);
    }
  }

  @override
  Widget build(BuildContext context) => SafeArea(
    child: Column(
      children: [
        Padding(
          padding: const EdgeInsets.all(16),
          child: TextField(
            decoration: const InputDecoration(hintText: 'Search', prefixIcon: Icon(Icons.search)),
            onChanged: (text) {
              _search = text;
              _load();
            },
          ),
        ),
        ListTile(
          leading: const Icon(Icons.edit_outlined),
          title: const Text('New conversation'),
          onTap: () => widget.onSelect(null),
        ),
        Expanded(
          child: ListView(
            children: [
              for (final session in _sessions)
                ListTile(
                  title: Text(session.title.isEmpty ? 'New conversation' : session.title),
                  onTap: () => widget.onSelect(session),
                ),
            ],
          ),
        ),
      ],
    ),
  );
}
```

## Put it together

`AssistantHome` holds the open conversation. Switching to another closes the one before, so the screen stops following it. Closing a session keeps what it recorded, so it opens again from the list. A `Drawer` puts the list beside the open conversation, and the user opens it from the menu button or by dragging from the leading edge. It shows the conversation with `AssistantConversation`, which follows below, and its Talk button opens `VoiceModeScreen`, from [Talk instead of typing](#talk-instead-of-typing). The conversation opens connector sign-ins with `flutter_web_auth_2`, so add it to your `pubspec.yaml`:

```yaml title="pubspec.yaml"
dependencies:
  # Opens a connector's sign-in, and brings the user back to the app.
  flutter_web_auth_2: ^5.1.0
```

Then the home screen:

```dart title="lib/assistant_home.dart"
import 'dart:async';

import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:flutter_web_auth_2/flutter_web_auth_2.dart';
import 'package:stream_chat_flutter_ai/stream_chat_flutter_ai.dart';
import 'package:stream_agents_core/stream_agents_core.dart';

import 'agent_transport.dart';
import 'conversation_list.dart';
import 'open_conversation.dart';
import 'voice_mode_screen.dart';

class AssistantHome extends StatefulWidget {
  const AssistantHome({super.key});

  @override
  State<AssistantHome> createState() => _AssistantHomeState();
}

class _AssistantHomeState extends State<AssistantHome> {
  final _scaffold = GlobalKey<ScaffoldState>();
  AgentTransport? _transport;

  /// How many conversations were opened, so only the latest one is shown.
  var _opened = 0;

  @override
  void initState() {
    super.initState();
    _open(null);
  }

  /// Opens [session], or a new conversation, and closes the one before.
  Future<void> _open(Session? session) async {
    final opening = ++_opened;
    final previous = _transport;
    setState(() => _transport = null);
    // What the previous conversation recorded is kept, to carry on later.
    if (previous != null) unawaited(_close(previous));
    try {
      final transport = await openConversation(session);
      if (!mounted || opening != _opened) {
        await _close(transport);
        return;
      }
      setState(() => _transport = transport);
    } on Object {
      // Not opened: the user can pick it again.
    }
  }

  // The conversation that showed it stops following it as it is disposed.
  static Future<void> _close(AgentTransport transport) => transport.session.close();

  @override
  void dispose() {
    final transport = _transport;
    if (transport != null) unawaited(_close(transport));
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    final transport = _transport;
    return Scaffold(
      key: _scaffold,
      appBar: AppBar(
        title: const Text('Assistant'),
        leading: IconButton(
          tooltip: 'Conversations',
          icon: const Icon(Icons.menu),
          onPressed: () => _scaffold.currentState?.openDrawer(),
        ),
        actions: [
          IconButton(
            tooltip: 'Talk',
            icon: const Icon(Icons.graphic_eq),
            onPressed: transport == null
                ? null
                : () => Navigator.of(context).push(
                    MaterialPageRoute<void>(
                      fullscreenDialog: true,
                      builder: (context) => VoiceModeScreen(session: transport.session),
                    ),
                  ),
          ),
        ],
      ),
      drawer: Drawer(
        child: ConversationList(
          onSelect: (session) {
            _scaffold.currentState?.closeDrawer();
            _open(session);
          },
        ),
      ),
      body: transport == null
          ? const Center(child: CircularProgressIndicator())
          : SafeArea(
              child: AssistantConversation(key: ValueKey(transport.session.id), transport: transport),
            ),
    );
  }
}
```

The conversation's approver gets an `AIToolAuthorizer`, so when the agent first needs Gmail, the reply shows a **Connect Gmail** card that signs the user in. Add `AssistantConversation` to `lib/assistant_home.dart`. Like the quickstart's, its `actions` keep only rating:

```dart title="lib/assistant_home.dart"
class AssistantConversation extends StatefulWidget {
  const AssistantConversation({super.key, required this.transport});

  final AgentTransport transport;

  @override
  State<AssistantConversation> createState() => _AssistantConversationState();
}

class _AssistantConversationState extends State<AssistantConversation> {
  late final _approver = widget.transport.approver(
    authorizer: AIToolAuthorizer(
      connect: (call, url) => _signIn(url),
      decline: (call) async => widget.transport.session.declineLogin(call.id),
    ),
  );
  late final _conversation = AIConversation(
    transport: widget.transport,
    approver: _approver,
    actions: AIConversationActions.rate,
  );

  /// Opens a connector's sign-in, and completes with whether the user connected.
  static Future<bool> _signIn(Uri url) async {
    try {
      final back = await FlutterWebAuth2.authenticate(url: '$url', callbackUrlScheme: 'com.example.assistant');
      return Uri.tryParse(back)?.queryParameters['status'] == 'connected';
    } on PlatformException catch (error) {
      // Closed before signing in.
      if (error.code == 'CANCELED') return false;
      rethrow;
    }
  }

  @override
  void dispose() {
    _conversation.dispose();
    _approver.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) => Column(
    children: [
      Expanded(child: AIConversationView.conversation(conversation: _conversation)),
      ChatComposer.conversation(conversation: _conversation),
    ],
  );
}
```

`FlutterWebAuth2` opens the sign-in page in a web authentication sheet on iOS, and in a Custom Tab on Android, and resumes once the browser comes back to `com.example.assistant:/connected`, as described in [Tool sign-ins](https://getstream.io/chat/docs/sdk/flutter/ai-components/reasoning-and-tool-calls/#tool-sign-ins). iOS needs nothing registered. On Android, register the scheme on `flutter_web_auth_2`'s callback activity, inside the `<application>` of your `AndroidManifest.xml`:

```xml title="android/app/src/main/AndroidManifest.xml"
<activity
    android:name="com.linusu.flutter_web_auth_2.CallbackActivity"
    android:exported="true"
    android:taskAffinity="">
    <intent-filter android:label="flutter_web_auth_2">
        <action android:name="android.intent.action.VIEW" />
        <category android:name="android.intent.category.DEFAULT" />
        <category android:name="android.intent.category.BROWSABLE" />
        <data android:scheme="com.example.assistant" />
    </intent-filter>
</activity>
```

The reply shows each tool call as the agent makes it, and the question of `add_reminder` under its call.

## Send photos

The quickstart's `AgentTransport` sends only the text. To send the photos the user attaches, shrink each one and send it inline. Add `flutter_image_compress` to your `pubspec.yaml`:

```yaml title="pubspec.yaml"
dependencies:
  # Shrinks the photos the composer hands over before they are sent.
  flutter_image_compress: ^2.5.1
```

Then replace the `send` of `AgentTransport`, and add `import 'dart:convert';` and `import 'package:flutter_image_compress/flutter_image_compress.dart';` to its file:

```dart title="lib/agent_transport.dart"
@override
Future<void> send(AIConversationRequest<AgentMessage> request) async {
  await session.responses.create(request.text, images: await asImages(request.attachments));
}
```

`asImages` goes in the same file, after the class:

```dart title="lib/agent_transport.dart"
/// The composer's photos as the session takes them: JPEG data URIs, shrunk from the
/// full-size files the picker returns so that its three fit in one request.
Future<List<AgentImage>> asImages(List<XFile> photos) async => [
  for (final photo in photos)
    if (await FlutterImageCompress.compressWithFile(
      photo.path,
      minWidth: 768,
      minHeight: 768,
      quality: 70,
    ) case final jpeg?)
      AgentImage('data:image/jpeg;base64,${base64Encode(jpeg)}'),
];
```

Pick a model that reads images in `agent.yaml`. `responses.create` also takes a `reasoningEffort`, for an option such as "Think harder" in the composer.

## Talk instead of typing

The Talk button switches the open conversation to voice. The agent joins a call with everything said so far, and what's said on the call is written into the same conversation:

```dart title="lib/voice_mode_screen.dart"
import 'dart:async';
import 'dart:io' show Platform;

import 'package:flutter/material.dart';
import 'package:permission_handler/permission_handler.dart';
import 'package:stream_agents_core/stream_agents_core.dart';
import 'package:stream_agents_rtc/stream_agents_rtc.dart';

import 'agents.dart';
import 'voice_call_screen.dart';

class VoiceModeScreen extends StatefulWidget {
  const VoiceModeScreen({super.key, required this.session});

  /// The typed conversation to switch to voice.
  final AgentSession session;

  @override
  State<VoiceModeScreen> createState() => _VoiceModeScreenState();
}

class _VoiceModeScreenState extends State<VoiceModeScreen> {
  VoiceSession? _voice;
  late final Future<void> _starting;
  var _hungUp = false;

  @override
  void initState() {
    super.initState();
    _starting = _start();
  }

  Future<void> _start() async {
    // iOS asks for the microphone by itself as the call is joined.
    if (Platform.isAndroid && !await Permission.microphone.request().isGranted) {
      _close();
      return;
    }
    try {
      // The agent joins the call with everything said so far.
      await widget.session.startVoice();
      _voice = await VoiceSession.attach(agents, widget.session.id);
      if (mounted) setState(() {});
    } on Object {
      _close();
    }
  }

  /// Leaves the call and takes the agent off it, and the conversation goes on in writing.
  Future<void> _leave() async {
    if (_hungUp) return;
    _hungUp = true;
    // Leaving a call you attached to keeps the session.
    await _voice?.leave();
    try {
      await widget.session.stopVoice();
    } on Object {
      // The call already ended.
    }
  }

  void _close() {
    if (mounted) Navigator.of(context).pop();
  }

  @override
  void dispose() {
    unawaited(_starting.whenComplete(_leave));
    super.dispose();
  }

  @override
  Widget build(BuildContext context) => switch (_voice) {
    final voice? => VoiceCallScreen(
      voice: voice,
      // The chat's session holds the tools, so their questions show over the call.
      toolSession: widget.session,
      onHangUp: () async {
        await _leave();
        _close();
      },
    ),
    null => const Scaffold(body: Center(child: CircularProgressIndicator())),
  };
}
```

`VoiceCallScreen`, in `lib/voice_call_screen.dart`, is the call screen of the [Voice agent](https://getstream.io/chat/docs/sdk/flutter/ai-components/use-cases/voice-agent/) use case, which joins the call, shows why if joining fails, and leaves it when the user hangs up. Then `stopVoice()` takes the agent off the call, and the conversation goes on in writing. Add it, with `agent_glow.dart` and `approval_card.dart`, which shows the question of `add_reminder` over the call, and the microphone permission, to your app from there.

## Next steps

- [Voice agent](https://getstream.io/chat/docs/sdk/flutter/ai-components/use-cases/voice-agent/): the call screen in full.
- [Sessions](https://getstream.io/agents/docs/sessions/?client=flutter): listing, searching, forking and rewinding conversations.
- [Tool calling & Dispatch](https://getstream.io/agents/docs/tool-calling-and-dispatch/?client=flutter): tools, approvals, timeouts and tools that return images.
- [Generative UI](https://getstream.io/chat/docs/sdk/flutter/ai-components/generative-ui/): answers that are UI, such as a day plan with a form to change it.

---

For the most recent version of this documentation, visit [https://getstream.io/chat/docs/sdk/flutter/ai-components/use-cases/personal-assistant/](https://getstream.io/chat/docs/sdk/flutter/ai-components/use-cases/personal-assistant/).