Build On-Device Reply Suggestions with Apple's Foundation Models

TL;DR

This tutorial builds on-device reply suggestions for a SwiftUI chat app, pairing Apple's Foundation Models framework (iOS 26) with Stream Chat's messaging layer.
Generation runs entirely on-device using Apple's ~3B parameter model: no API cost, no network round-trip, and conversation context never leaves the device.
The integration surface is small -- one ViewFactory override, one actor-wrapped LanguageModelSession, and graceful degradation when Apple Intelligence isn't available.

Apple's Foundation Models framework, available in iOS 26, gives Swift developers direct access to an on-device ~3B parameter model -- the same one powering Apple Intelligence -- with no API key, no inference cost, and no data leaving the device.

It's designed for the small, targeted tasks that show up constantly across an app, like summarizing a thread, extracting a date from a message, or drafting a short reply.

That last one is what we'll build here: suggested replies in a SwiftUI chat app.

Three tappable chips sit above the composer, generated on-device from the recent conversation. Stream Chat handles the messaging layer, and the Foundation Models framework handles only the suggestion drafting.

It's a natural split: Stream for the infrastructure that has to be reliable and real-time, Apple's on-device model for the small inference that benefits from running locally.

Why On-Device Fits This Feature

Suggested replies are a good match for local inference: frequent, latency-sensitive, and scoped to content that shouldn't leave the device.

Running them on-device means:

No token cost. Suggestions are generated locally, so there's nothing to meter and no per-suggestion API charge.
Low latency. Generation runs on the device with no network round-trip before the chips appear.
Privacy. The conversation context fed to the model stays on the device.
Graceful degradation. If Apple Intelligence isn't available, the row of suggestions disappears, and chat keeps working.

The finished result is three tappable suggestions sitting directly above the composer, generated from the conversation above them.

Reply suggestions chips above the Stream Chat composer

Prerequisites

Xcode 26 and an iOS 26 simulator.
An Apple-Intelligence-capable Mac. The iOS Simulator uses the host Mac's model, so on-device generation works in the Simulator on eligible hardware.
A free Stream account (dashboard.getstream.io).
A deployment target set to iOS 26.

Building with Stream Skills

Instead of writing boilerplate Stream setup code, we use Stream's Agent Skills -- a set of markdown instruction files that give an AI coding agent the knowledge to scaffold the Stream integration for you.

A skill is a SKILL.md file that tells the agent how to do a task the way the tool's authors intend: which docs to read, which commands to run, and which mistakes to avoid. The agent loads one when the work matches, so the knowledge stays outside your codebase and carries over between projects.

Stream publishes an open-source set of these for building using Chat, Video, Feeds, and Moderation across web and mobile. You install the skills, and your coding agent (Claude Code, Codex, Cursor, etc.) pulls in whichever skill fits what you're doing.

You add it to your agent with one command:

shell

1
npx skills add GetStream/agent-skills

Here, we are using two to handle the Stream side of this app:

stream-cli reads credentials, mints a token, and seeds demo data.
stream-swift wires the Chat SwiftUI SDK following Stream's own patterns.

stream-swift is deliberately small. Rather than bundling a fixed copy of the SDK reference, it points the agent at the exact official iOS docs page, fetches the current version, and applies it in your project, along with Stream's setup flow and a set of iOS pitfalls. So the SwiftUI wiring follows Stream's live patterns: the client is created once at launch, and the composer is customized through a ViewFactory, rather than an API the model recalls from training.

stream-cli is Stream's command-line tool, a single Go binary that manages your Stream apps from the terminal. The setup flow uses it to read your API key and secret, mint a user token, and seed the demo channel and messages. That's what puts a real conversation on screen and gives the app a token to connect with, with no backend to write.

Standing Up Chat Using Stream Skills

A demo only looks convincing with real content on screen, so before writing any code, we'll create two users, a channel, and a short conversation.

Install the CLI and point it at your app:

shell

1
curl -sSL https://getstream.io/cli/install.sh | bash

The CLI reads your credentials from exported environment variables rather than picking up a .env file automatically, so export them for the session:

shell

1
2
export STREAM_API_KEY=<your_key>
export STREAM_API_SECRET=<your_secret>

These secrets are for server-side only. The API secret lives only in your shell or CLI, never in the app. If you add a .env file, you must add it to .gitignore if you store it there.

Mint a non-expiring demo token for the user we'll log in as:

shell

1
stream token alice

Now seed the data. You can do this with the CLI by calling the API endpoints directly:

shell

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Two users
stream api UpdateUsers --body '{"users":{
  "alice":{"id":"alice","name":"Alice"},
  "bob":{"id":"bob","name":"Bob"}
}}'

# One messaging channel with both members
stream api GetOrCreateChannel type=messaging id=weekend-hike \
  --body '{"data":{"name":"Weekend Hike","created_by_id":"alice","members":["alice","bob"]}}'

# A few back-and-forth messages
stream api SendMessage type=messaging id=weekend-hike \
  --body '{"message":{"text":"Hey! Are we still on for the hike this weekend?","user_id":"alice"}}'
stream api SendMessage type=messaging id=weekend-hike \
  --body '{"message":{"text":"Yeah! Saturday morning works for me.","user_id":"bob"}}'
# ...continue alternating alice/bob for ~8 messages

Realistic content matters here because the quality of the on-device suggestions depends on the conversation we feed the model.

From there, it adds the Stream SDK. For an Xcode app, that's Swift Package Manager pulling in StreamChatSwiftUI, which brings StreamChat and StreamCore with it, so the project gains a single package dependency. It keeps the API key and the demo token in one place, both of which are safe to bundle for a sample.

swift

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// StreamConfig.swift
import Foundation

enum StreamConfig {
    /// Stream app API key (safe to ship in the client).
    static let apiKey = "<your_api_key>"

    /// The demo user we connect as on launch (seeded via the Stream CLI).
    static let userId = "alice"
    static let userName = "Alice"

    /// Non-expiring demo token minted with `stream token alice`.
    /// Demo only; production should issue short-lived tokens from a backend.
    static let userToken = "<token_from_stream_token_alice>"
}

One note for production: don't bundle a static token in a real app. Issue short-lived tokens from your backend and feed them to the SDK via a TokenProvider closure (chatClient.connectUser(userInfo:tokenProvider:)), so tokens can expire and refresh without shipping a new build.

The last setup step creates the client. Stream's rule, and a SwiftUI best practice, is to create it once in an owned lifecycle entry point, never in a View body or a computed property. The skill wires it at launch and connects the user:

swift

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// StreamLocalAIApp.swift
import SwiftUI
import StreamChat
import StreamChatSwiftUI

@main
struct StreamLocalAIApp: App {
    // `@State` pins the StreamChat instance across SwiftUI's possible re-creations
    // of the App value type. The client is created once, here.
    @State private var streamChat: StreamChat

    init() {
        let config = ChatClientConfig(apiKey: .init(StreamConfig.apiKey))
        let chatClient = ChatClient(config: config)
        _streamChat = State(wrappedValue: StreamChat(chatClient: chatClient))

        connectDemoUser(using: chatClient)
    }

    var body: some Scene {
        WindowGroup { ContentView() }
    }

    private func connectDemoUser(using chatClient: ChatClient) {
        let userInfo = UserInfo(id: StreamConfig.userId, name: StreamConfig.userName)
        let token = Token(stringLiteral: StreamConfig.userToken)
        chatClient.connectUser(userInfo: userInfo, token: token) { error in
            if let error { print("Stream connect failed: \(error)") }
        }
    }
}

With setup done, the channel list and the message screen come straight from the SDK. The custom viewFactory propagates automatically to the channel screen that the list pushes to.

swift

1
2
3
4
5
6
7
8
9
10
11
12
13
14
// ContentView.swift
import SwiftUI
import StreamChat
import StreamChatSwiftUI

struct ContentView: View {
    var body: some View {
        // ChatChannelListView brings its own NavigationView, so never wrap it.
        ChatChannelListView(
            viewFactory: SuggestedRepliesViewFactory.shared,
            title: "Messages"
        )
    }
}

Run it, and you have a working chat: a channel list, and tapping a channel opens the standard message list with a composer. That covers the entire messaging layer, with Stream handling real-time sync, history, and delivery.

Building your own app? Get access to our Livestream or Video Calling API and launch in days!

Adding On-Device Suggested Replies

The suggestions are a horizontal row of tappable chips directly above the composer, generated on-device from the recent messages.

It comes together in four files: the model call, a driver that watches the channel and builds the prompt, a ViewFactory override that injects the row, and the chips UI itself.

The model call uses Foundation Models with structured output. @Generable tells the model to emit a value matching a Swift type, and @Guide steers each field, so there's no string parsing. The framework constrains the model's decoding to that type, so the call returns a populated ReplySuggestions value with no parsing step on your side.

We keep a single LanguageModelSession and wrap it in an actor to prevent requests from overlapping. A session handles one request at a time, and the actor enforces that without extra bookkeeping.

swift

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// ReplySuggestionGenerator.swift
import Foundation
import FoundationModels

/// Structured output for the on-device model.
@Generable
struct ReplySuggestions {
    @Guide(description: "Three short, casual replies the current user could send next, each under 8 words")
    let replies: [String]
}

/// Generates short reply suggestions fully on-device.
actor ReplySuggestionGenerator {
    private let session = LanguageModelSession(
        instructions: "You suggest short, natural chat replies. Return only the replies."
    )

    /// Whether on-device generation is usable right now (model downloaded,
    /// Apple Intelligence enabled, device eligible).
    static var isModelAvailable: Bool {
        SystemLanguageModel.default.availability == .available
    }

    func generate(prompt: String) async -> [String] {
        guard SystemLanguageModel.default.availability == .available else { return [] }
        do {
            let result = try await session.respond(to: prompt, generating: ReplySuggestions.self)
            return result.content.replies
        } catch {
            // Unavailable mid-flight, guardrail rejection, decode failure, etc.
            // Suggestions are best-effort, so fail silently.
            return []
        }
    }
}

SystemLanguageModel.default.availability reports not only whether the model can run but also why it can't: whether the device is ineligible, Apple Intelligence is off, or the model is still downloading. We treat all of those the same way and hide the row.

The driver keeps prompts small and avoids overlap. The on-device context window is small, so we send only the last ~8 messages rather than the full history. A few details matter here:

We observe the channel with our own ChatChannelController and its delegate, so we don't replace the delegate that the SDK's own channel view relies on.
We coalesce requests so that at most one generation runs at a time. A new request that arrives mid-flight runs as soon as the current one finishes, keeping the session busy during bursts without overlapping calls.
A signature of the recent messages skips regeneration when nothing has changed.

The prompt itself is plain: the recent messages as a labeled transcript, plus a line naming the user, so the model drafts from their side rather than the other person's.

swift

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
// SuggestedRepliesViewModel.swift
import Combine
import Foundation
import StreamChat

@MainActor
final class SuggestedRepliesViewModel: ObservableObject, ChatChannelControllerDelegate {
    @Published private(set) var replies: [String] = []
    @Published private(set) var isGenerating = false

    /// When false, the row is hidden entirely.
    let isModelAvailable = ReplySuggestionGenerator.isModelAvailable

    private let recentMessageLimit = 8
    private let controller: ChatChannelController?
    private let currentUserName: String
    private let generator = ReplySuggestionGenerator()

    private var generationInFlight = false
    private var pendingPrompt: (signature: String, prompt: String)?
    private var shownSignature: String?

    init(cid: ChannelId?, client: ChatClient) {
        currentUserName = client.currentUserController().currentUser?.name
            ?? client.currentUserId ?? "You"

        if let cid {
            let controller = client.channelController(for: cid)
            self.controller = controller
            if isModelAvailable {
                controller.delegate = self      // our own controller, safe to own the delegate
                controller.synchronize()
            }
        } else {
            controller = nil
        }
    }

    // Regenerate whenever the channel's messages change.
    nonisolated func channelController(
        _ channelController: ChatChannelController,
        didUpdateMessages changes: [ListChange<ChatMessage>]
    ) {
        Task { @MainActor in self.regenerate() }
    }

    func regenerate() {
        guard isModelAvailable, let controller else { replies = []; return }

        // `messages` is newest-first; take the newest N and flip to chronological.
        let recent = Array(controller.messages.prefix(recentMessageLimit).reversed())
        guard !recent.isEmpty else { replies = []; return }

        let signature = recent.map { "\($0.id):\($0.text)" }.joined(separator: "|")
        guard signature != shownSignature else { return }

        pendingPrompt = (signature, buildPrompt(from: recent))
        drainQueue()
    }

    /// Runs the pending prompt only if no generation is in flight, so the
    /// single session never runs two requests at once.
    private func drainQueue() {
        guard !generationInFlight, let job = pendingPrompt else { return }
        pendingPrompt = nil
        generationInFlight = true
        isGenerating = true

        Task {
            let result = await generator.generate(prompt: job.prompt)
            generationInFlight = false
            if !result.isEmpty {
                replies = Array(result.prefix(3))
                shownSignature = job.signature
            }
            if pendingPrompt != nil { drainQueue() } else { isGenerating = false }
        }
    }

    private func buildPrompt(from messages: [ChatMessage]) -> String {
        let transcript = messages
            .map { "\($0.author.name ?? $0.author.id): \($0.text)" }
            .joined(separator: "\n")

        return """
        Conversation so far:
        \(transcript)

        You are \(currentUserName). Suggest short, casual replies you could send next.
        """
    }
}

The injection point is a single ViewFactory slot. Stream's ViewFactory lets you swap a single piece of the UI while everything else stays default. We override makeMessageComposerViewType to stack our chip row on top of the SDK's composer.

The key to making tap-to-insert work is that our wrapper owns the MessageComposerViewModel and passes it to the SDK's MessageComposerView. Because MessageComposerViewModel.text is a public @Published property, tapping a chip sets composerViewModel.text, which inserts the text into the composer without sending it.

swift

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
// SuggestedRepliesViewFactory.swift
import SwiftUI
import StreamChat
import StreamChatSwiftUI

/// Overrides only the composer slot. Every other slot uses the SDK defaults.
class SuggestedRepliesViewFactory: ViewFactory {
    @Injected(\.chatClient) public var chatClient

    // Docked composer so the suggestions row sits cleanly on top of it.
    public var styles = RegularStyles()

    public static let shared = SuggestedRepliesViewFactory()
    private init() {}

    func makeMessageComposerViewType(
        options: MessageComposerViewTypeOptions
    ) -> some View {
        SuggestedRepliesComposer(
            factory: self,
            channelController: options.channelController,
            messageController: options.messageController,
            quotedMessage: options.quotedMessage,
            editedMessage: options.editedMessage,
            willSendMessage: options.willSendMessage
        )
    }
}

/// Stacks a suggestions row directly above the SDK's MessageComposerView.
struct SuggestedRepliesComposer<Factory: ViewFactory>: View {
    private let factory: Factory
    private let channelController: ChatChannelController
    private let messageController: ChatMessageController?
    @Binding private var quotedMessage: ChatMessage?
    @Binding private var editedMessage: ChatMessage?
    private let willSendMessage: () -> Void

    // Shared with the SDK composer below, so we can write into its text field.
    @StateObject private var composerViewModel: MessageComposerViewModel
    @StateObject private var suggestions: SuggestedRepliesViewModel

    init(
        factory: Factory,
        channelController: ChatChannelController,
        messageController: ChatMessageController?,
        quotedMessage: Binding<ChatMessage?>,
        editedMessage: Binding<ChatMessage?>,
        willSendMessage: @escaping () -> Void
    ) {
        self.factory = factory
        self.channelController = channelController
        self.messageController = messageController
        _quotedMessage = quotedMessage
        _editedMessage = editedMessage
        self.willSendMessage = willSendMessage

        let composerVM = ViewModelsFactory.makeMessageComposerViewModel(
            with: channelController,
            messageController: messageController,
            quotedMessage: quotedMessage,
            editedMessage: editedMessage,
            willSendMessage: willSendMessage
        )
        _composerViewModel = StateObject(wrappedValue: composerVM)
        _suggestions = StateObject(
            wrappedValue: SuggestedRepliesViewModel(
                cid: channelController.cid,
                client: channelController.client
            )
        )
    }

    var body: some View {
        VStack(spacing: 0) {
            SuggestedRepliesBar(viewModel: suggestions) { reply in
                composerViewModel.text = reply   // insert, do NOT send
            }

            MessageComposerView(
                viewFactory: factory,
                viewModel: composerViewModel,
                channelController: channelController,
                messageController: messageController,
                quotedMessage: $quotedMessage,
                editedMessage: $editedMessage,
                willSendMessage: willSendMessage
            )
        }
        .onAppear { suggestions.regenerate() }
    }
}

The chips UI is the last piece, and the row handles three states:

Model unavailable: render nothing (EmptyView), with no error UI.
Generating the first batch: a small loading indicator.
Suggestions ready: scrollable capsule chips.

swift

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
// SuggestedRepliesBar.swift
import SwiftUI

struct SuggestedRepliesBar: View {
    @ObservedObject var viewModel: SuggestedRepliesViewModel
    let onSelect: (String) -> Void

    var body: some View {
        if !viewModel.isModelAvailable {
            EmptyView()   // model unavailable -> no row, no error UI
        } else if viewModel.isGenerating && viewModel.replies.isEmpty {
            HStack(spacing: 8) {
                ProgressView().controlSize(.small)
                Text("Suggesting replies...")
                    .font(.footnote).foregroundStyle(.secondary)
                Spacer()
            }
            .padding(.horizontal, 12).padding(.vertical, 8)
        } else if !viewModel.replies.isEmpty {
            ScrollView(.horizontal, showsIndicators: false) {
                HStack(spacing: 8) {
                    ForEach(viewModel.replies, id: \.self) { reply in
                        Button { onSelect(reply) } label: {
                            Text(reply)
                                .font(.subheadline).lineLimit(1)
                                .padding(.horizontal, 14).padding(.vertical, 8)
                                .background(Color.accentColor.opacity(0.12), in: Capsule())
                                .foregroundStyle(Color.accentColor)
                        }
                        .buttonStyle(.plain)
                    }
                }
                .padding(.horizontal, 12).padding(.vertical, 8)
            }
        } else {
            EmptyView()
        }
    }
}

Running the App

Build and run on an iOS 26 simulator. Open the seeded Weekend Hike channel, and you'll see:

The back-and-forth messages render through Stream Chat.
A moment later, two or three suggestion chips appear above the composer, generated on-device from the conversation.
Tapping a chip drops its text into the composer, ready to be edited or sent.
Send a message, and the suggestions regenerate for the new context.

Here it is in action:

On a device or Mac without Apple Intelligence enabled, the row does not appear, and chat works exactly as before.

What We Built, and Why the Split Works

Stream Chat owns messaging: real-time transport, history, sync, and a full message UI, none of which we want an LLM near.

The on-device model enhances this Stream Chat. Reply suggestions are frequent and latency-sensitive, which is where local inference helps most. It runs without a token cost, without a network round trip, and without sending conversation context off the device.

The integration surface stays small: one ViewFactory override, one shared composer view model, and one actor around a single LanguageModelSession.

The wider point is that on-device AI can run alongside a cloud chat stack rather than replace it. The good candidates are the small, high-frequency inferences where running locally is cheaper and faster, while the heavier work stays in the cloud.

If we wanted to extend this, we could add:

Smart actions beyond replies: summarize a long thread, or suggest a calendar time when a date is mentioned, all on-device.
Tone control: add a @Guide field for tone (casual or formal) and a toggle.
Streaming: surface partial generations so the chips appear faster.
Hybrid fallback: drop to a cloud model only when the on-device model is unavailable, so the feature continues to work rather than disappearing.

If you want to build this, the setup is short. Spin up a free Stream app, add the skills with npx skills add GetStream/agent-skills, and let your agent handle the chat while you write the on-device suggestions. Once you've wired it once, the same split works for any local inference you want to add next.

Build On-Device Reply Suggestions with Apple's Foundation Models & Stream Chat

Why On-Device Fits This Feature

Prerequisites

Building with Stream Skills

Standing Up Chat Using Stream Skills

Adding On-Device Suggested Replies

Running the App

What We Built, and Why the Split Works