Build Your App With Reusable SwiftUI AI Chat Components

With coding agents getting so good in recent times, it's now easy to integrate AI features into an app. However, there are still various aspects that need to be carefully considered and handled correctly to improve AI performance and response generation.

Stream's SwiftUI AI components library provides the pieces for that interface, including a streaming message view, typing indicator, prompt composer, and speech-to-text button. They work with Stream Chat.

The preview above demonstrates the various AI modules that you'll build in this tutorial.

SwiftUI AI Chat UIs: Overview & Features

The SwiftUI AI components are designed specifically to help developers easily integrate AI features into their applications with curated, ready-to-use UI elements, animations/interactions that can be scaled to millions of users.

You can use the components with our real-time Chat API, making it quicker to integrate and render responses from LLM providers such as OpenAI, Google, and Anthropic, or any custom backend, with out-of-the-box Markdown, code blocks, tables, thinking indicators, image support, and more.

The demo app for this article has features outlined in the table below.

Architecture, Dependencies, and Frameworks

The sample Xcode project doesn't communicate with an LLM directly. It sends Stream messages and the backend joins channels as a bot user, reads new messages, and calls an LLM.

Responses are streamed back via Stream as messages that are updated in place. It uses the Apple Speech/AVFoundation framework for voice input and camera. The project's Info.plist requires usage descriptions for the iOS device's camera, microphone, photo library, and speech recognition.

All these configurations can be handled automatically by your coding agent with its computer-use feature.

The demo project depends on the following packages, which can be quickly and automatically installed by your coding agent via Swift Package Manager.

What You'll Build

Let's look at an overview and usage of the sample app's major components.

Response Streaming Component

AI response streaming is handled by the library's StreamingMessageView. It renders Markdown content and highlights code syntax with support for all major coding languages. It can render standard Markdown, such as tables and images. Under the hood, it implements typewriting animation, with a character queue, similar to ChatGPT.

This code snippet shows how you can use it for in-app content generation and streaming:

StreamingMessageView(
    content: content,
    isGenerating: true
)

It also accepts an optional animation speed with the letterInterval parameter. The default value is 0.005 (5ms).

AI Typing Animation and Indicator Component

The AITypingIndicatorView helps present different LLM states, such as "Thinking" and "Checking External Sources". It can be customized with different text to represent a specific AI thinking state, such as Architecting, Baking, Beaming, Channeling, Catapulting, Dilly-Dallying, Smooshing, etc.

AITypingIndicatorView(text: "Swooping")

Prompt Writing Component

The ComposerView provides a text-entry UI with attachment previews, suggestion chips, and a button to send a prompt. It can be implemented with ComposerViewModel to handle states. You can also pass a closure to receive every MessageData payload when a prompt is sent.

@available(iOS 16, *)
ComposerView(
    viewModel: ComposerViewModel(),
    colors: colors
) { message in
    print(message.text, message.attachments)
}

Speech Recognition Component

The AI library uses its SpeechToTextButton component to transcribe voice input with Apple's Speech framework. When the button is tapped, it requests microphone access, records audio, and forwards the recognized transcript to the speech framework for processing.

SpeechToTextButton(
    locale: Locale(identifier: "en-US"),
    colors: colors
) { transcript in
    print("User said:", transcript)
}

Installation and Setup

To quickly test the SwiftUI AI components and experience how they fit into your app, you can clone and run the sample app in the official repo by configuring its companion Node.js server. Alternatively, you can scaffold a chat messaging app using Stream agent skills and integrate the AI components.

Use Stream CLI and Swift Skills

Stream has agent skills for adding in-app activity feeds, chat messaging, moderation, and audio/video features to products and services. Visit skills.sh, pick and install Stream skills for Android, Flutter, iOS, React, and React Native.

For the SwiftUI demo, you should install the Stream CLI and Swift skills.

Next, use your favorite coding agent on a platform like Cursor to create an app that integrates the AI library with Stream Chat iOS, using the CLI and Swift skills. For an agent to scaffold the Stream-powered SwiftUI app, you must possess a Stream credential and have at least one app created on your dashboard. You can sign up for a free Stream account if you do not have one yet.

Scaffolding the app with an agent will create a project structure similar to this (below). You do not have to install any Stream Chat SDK and AI components dependencies. The agent automatically handles everything.

StreamChatSwiftAI/
├── StreamChatSwiftAIApp.swift    App entry: ChatClient, StreamChat(appearance, utils), connectUser
├── StreamConfig.swift            API key, user, token, agent host (simulator vs device), LLM platform
├── Info.plist                    Privacy descriptions/definitions for microphone and camera
├── Theme/
│   ├── AuroraPalette.swift       Color scheme
│   ├── AuroraBackground.swift    Animated gradient field
│   ├── GlassStyle.swift          .auroraGlass / .auroraGlassCapsule / .auroraGlassCircle, GlassIconButton
│   └── StreamAppearance.swift    Stream Appearance + Utils (palette mapping, AIMessageTypeResolver)
├── AI/
│   ├── TypingIndicatorHandler.swift  Listens to AI indicator + client‑tool events, tracks bot presence
│   ├── AIAgentOverlayView.swift      Floating "Thinking…" glass pill
│   └── AuroraViewFactory.swift       ViewFactory + Styles overrides (streaming AI message, glass bubbles, floating composer)
├── Agent/
│   ├── AgentService.swift            HTTP client for the Node backend
│   ├── ClientTools.swift             Event payload + GreetClientTool, AppearanceClientTool
│   └── ClientToolActionHandler.swift Executes tool actions (alerts, colour scheme)
└── Views/
    ├── RootView.swift                Connection gate (connecting / connected / failed)
    ├── AssistantHomeView.swift       Shell: SidebarView (iPhone) or NavigationSplitView (iPad), toolbar
    ├── AssistantViewModel.swift      Channel + agent lifecycle, sending, chat options, toasts
    ├── WelcomeHeroView.swift         Empty state: orb, headline, SuggestionsView
    ├── ConversationView.swift        Stream MessageListView with AuroraViewFactory
    ├── ConversationListView.swift    Sidebar list backed by ChatChannelListViewModel
    ├── AuroraComposer.swift          StreamChatAI.ComposerView in glass; input is re-identified after each send
    └── AgentStatusBadge.swift        Toolbar agent state badge

Configure Your Node Server

The app runs on a Node.js server from Stream's chat-ai-samples repo. The server should be started before running the app.

It can be configured by cloning and running the commands below, following the step-by-step instructions:

# Server
git clone https://github.com/GetStream/chat-ai-samples

# Install dependencies
cd chat-ai-samples/ai-sdk-sample

npm install

# In your .env
cp .env.example .env

# Required
STREAM_API_KEY=your_api_key
STREAM_API_SECRET=your_api_key
# Bring your own API key
ANTHROPIC_API_KEY=your_api_key

# Optional
OPENAI_API_KEY=your_api_key
OPENWEATHER_API_KEY=your_api_key
GOOGLE_GENERATIVE_AI_API_KEY=your_api_key
GEMINI_API_KEY=your_api_key
XAI_API_KEY=your_api_key

From the above, you can obtain STREAM_API_KEY=your_api_key and STREAM_API_SECRET=your_api_key from your dashboard. It also requires an API credential from your chosen AI provider. The other API keys are optional.

Run With an iOS Simulator/Localhost

Running npm start will start the server and make it available at http://localhost:3000/. With the server running successfully, you can open the Xcode project and run the app. You should now be able to start the AI assistant, ask about general questions, and also render Markdown, math expressions, and code, as shown below.

The app can also be run from the Terminal by specifying the Xcode project's name and the iOS simulator's destination.

xcodebuild build -project StreamChatSwiftAI.xcodeproj -scheme StreamChatSwiftAI \
 -destination 'platform=iOS Simulator,name=iPhone 17 Pro'

Run With an iPhone or iPad

Aside from using the iOS simulator to preview the app, you can test it on an iPhone or iPad by ensuring the iPhone or iPad and the Mac (on which the server is located) are on the same network. For this to be possible, you should open the project's Info.plist and set NSAppTransportSecurity → NSAllowsLocalNetworking. Launching the app on iOS for the first time will require local-network permission on the first connection.

Troubleshoot and Fix Errors

Connecting the backend server with the app and launching it may encounter some issues. Here are the common ones and how to fix them. You can use your favorite coding agent to fix any of these known issues.

• Agent Offline or Couldn't Connect on Launch: Check your Stream apiKey / userToken. Also, try regenerating a new token with getstream token your_token_name
• Agent Not Answering: When the agent joins but never answers, it can be due to a missing/invalid LLM API key in the server's .env. Ensure you have added the correct AI provider's API key in the project's .env.
• Incomplete .env Credentials: Using the SwiftUI AI components in your app requires STREAM_API_KEY, STREAM_API_SECRET, and at least one API credential from any model provider. Failing to provide any of the above may display an error about the server's .env being incomplete.

Customizing the AI Components

The SwiftUI AI chat components we used in this article can be modified, themed, or swapped entirely with custom implementations. Let's look at how you can customize and replace some of these components.

Customize the Message/Prompt Composer

To customize the ComposerView, you should pass the parameter viewFactory to any type that conforms to ComposerViewFactory. The underlying protocol exposes the following independent slots you can override individually.

How To Swap a Single View Slot

To replace a single view slot near the prompt composer, you should create a class that conforms to ComposerViewFactory. This class allows you to override the specific method you need. The methods that are not overridden automatically retain their default implementations.

class MyComposerFactory: ComposerViewFactory {
    // Replace the leading button with a paperclip icon.
    func makeLeadingComposerView(options: LeadingComposerViewOptions) -> some View {
        Button {
            options.onTap()
        } label: {
            Image(systemName: "paperclip")
                .padding(10)
                .background(.ultraThinMaterial, in: Circle())
        }
    }
}

Swap the Prompt Composer Input

To replace the composer view (input + send button) entirely, you should override the makeComposerInputView(options:) of the library to take full control of the text field and send button. The ComposerInputViewOptions struct provides access to the view model, color palette, generating state, and send/stop callbacks.

class MyComposerFactory: ComposerViewFactory {
    func makeComposerInputView(options: ComposerInputViewOptions) -> some View {
        MyCustomInputView(
            viewModel: options.viewModel,
            isGenerating: options.isGenerating,
            onSend: options.onMessageSend,
            onStop: options.onStopGenerating
        )
    }
}

Insert Into the Trailing Slot

There is nothing in the trailing section of the composer by default. You can insert views in this section using makeTrailingComposerView(options:).

class MyComposerFactory: ComposerViewFactory {
    func makeTrailingComposerView(options: TrailingComposerViewOptions) -> some View {
        Button {
            toggleMode()
        } label: {
            Image(systemName: "wand.and.sparkles")
        }
    }
}

Modify Colors

The library's Colors class can be used to create a palette for the AI components. To use this class, you should create a single instance and inject it into the views you want to render to keep them in sync.

let colors = Colors(
    composer: .init(
        attachmentButtonIcon: .pink,
        selectedOptionForeground: .purple
    ),
    suggestions: .init(background: .mint.opacity(0.3)),
    transcription: .init(icon: .orange)
)

ComposerView(colors: colors) { message in
    // Handle message
}

SuggestionsView(
    suggestions: ["What are the docs for the AI SDK?"],
    colors: colors,
    onMessageSend: handleSuggestion
)

SpeechToTextButton(colors: colors) { transcript in
    print(transcript)
}

Limitations

To run your project successfully, using the library, the backend server is required for AI generation.

Without the server, the app is still a working Stream chat client. However, AI replies cannot be generated.

Currently, the library consists of only these components StreamingMessageView, ComposerView, SpeechToTextButton, and AITypingIndicatorView. We will iterate and add more components over time. If there's a specific component that should be included in the library, please add a GitHub issue.

Build Your App With AI Components on Other Stream SDK Platforms

In this article, we covered how to use the Stream's SwiftUI AI Chat components and different ways to add intelligence features to your app.

Although there are fewer reusable AI elements in the library, they help build your app's features faster without having to implement your own. The article highlighted a demo showcasing all components of the AI library, its setup, and usage code snippets in iOS/Swift.

To build similar AI integration experiences using a ready-built library of LLM response-generation UIs and backend, refer to the following and use our various SDK platforms.

• Android AI UI Components and Chat SDK
• Flutter AI Components and Chat SDK
• React AI Components and Chat SDK
• React Native AI Components and Chat SDK