Until WWDC 26, experimenting with on-device AI models for Swift/SwiftUI projects has always involved using Apple's Foundation Models and Core ML. WWDC 26 introduced the Apple Core AI framework, enabling developers to integrate third-party AI models into their apps.
This guide will assist you in bringing small/on-device models to Core AI, using them in a SwiftUI project, and then downloading and running them directly on iOS in Stream Chat.
Requirements
Core AI is new and in beta when writing this article. To use it in a project, it requires macOS Golden Gate and the following:
- iOS and macOS 27 (Mac computers with Apple Silicon): Apple Intelligence must be enabled for Foundation Models.
- Xcode 27: At the time of writing this post, it's in beta.
- A physical iOS device (iPhone or iPad) to install and run Core AI models.
- A Hugging Face or GitHub-hosted Core AI model.
Note: Core AI is not available on the iOS simulator.
To quickly create a chat experience to run Core AI models, you need:
- Stream CLI Skills
- Stream Swift Skills
What You Can Build & How It Works
The demos in this article showcase a fully functional iOS/SwiftUI chat app built on Stream's Chat SwiftUI SDK, with integrated AI features running on-device via Apple Intelligence, Foundation Models, the SpeechAnalyzer API, and Core AI (.aimodel) exports.
Upon launching the app on an iOS device, it boots straight into a real-time chat messaging experience. Here is a highlight of the various parts:
- Real-Time Chat: A chat channel list and message screens powered by
StreamChatSwiftUI(v5.5.1), connected to an app on Stream's dashboard with seeded demo channels and users. - Voice Input: To allow users to dictate messages using speech. The default engine uses Apple's SpeechAnalyzer API and its SpeechTranscriber model. Alternatively, users can use the Whisper large-v3-turbo Core AI export. It records a clip and transcribes it on-device.
- AI Text Refinement Toolbar: Refine, Summarize, Grammar, and a Style menu (friendly, professional, concise, poetic) to rewrite draft messages in the composer.
- Photo Q&A in Chat (vision language model): To allow users to attach images from their Photo Library with the composer's standard media picker, ask a question about it, and send. The photo uploads via the iOS Chat SDK's media upload path and uses MiniCPM-V 4.6 to process and send responses into an incoming chat bubble.
- Model settings: The trailing toolbar on the messages screen opens an AI Models sheet for instant picking of desired models.
The source code is available on GitHub.
To run the demo, copy the Swift files into your project, install the Core AI package, and add all the models used in this project.
What is Core AI?
Previously, bringing intelligence to an app on Apple platforms relied on Apple Intelligence, Foundation models, and Core ML to run models on-device. During WWDC 26, Apple released the Core AI framework to make the process more open and flexible.
Core AI is a bring-your-own machine learning framework that allows developers to use any model that can run on-device in their projects.
Core AI Models and Role
Before you can use open-source and Hugging Face models in Core AI, they must be converted to the Apple format, .aimodel. The following highlights GitHub repos where you can find such models for your projects:
The above resources offer a range of Core AI models for various use cases. However, if there is a specific Hugging Face model you want to use in a project but have not found in any of the GitHub repos, you can convert it to a .aimodel format if it is PyTorch-based.
Once you convert a PyTorch-based model to a Core AI-compatible one, the operating system decides whether to run it on the CPU, GPU, or Apple Neural Engine (ANE), as shown above. The operating system selects the best option to achieve optimal performance.
For the context of the sample app in this article, the following Core AI models are required.
| Model Name | Description | Role |
|---|---|---|
| SpeechTranscriber (SpeechAnalyzer) | Apple's built-in on-device speech model behind the SpeechAnalyzer API | Converts voice to draft messages (default) |
| Whisper large-v3-turbo | OpenAI's 809M-parameter ASR model as a Core AI export | Speech-to-text (alternative) |
| Apple Intelligence / Foundation Models | Apple's on-device system language model driven through LanguageModelSession | Text refinement & styling (default) |
| Qwen3.5 0.8B | For faster and reliable responses | Text refinement & styling (alternative) |
| Qwen3.5 2B | For higher-quality rewriting | Text refinement & styling (alternative) |
| LFM2.5 1.2B | Liquid AI instruct model for snappy non-thinking answers | Text refinement & styling (alternative) |
| Granite 4.0-H 1B | IBM's Mamba2 + attention hybrid | Text refinement & styling (alternative) |
| MiniCPM-V 4.6 | OpenBMB's 1.3B vision-language model | Photo Q&A / vision (default) |
Build a UI To Interact With Core AI Models
To interact with Core AI models in any project, developers can use SwiftUI to quickly assemble UIs. In the context of our project, the Core AI functionalities are all in-app features of a chat messaging app. Therefore, we do not need custom UIs. Instead, we should modify some of the reusable components of the Stream Chat iOS SDK to enable seamless integration with Core AI models.
Having already built chat messaging UI and components, you can easily insert in-app AI features in no time.
Start With Stream Skills and an iOS/SwiftUI Messaging App
To create an iOS chat interface for AI integrations with Stream, you can use either of the following options:
- Step-by-step tutorial: This option is excellent for discovering the basics of Stream Chat SDKs and connecting your app.
- Stream Agent Skills: To quickly scaffold an app with functional chat messaging features such as offline support, chat channels, threads, replies, reactions, media attachment, and more.
The second option above offers a faster way for teams to bring ideas to life using Stream SDKs. You can use these three steps to create a fully working SwiftUI chat messaging app powered by the Stream's iOS Chat SDK and add AI features in the next sections.
Step 1: Install Stream CLI and Swift Skills
The Stream Skills are available on skills.sh and can be found by searching for 'getstream'. You can install the CLI and Swift Skills with the following commands:
12npx skills add https://github.com/getstream/agent-skills --skill stream-cli npx skills add https://github.com/getstream/agent-skills --skill stream-swift
The CLI skill is used for authentication and for generating demo users in the messaging app. The Swift skill helps your coding agent to add SwiftUI chat as an in-app feature.
Step 2: Generate a Full-Featured iOS Chat App With Your Favorite Coding Agent
With your favorite agentic coding tool, you can open a blank SwiftUI app (starter Xcode project) and reference the two skills to create a chat messaging application (which can also be an in-app feature) with Stream's SwiftUI SDK. The agent will use the installed skill to set up a Stream Chat backend, create and authenticate users, install all the necessary dependencies, and configure permissions in your Xcode project to make the app ready for launch.
Step 3: Integrate Your AI Features
Your Stream Chat for SwiftUI app does not include Core AI integration to add custom models. Similar to how you use the Stream Skills to create the messaging app, you can use your preferred coding agent, such as Fable 5 or GPT-5.6, to implement intelligence features to iOS apps with Core AI.
The following sections will get you up and running with three integrated AI features in the iOS chat app.
Integrate a Custom AI-Powered Message Dictation Into Stream Chat
One great feature of a messaging app is the ability to write messages with your voice. Although Stream Chat supports voice messaging (speech-to-audio), the message composer does not have built-in dictation (automatic speech recognition). We can implement this feature with Core AI using custom on-device speech-to-text (STT) models.
With your Stream Chat project already built with the skills, you can now prompt your coding agent (with specified URLs and references) to implement Core AI and on-device models in the messaging app by identifying and modifying a specific chat SDK component to support AI.
For simplicity, you can ask your coding agent to use the Apple SpeechAnalyzer API and its SpeechTranscriber model for this purpose.
| Model Name | Description | Role |
|---|---|---|
| SpeechTranscriber (SpeechAnalyzer) | Apple's built-in on-device speech model behind the SpeechAnalyzer API | Converts voice to draft messages (default) |
Check out the official Core AI Models and Core AI Model Zoo repos for other speech-to-text models you can use in your project for transcription.
Note: Once you instruct a coding agent to integrate Core AI and specific models into the messaging app, it will configure the necessary permissions in Xcode and install all Swift packages required for Core AI to function correctly.
Add a Model That Can See Media Attachments and Respond to Questions
Wouldn't it be fun if you could upload a document (PDF) or photo to a messaging app like WhatsApp and use an in-app AI to get insights? Lucky for developers, Core AI makes your app's users enjoy a feature like this in Stream Chat.
Begin by finding a vision model on Hugging Face and convert it to a Core AI format (.aimodel) or grab an already-made one from the official Core AI Models and Core AI Model Zoo repos. After testing several models for downstream vision tasks, I recommend experimenting with the MiniCPM-V 4.6 vision-language model.
Next, ask your coding agent to implement the MiniCPM-V 4.6 Core AI model in the same iOS chat messaging codebase. This example used an agent to specifically integrate MiniCPM-V 4.6 into the Stream Chat message composer, allowing users to query attached photos and documents.
| Name | Description | Role |
|---|---|---|
| MiniCPM-V 4.6 | OpenBMB's 1.3B vision-language model | Photo Q&A / vision (default) |
Add AI-Powered Message Refinement and Styling Models
Using Core AI and Foundation Models, we can build an in-app AI feature that improves and styles user-written messages in the Stream Chat text composer with local on-device models before sending them in chat. This is useful in many use cases, such as message summarization and making messages feel personal, professional, casual, and more.
You can use any general-purpose Core AI model from the GitHub repos mentioned in the previous sections to provide this support with a coding agent.
The demo in this section overlays the AI refinement button on top of the Stream Chat message composer.
When users draft messages in the chat composer, they can refine, summarize, or fix grammar or style to convey a certain look and feel.
Experiment Further With AI Models in Stream Chat
Stream Chat offers many options for integrating in-app AI features into any app. You now know how to use the Apple Core AI framework and its models (.aimodel) with the Stream Chat SwiftUI SDK. Core AI helps developers run models locally in SwiftUI on iPhone and Mac, but is not available in other Stream mobile SDKs.
To create similar AI chat experiences with other Stream SDK platforms like Android, Flutter, and React Native, you can start with the Stream Agent Skills and your preferred SDK.
Finally, you can explore all the Swift files used for the demos in this article on GitHub.