Build a Local On-Device AI Agent for Mac Using Qwen 3.5 Small

Qwen 3.5 Small is a family of lightweight models (0.8B, 2B, 4B, and 9B parameters) on Ollama.

Developers can use these models for projects requiring multimodal input, tool calling, and on-device reasoning.

In this demo, the agent runs locally/offline using the Qwen 3.5:2b model to describe what it sees in the user's live camera feed:

Here's how to build the same locally-running vision + voice agent in Python using Qwen 3.5 Small and VisionAgents in less than five minutes.

The Agentic Pipeline

Create a custom voice and vision pipeline for the agentic workflow.

LLM: Qwen 3.5 Small (via Ollama)
TTS: ElevenLabs
STT: Deepgram
Turn Detection: Smart-Turn
Transport: Stream WebRTC
Python Framework: Vision Agents (open-source)

Requirements

Install Ollama and download any of the Qwen-3.5 models to run locally.
API keys for: Stream, Deepgram, ElevenLabs

Step 1: Scaffold Your Vision Agents Project

uvx vision-agents init my-agent && cd my-agent
uv add "vision-agents[getstream, deepgram, elevenlabs, smart-turn]"

Step 2: Add Your API Credentials

cp .env.example .env

# .env
STREAM_API_KEY=...
STREAM_API_SECRET=...
DEEPGRAM_API_KEY=...
ELEVENLABS_API_KEY=...

Step 3: Create and Run the Agent

import logging
from pathlib import Path

from dotenv import load_dotenv

from vision_agents.core import Agent, AgentLauncher, User, Runner
from vision_agents.plugins import getstream, deepgram, elevenlabs, smart_turn
from vision_agents.plugins.ollama import VLM as OllamaVLM

logger = logging.getLogger(__name__)

def create_agent(**kwargs) -> Agent:
    """Create a video analysis agent using Ollama with qwen3.5:9b."""
    agent = Agent(
        edge=getstream.Edge(),
        agent_user=User(name="Video Analyst", id="agent"),
        instructions=(
            "You are a video analysis assistant. Analyze the video feed and "
            "answer questions about what you see. Be detailed and descriptive "
            "in your observations."
        ),
        llm=OllamaVLM(
            model="qwen3.5:9b",
            fps=1,
            frame_buffer_seconds=10,
        ),
        stt=deepgram.STT(),
        tts=elevenlabs.TTS(),
        turn_detection=smart_turn.TurnDetection(),
    )
    agent._audio_buffer_limit_ms = 90_000
    return agent

async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None:
    """Join a call and start video analysis."""
    try:
        await agent.create_user()
        call = await agent.create_call(call_type, call_id)
        async with agent.join(call):
            await agent.simple_response("Tell the user a story about the video.")
            await agent.finish()
    except Exception as e:  # noqa: BLE001
        from getstream.video.rtc.connection_utils import SfuConnectionError

        if isinstance(e, SfuConnectionError):
            cause = e.__cause__ or e
            logger.error(
                "GetStream SFU connection failed: %s. Check STREAM_API_KEY and "
                "STREAM_API_SECRET in .env, and that your network allows WebRTC.",
                cause,
            )
        raise

if __name__ == "__main__":
    Runner(AgentLauncher(create_agent=create_agent, join_call=join_call)).cli()

Running the above Python script will launch the agent in your browser for realtime voice and vision interactions.

Why Build a Local AI With Qwen-3.5 Small?

The series of models give reliable output when you run them on edge devices such as mobile and desktop. You get full privacy, no API bills, and the ability to use the models in airplane mode without internet.