Build a Vision AI Agent with Gemini 3 in < 3 Minutes
The Gemini 3 models are supported in Vision Agents, the open-source Python framework for building voice and video AI apps in Python. Let's use gemini-3-pro-preview as LLM, install Vision Agents + the Gemini plugin to build a live video-call agent that can see and describe anything on your screen.
Create a Fresh uv-Based Python Project
# Initialize a new Python project
uv init
# Activate your environment
uv venv && source .venv/bin/activateAdd Vision Agents and its associated plugins to the project for the context of the demo in this tutorial.
# Install Vision Agents
uv add vision-agents
# Install required plugins
uv add "vision-agents[getstream, gemini, elevenlabs, deepgram, smart-turn]"You'll also need:
- A free Gemini API key.
- A free Stream API key and secret.
- ElevenLabs API key to access a voice model for generating speech.
- Deepgram API key to access a transcription model for automatic speech recognition.
- Pipecat's open-source project to detect turns.
Sample Source Code
Rename your uv project's main.py to gemini_vision_demo.py and replace its content with this sample code.
import asyncio
import logging
from dotenv import load_dotenv
from vision_agents.core import User, Agent, cli
from vision_agents.core.agents import AgentLauncher
from vision_agents.plugins import elevenlabs, getstream, smart_turn, gemini, deepgram
logger = logging.getLogger(__name__)
load_dotenv()
async def create_agent(**kwargs) -> Agent:
"""Create the agent with Inworld AI TTS."""
agent = Agent(
edge=getstream.Edge(),
agent_user=User(name="Friendly AI", id="agent"),
instructions="You are a friendly AI assistant powered by Gemini 3. You are able to answer questions and help with tasks. You carefully observe a users' camera feed and respond to their questions and tasks.",
tts=elevenlabs.TTS(),
stt=deepgram.STT(),
# Gemini 3 model
llm=gemini.LLM("gemini-3-pro-preview"),
turn_detection=smart_turn.TurnDetection(),
)
return agent
async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None:
"""Join the call and start the agent."""
# Ensure the agent user is created
await agent.create_user()
# Create a call
call = await agent.create_call(call_type, call_id)
logger.info("🤖 Starting Inworld AI Agent...")
# Have the agent join the call/room
with await agent.join(call):
logger.info("Joining call")
logger.info("LLM ready")
await asyncio.sleep(5)
await agent.llm.simple_response(text="Describe what you currently see")
await agent.finish() # Run till the call ends
if __name__ == "__main__":
cli(AgentLauncher(create_agent=create_agent, join_call=join_call))This script is all you need to build a vision assistant using Gemini 3, Deepgram, and ElevenLabs.