Kimi K2.5: Build a Video & Vision Agent in Python
The Kimi K2.5 vision model can see the environment around you and asnwers questions about what it sees. This capability is similar to the Gemini Live Android or iOS app. Let's implement the model in Vision Agents for integrated video, vision, and voice experience.
Overview of the Demo
- A real-time voice agent that analyzes your live camera feed, answers questions about what it sees, and helps with vision tasks.
- Leverages Kimi K2.5's native multimodal understanding for accurate visual descriptions and reasoning.
- Low-latency conversations with seamless turn-taking using Smart Turn.
- A swappable voice pipeline using OpenAI-compatible API access to Kimi K2.5 via Vision Agents.
Voice Architecture
The demo uses the peech-to-text -> LLM -> text-to-speech pipeline approach, where each component is replaceable.
- Vision Agents: A platform for building voice apps in Python.
- Language Model: Kimi K2.5 (via OpenAI-compatible API).
- Text-to-Speech: ElevenLabs.
- Speech-to-Text: Deepgram.
- Smart-Turn: A free and open-source turn detection project.
- Stream Video: For audio/video communication.
Required Credentials
MOONSHOT_API_KEY=...
ELEVENLABS_API_KEY=...
DEEPGRAM_API_KEY=...
STREAM_API_KEY=...
STREAM_API_SECRET=...
EXAMPLE_BASE_URL=https://demo.visionagents.aiInitialize and Install Project Dependencies
uv init kimi-k25-agent
cd kimi-k25-agent
uv add vision-agents
uv add "vision-agents[getstream, elevenlabs, deepgram, smart-turn, openai]"Run the Complete Python Script
import os
from dotenv import load_dotenv
from vision_agents.core import Agent, AgentLauncher, Runner, User
from vision_agents.plugins import openai, getstream, deepgram, elevenlabs, smart_turn
load_dotenv()
async def create_agent(**kwargs) -> Agent:
llm = openai.ChatCompletionsLLM(
model="kimi-k2.5",
base_url="https://api.moonshot.ai/v1",
api_key=os.getenv("MOONSHOT_API_KEY"),
)
# Create an agent with video understanding capabilities
agent = Agent(
edge=getstream.Edge(),
agent_user=User(name="Video Assistant", id="agent"),
instructions="You are a voice/video/vision agent powered by Kimi K2.5. You can answer questions about the users' video camera feed and help them perform coding tasks via screen sharing.",
llm=llm,
stt=deepgram.STT(),
tts=elevenlabs.TTS(),
turn_detection=smart_turn.TurnDetection(),
processors=[],
)
return agent
async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None:
await agent.create_user()
call = await agent.create_call(call_type, call_id)
async with agent.join(call):
# The agent will automatically process video frames and respond to user input
await agent.finish()
if __name__ == "__main__":
Runner(AgentLauncher(create_agent=create_agent, join_call=join_call)).cli()This script launches a realtime video in your browser, where you can enable your camera/mic, and ask what the agent sees or share your screen for vision help.