The Kimi K2.5 vision model can see the environment around you and asnwers questions about what it sees. This capability is similar to the Gemini Live Android or iOS app. Let's implement the model in Vision Agents for integrated video, vision, and voice experience.
Overview of the Demo
- A real-time voice agent that analyzes your live camera feed, answers questions about what it sees, and helps with vision tasks.
- Leverages Kimi K2.5's native multimodal understanding for accurate visual descriptions and reasoning.
- Low-latency conversations with seamless turn-taking using Smart Turn.
- A swappable voice pipeline using OpenAI-compatible API access to Kimi K2.5 via Vision Agents.
Voice Architecture
The demo uses the peech-to-text -> LLM -> text-to-speech pipeline approach, where each component is replaceable.
- Vision Agents: A platform for building voice apps in Python.
- Language Model: Kimi K2.5 (via OpenAI-compatible API).
- Text-to-Speech: ElevenLabs.
- Speech-to-Text: Deepgram.
- Smart-Turn: A free and open-source turn detection project.
- Stream Video: For audio/video communication.
Required Credentials
bash
123456MOONSHOT_API_KEY=... ELEVENLABS_API_KEY=... DEEPGRAM_API_KEY=... STREAM_API_KEY=... STREAM_API_SECRET=... EXAMPLE_BASE_URL=https://demo.visionagents.ai
Initialize and Install Project Dependencies
shell
12345uv init kimi-k25-agent cd kimi-k25-agent uv add vision-agents uv add "vision-agents[getstream, elevenlabs, deepgram, smart-turn, openai]"
Run the Complete Python Script
python
12345678910111213141516171819202122232425262728293031323334353637import os from dotenv import load_dotenv from vision_agents.core import Agent, AgentLauncher, Runner, User from vision_agents.plugins import openai, getstream, deepgram, elevenlabs, smart_turn load_dotenv() async def create_agent(**kwargs) -> Agent: llm = openai.ChatCompletionsLLM( model="kimi-k2.5", base_url="https://api.moonshot.ai/v1", api_key=os.getenv("MOONSHOT_API_KEY"), ) # Create an agent with video understanding capabilities agent = Agent( edge=getstream.Edge(), agent_user=User(name="Video Assistant", id="agent"), instructions="You are a voice/video/vision agent powered by Kimi K2.5. You can answer questions about the users' video camera feed and help them perform coding tasks via screen sharing.", llm=llm, stt=deepgram.STT(), tts=elevenlabs.TTS(), turn_detection=smart_turn.TurnDetection(), processors=[], ) return agent async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None: await agent.create_user() call = await agent.create_call(call_type, call_id) async with agent.join(call): # The agent will automatically process video frames and respond to user input await agent.finish() if __name__ == "__main__": Runner(AgentLauncher(create_agent=create_agent, join_call=join_call)).cli()
This script launches a realtime video in your browser, where you can enable your camera/mic, and ask what the agent sees or share your screen for vision help.