Voice-Enabled App in Python: Grok-4 + Fish Audio + Deepgram

Use xAI's Grok-4 with Fish Audio's high-quality, expressive TTS and Deepgram's fast STT to create a Siri-like experience.

The quick video demonstrates a voice agent to human conversation, chaining together the various AI providers in Vision Agents.

AI Services Required

Grok-4 (xAI) as the language model.
Fish Audio for realistic and emotional voice synthesis.
Deepgram to transcribe audio.
Smart-Turn to detect when users start or stop talking.

You'll need Stream credentials and API keys from the following.

XAI_API_KEY=...
FISH_AUDIO_API_KEY=...
DEEPGRAM_API_KEY=...
STREAM_API_KEY=...
STREAM_API_SECRET=...
EXAMPLE_BASE_URL=https://pronto-staging.getstream.io

Set Up the Project

Create a new Python project, install Vision Agents and the requied plugins as dependencies.

uv  init  grok-voice-agent
cd  grok-voice-agent

uv  add  vision-agents
uv  add  "vision-agents[getstream, deepgram, smart-turn]"
#  Fish  Audio  plugin  (assuming  it's  available;  check  pypi  or  visionagents.ai  for  latest)
uv  pip install vision-agents-plugins-fish

Create Your Siri-Like Agent

import  asyncio
import  os
from  vision_agents  import  Agent,  register
from  vision_agents.llm  import  XAILLM # or OpenRouterLLM for Grok-4
from  vision_agents.tts  import  FishAudioTTS
from  vision_agents.stt  import  DeepgramSTT
from  vision_agents.turn_detection  import  SmartTurn
from  vision_agents.stream  import  StreamVideoCall

async  def  main():
    # 1. Grok-4 LLM (use xAI direct or OpenRouter)
    llm  =  XAILLM(
        api_key=os.getenv("XAI_API_KEY"),
        model="grok-4" # or "grok-4-fast" for optimized version
    )

    # 2. Voice components
    tts  =  FishAudioTTS(
        api_key=os.getenv("FISH_AUDIO_API_KEY"),
        reference_id="your_reference_voice_id" # optional for cloned/custom voice
    )
    stt  =  DeepgramSTT(api_key=os.getenv("DEEPGRAM_API_KEY"))
    turn_detector  =  SmartTurn()

    # 3. Create the agent with Grok personality
    agent  =  Agent(
        llm=llm,
        tts=tts,
        stt=stt,
        turn_detector=turn_detector,
        name="Grok Voice Agent",
        system_prompt="""
        You are Grok, built by xAI. Be helpful, witty, and maximally truthful.
        Respond naturally in conversation, just like in the Grok chatbot.
        """
    )

    # 4. Register agent and launch Stream call
    register(agent)

    call  =  StreamVideoCall(
        api_key=os.getenv("STREAM_API_KEY"),
        api_secret=os.getenv("STREAM_API_SECRET"),
        call_type="default",
        call_id="grok-voice-demo"
    )

    await  call.join()
    print("Voice AI app ready! Open this URL in your browser:")
    print(call.url)

    # 5. Run the agent
    await  agent.run(call)

if  __name__  ==  "__main__":
    asyncio.run(main())

Run this Python script to open Stream Video call UI to speak with the agent consisting of Grok-4, Deepgram, and Fish Audio communication pipeline.