Build a Voice AI App with Cartesia Sonic 3 Text-to-Speech

Cartesia Sonic 3 is a multilingual speech synthesis model that can produce realistic, natural-sounding voice output when integrated in AI apps. It has about 200ms latency, adjustable emotions, multiple language support, and the ability to clone voices from short audio samples.

The Cartesia plugin for Vision Agents makes building voice-enabled apps easier. Installing the plugin allows developers to add voice generation with human intonation and timing to AI services.

This demo walks you through how to build an agent that responds with Sonic 3's emotionally nuanced voice.

Key Features of the Demo

A voice agent that speaks with realistic and low-latency response powered by Cartesia Sonic 3.
It offers voice customization, sample rate adjustment, and voice cloning.
The text-to-speech integration works in any agentic pipeline for natural-sounding conversations.

Architecture Overview

Cartesia Sonic 3 for speech synthesis.
Language Model: Use any preferred LLM (the demo uses Gemini 2.0 Flash).
Deepgram for automatic speech recognition (ASR).
Smart-Turn: Native audio turn detection model.
Vision Agents: Python framework for building voice/vision apps.
Stream Video: A network transport for audio/video communication.

Before You Start

Get the following API credentials and store them as environment vatiables in your project.

CARTESIA_API_KEY=...
GEMINI_API_KEY=... # Available in Google AI Studio
DEEPGRAM_API_KEY=...
STREAM_API_KEY=...
STREAM_API_SECRET=...
EXAMPLE_BASE_URL=https://demo.visionagents.ai

Quickstart

# Installation
uv add vision-agents

# Install plugins 
uv add "vision-agents[getstream, cartesia, deepgram, smart-turn, gemini]"

Executable Script

Add this Python script to your project and run it.

import logging

from dotenv import load_dotenv
from vision_agents.core import Runner
from vision_agents.core.agents import Agent, AgentLauncher
from vision_agents.core.edge.types import User
from vision_agents.plugins import cartesia, getstream, gemini, deepgram

logger = logging.getLogger(__name__)

load_dotenv()

async def create_agent(**kwargs) -> Agent:
    # Create agent with TTS
    agent = Agent(
        edge=getstream.Edge(),
        agent_user=User(name="TTS Bot", id="agent"),
        instructions="I'm a TTS bot that greets users when they join.",
        stt=deepgram.STT(),
        llm=gemini.LLM("gemini-2.0-flash"),
        tts=cartesia.TTS(model_id="sonic-3"),
    )

    return agent

async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None:
    # ensure the agent user is created
    await agent.create_user()
    # Create a call
    call = await agent.create_call(call_type, call_id)

    # Join call and wait
    async with agent.join(call):
        await agent.simple_response("tell me something interesting in a short sentence")
        await agent.finish()

if __name__ == "__main__":
    Runner(AgentLauncher(create_agent=create_agent, join_call=join_call)).cli()

You should be able to join the call in your browser, prompt the agent with your voice, and hear a respond from the Sonic 3 speech generation model.

Where To Go From Here

Grab the source code from GitHub.
Check out other Vision Agents sample apps.
Visit the Vision Agents documentation.
Refer to the Cartesia plugin for more configuration options.

Add Text-to-Speech to Apps with Cartesia Sonic 3 & Vision Agents