# Router Intro Routers are important when working with AI. You can use our routers when setting up an agent. You can also use them independently of the agent, which is what these docs show. We recommend using a router and naming it after the use case in your app. Names like `healthcare_transcriptions`, `ivy_voice_ai` or other project specific names are good. There are a few reasons why you want routers with use case based names like this: - The cost and performance of models change rapidly. In the past at Stream we've spent tens of thousands of additional dollars just because updating to a new model took time. - AI providers don't have perfect uptime. They are all dealing with rapid growth and limited GPU availability, so when a vendor goes down you want a fallback. - The router supports model specific overwrites, so you don't lose much flexibility. Note that your named router can also point to any of our predefined routes, such as `llm-fast` or `en-low-latency`. ## Router example One YAML file sets up transcription, voice and language for a dental office's voice agent: ```yaml title="routers/dentist_ai.yaml" name: dentist_ai stt: # Deepgram Flux first, then whatever is healthiest in the English low latency group. providers: [deepgram/flux-general-en, en-low-latency] keyterms: [crown, root canal, periodontal, Invisalign] # Patient audio only goes to providers that neither train on it nor keep it. data_policy: allow_training: false retention: none tts: target: en-low-latency voice: custom:front_desk llm: providers: [openai/gpt-5-mini, llm-fast] reasoning_effort: low ``` Sync it. The sync command picks up the YAML file in the `routers` folder, and configs are written by name, so running it twice edits rather than duplicates: ```sh curl -fsSL https://getstream.io/cli/beta/install.sh | bash # the agents commands are in the beta CLI getstream agents sync --router=dentist_ai ``` Then open it by name. Each modality uses the defaults from the file: ```python from vision_agents.plugins import stream router = stream.Client().router("dentist_ai") async with router.stt.realtime() as live: ... ``` When a new model is better or cheaper, change this file and sync again. Every call that uses `dentist_ai` follows, without a code change. ## Predefined routes Name any of these as a `target`, or inside a `providers` list. A route tries its preferred model first while it is healthy, and the rest of the group catches it when it is not. Stream's picks, one set per modality: | Route | Modality | What it is for | | -------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------- | | `stt-fast` | STT | A live call. Deepgram Flux first, then Cartesia Ink 2, Inworld and Nemotron. | | `stt-accurate` | STT | Hearing every word. Muse first, which also labels speakers, then ElevenLabs Scribe, Ink 2 and Nemotron. | | `tts-fast` | TTS | The quickest live voices. Cartesia Sonic 3.6 first, then the rest of the fast tier. | | `tts-quality` | TTS | The live voices listeners rate best. ElevenLabs v4 first, then Cartesia Sonic 3.6, Inworld, Breeze and ElevenLabs v4 Turbo. | | `llm-conversational` | LLM | Simple, short turns. Gemma 4 26B-A4B first, then Gemma 4 31B on Cerebras and GPT-5.6 Luna. | | `llm-fast` | LLM | The quickest strong answer in any language. Gemini 3.8 Flash first, then the rest of the fast tier. | | `llm-smart` | LLM | Hard questions an agent must get right, at the cost of speed. GPT-5.6 Sol first. | | `llm-thinking` | LLM | Skills: the part of a turn the talking model hands off. GPT-6 Sol first. | | `vlm` | LLM | Questions about images. GPT-6 Luna first. | | `sts-fast` | STS | The quickest speech-to-speech conversation. Gemini 3.8 Live first. | | `sts-vision` | STS | A speech-to-speech conversation the model can watch as well as hear. GPT-Realtime-2 first. | Groups by language and speed, with the same names in every modality: | Route | Languages | What it is for | | ---------------------------- | --------- | ------------------------------------------------------------- | | `en-low-latency` | English | Live, quickest to the first word. | | `multilingual-low-latency` | Any | Live, quickest to the first word. | | `en-high-accuracy` | English | The best quality, even if it is slower. | | `multilingual-high-accuracy` | Any | The best quality, even if it is slower. | | `en-recorded` | English | Whole recordings rather than a live stream. STT and TTS only. | | `multilingual-recorded` | Any | Whole recordings rather than a live stream. STT and TTS only. | `GET /v1/{modality}/routes` lists the routes a deployment offers, and `GET /v1/{modality}/routes/{target}` shows which models a route resolves to right now. ## Data policy Every provider in the catalogue declares what happens to what it is sent: `trains_on_data` (`yes`, `no` or `unknown`) and `retention` (`none`, a duration such as `30d`, `unspecified` or `unknown`). A config can require a policy: ```yaml stt: data_policy: allow_training: false # only providers that said they do not train on it retention: none # only providers that keep nothing ``` A request is then only routed to providers that meet it. `unknown` and `unspecified` meet nothing, because a vendor who has said nothing is not a vendor who has said no. If no provider qualifies, the request is refused rather than sent somewhere that does not qualify. ## Daily limits An app caps what one end user may spend in a UTC day with `quota` in its policy. Send it with a server-side token: ```json title="PUT /v1/policies/app" { "quota": { "messages_per_user_per_day": 500, "tokens_per_user_per_day": 10000000 } } ``` - `quota` is for an app. An organization policy that sets it is refused with a 400. - `0` is no cap. A number you leave out takes the deployment's default. - The PUT replaces the whole policy, so send the rest of the policy too. A PUT without `quota` clears it, and the deployment's default applies again. An end user with nothing left gets a 429 `rate_limited` with a `Retry-After` header: the seconds until the limit resets. On a socket that is already open, the refusal is an `error` frame. An HTTP response to an end user's request carries three headers while a limit applies. A turn on an open socket or call carries none. | Header | Value | | ------------------- | ------------------------------------------------------- | | `X-Quota-Limit` | The limit nearest to being spent, in messages or tokens | | `X-Quota-Remaining` | What is left of that limit | | `X-Quota-Reset` | The unix time of the next UTC midnight | The router counts each user id and each address against the limit. The one nearest to being spent decides. End users behind one proxy share an address. The router counts an end user's own turns, also in a session a backend opened for them. It never counts a backend's own turns, or answers from `mock/*` models. ## Learn more - [STT](https://getstream.io/agents/docs/python/stt/): Live and recorded transcription, and the options each provider takes. - [TTS & Voices](https://getstream.io/agents/docs/python/tts/): Speaking live, recordings, stock voices and cloning your own. - [LLM](https://getstream.io/agents/docs/python/llm/): Asking a model, the options each provider takes, and fallback. --- For the most recent version of this documentation, visit [https://getstream.io/agents/docs/python/router-intro/](https://getstream.io/agents/docs/python/router-intro/).