# STT

Speech to text models are improving daily. It's hard to keep up with the new model launches.
That's why it's generally best to use a router.

STT models fall into two groups. Models are either optimized for speed or accuracy.
For voice AI you want to use the fast models.
For adding transcriptions to movies, clips etc it's typically better to use the slower and more accurate models.

## Fast Speech to Text - Voice AI

`en-low-latency` is the fast, English optimized router.
It routes to whatever is currently state of the art for speed and English.

Name it as the `target` of your router:

```yaml title="routers/my-router-name.yaml"
name: my-router-name

stt:
  target: en-low-latency
```

Then open the router from the client:

```ts label="JavaScript"
import { Socket, flag, text } from "@stream-io/calypso";

const live = await Socket.open(
  client.backend,
  await client.backend.socketURL("/v1/stt/stream"),
);
try {
  live.send({ type: "start", config_id: "my-router-name", stt: {} });

  live.sendAudio(pcm); // 16-bit PCM at 16 kHz, sent chunk by chunk as it is captured
  for await (const transcript of live.messages()) {
    if (transcript instanceof Uint8Array) continue;
    if (transcript.type === "transcript" && flag(transcript, "final")) {
      console.log(text(transcript, "text"));
    }
  }
} finally {
  live.close();
}
```

```python label="Python"
from vision_agents.core.stt import Transcript
from vision_agents.plugins import stream

client = stream.Client()
router = client.router("my-router-name")

async with router.stt.realtime() as live:
    await live.process_audio(pcm, participant)  # resampled to 16 kHz for you
    async for transcript in live.output:
        if isinstance(transcript, Transcript) and transcript.final:
            print(transcript.text)
```

```ruby label="Ruby"
router = api.router("my-router-name")

router.stt.realtime do |live|
  live.send_audio(pcm) # 16-bit PCM at 16 kHz, sent chunk by chunk as it is captured
  live.each do |frame|
    puts frame["text"] if frame["type"] == "transcript" && frame["final"]
  end
end
```

```php label="PHP"
$live = $client->router('my-router-name')->stt->realtime();
try {
    $live->sendAudio($pcm); // 16-bit PCM at 16 kHz, sent chunk by chunk as it is captured
    foreach ($live->frames() as $frame) {
        if (is_array($frame) && $frame['type'] === 'transcript' && ($frame['final'] ?? false) === true) {
            echo $frame['text'], "\n";
        }
    }
} finally {
    $live->close();
}
```

```go label="Go"
live, err := client.Router("my-router-name").STT().Realtime(ctx, nil)
if err != nil {
	return err
}
defer live.Close()

if err := live.Send(pcm); err != nil { // 16-bit PCM at 16 kHz, sent chunk by chunk as it is captured
	return err
}
for transcript := range live.Transcripts() {
	if transcript.Final {
		fmt.Println(transcript.Text)
	}
}
```

```csharp label="C#"
await using var live = await client.Router("my-router-name").Stt.RealtimeAsync();

await live.SendAsync(pcm); // 16-bit PCM at 16 kHz, sent chunk by chunk as it is captured
await foreach (var transcript in live.ReadAllAsync())
{
    if (transcript.Final)
    {
        Console.WriteLine(transcript.Text);
    }
}
```

```rust label="Rust"
use vision_agents::types::SttOptions;

let mut live = client
    .router("my-router-name")
    .transcriber(SttOptions::default())
    .await?;

live.send_audio(&pcm).await?; // 16-bit PCM at 16 kHz, sent chunk by chunk as it is captured
while let Some(frame) = live.next().await {
    let frame = frame?;
    if frame.kind() == "transcript" && frame.flag("final") {
        println!("{}", frame.text("text"));
    }
}
live.close().await;
```

For multi language use `multilingual-low-latency`.

## Accurate/Slow Speech To Text

`en-recorded` is the accurate, English optimized route for files. Give it its own router:

```yaml title="routers/my-recordings.yaml"
name: my-recordings

stt:
  target: en-recorded
```

Hand over a whole recording, audio or video, and get one transcript back:

```ts label="JavaScript"
import { readFile } from "node:fs/promises";

let transcript = await client.post("/v1/stt/recordings", {
  body: {
    config_id: "my-recordings",
    source: { audio: (await readFile("interview.mp4")).toString("base64") },
  },
});
while (transcript.status === "queued" || transcript.status === "running") {
  await new Promise((resolve) => setTimeout(resolve, 1000));
  transcript = await client.get("/v1/stt/recordings/{id}", {
    path: { id: transcript.id },
  });
}
console.log(transcript.text);
```

```python label="Python"
from vision_agents.plugins import stream

router = stream.Client().router("my-recordings")

transcript = await router.stt.recording("interview.mp4")
print(transcript.text)
```

```ruby label="Ruby"
router = api.router("my-recordings")

transcript = router.stt.recording("interview.mp4")
puts transcript["text"]
```

```php label="PHP"
use GetStream\VisionAgents\Generated\RecordingSource;

$transcript = $client->router('my-recordings')->stt->recording(
    new RecordingSource(audio: base64_encode(file_get_contents('interview.mp4'))),
);
echo $transcript->text, "\n";
```

```go label="Go"
recording, err := stream.File("interview.mp4")
if err != nil {
	return err
}
transcript, err := client.Router("my-recordings").STT().Recording(ctx, recording, nil)
if err != nil {
	return err
}
fmt.Println(*transcript.Text)
```

```csharp label="C#"
var transcript = await client.Router("my-recordings").Stt.RecordingAsync(Recorded.File("interview.mp4"));
Console.WriteLine(transcript.Text);
```

```rust label="Rust"
use vision_agents::Recording;
use vision_agents::types::SttOptions;

let transcript = client
    .router("my-recordings")
    .transcribe(
        Recording::Audio(std::fs::read("interview.mp4")?),
        SttOptions::default(),
        None,
    )
    .await?;
println!("{}", transcript.text.unwrap_or_default());
```

There is also a multi language optimized router, `multilingual-recorded`.

## Custom STT router

You can also create your own router with a custom set of models. This config tries Deepgram Flux first, then Cartesia Ink 2, and ends with the `en-low-latency` group.

```yaml
name: live-support

stt:
  providers: [deepgram/flux-general-en, cartesia/ink-2, en-low-latency]
```

The model used will automatically switch from deepgram to cartesia when deepgram has an outage.

## Options

| Option                           | Path      | What it asks for                                                                               |
| -------------------------------- | --------- | ---------------------------------------------------------------------------------------------- |
| `target`, `providers`            | both      | A route, provider or `provider/model`, or a priority list of them.                             |
| `languages`                      | both      | ISO codes the model must cover.                                                                |
| `mode`                           | both      | `verbatim` keeps the ums and false starts; `smart` removes them and tidies the grammar.        |
| `detect_language`                | both      | Let the provider identify the language instead of being told it.                               |
| `sample_rate`                    | realtime  | Rate of the PCM you send. Zero means 16 kHz.                                                   |
| `interim`                        | realtime  | Partial transcripts as they firm up, not only final ones.                                      |
| `endpointing`                    | realtime  | How a turn is judged over: `silence` for a pause, or `semantic` for a model reading the words. |
| `silence_ms`, `utterance_end_ms` | realtime  | How long a pause ends a turn, for pause endpointing.                                           |
| `diarize`                        | both      | Label each stretch of speech with who said it.                                                 |
| `max_speakers`                   | both      | The most speakers to tell apart.                                                               |
| `keyterms`                       | both      | Words to listen for: names, drugs, product terms.                                              |
| `format`                         | both      | Punctuation, capitalisation and smart formatting of numbers and dates.                         |
| `redact`                         | both      | Remove personally identifying information.                                                     |
| `profanity_filter`               | both      | Mask profanity.                                                                                |
| `events`                         | both      | Tag non-speech audio such as laughter or music.                                                |
| `words`                          | recording | Word-level timestamps.                                                                         |
| `summary`, `entities`            | recording | A summary, and the named entities in the recording.                                            |
| `channels`                       | recording | Transcribe each channel of a multichannel file separately.                                     |
| `output`                         | recording | `srt` or `vtt` subtitles, rendered by the router from the word timings.                        |

## What varies between providers

Not every provider can do every option, and the router refuses rather than drops one it cannot serve:

- **Diarization.** Live, only Grok STT and Muse label speakers; for a recording, Deepgram Nova 3 does. No model can yet be told how many speakers there are, so `max_speakers` is refused.
- **Endpointing.** Deepgram Flux, Cartesia Ink 2, Inworld and ElevenLabs Scribe take endpointing settings. The rest decide where a turn ends themselves.
- **Vocabulary.** Every live model takes `keyterms` except Parakeet and Nemotron. A call full of drug names is routed away from those two.
- **Recording-only features.** Word timings, summaries, entities, redaction, formatting, channels and language detection are served by the batch model, Deepgram Nova 3, so asking for them live is refused.
- **Not served yet.** No model in the catalogue offers `interim` or `events` yet, so both are refused for now. `mode: verbatim` reaches Gemini and Grok, and `mode: smart` only Gemini.

---

For the most recent version of this documentation, visit [https://getstream.io/agents/docs/ruby/stt/](https://getstream.io/agents/docs/ruby/stt/).