Mistral Voxtral 3 Speech¶
Mistral's Voxtral 3 family (October 2026, public preview) transcribes and speaks.
With MistralAIProvider for the answers, a RoomKit voice agent runs on Mistral end
to end, on one API key (MISTRAL_API_KEY).
| Model | Id | RoomKit provider | What it does |
|---|---|---|---|
| Voxtral Mini Transcribe 3 | voxtral-mini-transcribe-3 |
VoxtralSTTProvider.transcribe() |
File transcription (default) |
| Voxtral Small Transcribe 3 | voxtral-small-transcribe-3 |
VoxtralSTTProvider.transcribe() |
File transcription, most accurate |
| Voxtral Mini Transcribe Realtime 3 | voxtral-mini-transcribe-realtime-3 |
VoxtralSTTProvider.transcribe_stream() |
Live transcription with turn detection |
| Voxtral Speech 3 | voxtral-mini-tts-3 |
VoxtralTTSProvider |
Text-to-speech, streamed |
Install with pip install roomkit[mistral-voice]. The providers speak Mistral's wire
protocol over websockets and httpx, with no Mistral SDK: the SDK's realtime
client (3.0.0) cannot send turn detection.
Voxtral 3 is served by Mistral's API only; its weights are not published. Mistral's earlier open models (Voxtral Mini 4B Realtime 2602, Apache 2.0; Voxtral 4B TTS 2603, CC BY-NC 4.0) run on vLLM, and are not what these providers call.
Speech-to-text¶
from roomkit.voice.stt import VoxtralSTTConfig, VoxtralSTTProvider
stt = VoxtralSTTProvider(VoxtralSTTConfig(
api_key="...",
language="fr", # None: the model detects it; "fr-CA" is sent as "fr"
context_bias=["Northwind", "Alder Creek Design"], # names to spell right
))
Files¶
transcribe() sends one clip to the file endpoint: PCM (an AudioChunk or an
AudioFrame) as a WAV, a data: URI as the file it carries (MP3, WAV, FLAC…), and
an http(s) URL as file_url, which Mistral fetches (RoomKit never does). Set
model="voxtral-small-transcribe-3" for the most accurate model. Mistral reports no
confidence and no language (null in every answer measured).
Live: two ways a stream ends¶
How a stream ends depends on the VoiceChannel, which the provider cannot see, so it
is configuration:
turn_detection |
Channel | Finals |
|---|---|---|
VoxtralTurnDetection() (default) |
No pipeline VAD (continuous mode) | One per turn the model ends; each turn's start fires the speech-start hooks, and its words, as they arrive, can cut the agent off |
None |
Behind a pipeline VAD | One per utterance, once the stream ends |
The wrong choice loses words: behind a VAD with turn detection on, the channel keeps only the last turn of an utterance; without a VAD and with it off, no final comes until the audio stops.
from roomkit import VoiceChannel
from roomkit.voice.pipeline import AudioPipelineConfig
# The model finds the turns: no VAD in the pipeline
voice = VoiceChannel("voice", stt=stt, tts=tts, backend=backend,
pipeline=AudioPipelineConfig())
Partials carry the text so far: Voxtral's text is final as it arrives and only ever grows. The documented input rates (8, 16, 22.05, 24, 44.1, 48 kHz) go through as they are; any other is resampled to 16 kHz.
Turn detection and latency¶
The model scores how likely the person is still speaking, and its thresholds turn the
score into turn events. VoxtralTurnDetection() carries the service's defaults;
VoxtralTurnDetection.aggressive() is Mistral's faster preset. streaming_delay_ms
is how long the model listens past a word before writing it: any value from 80 to
1200, or 2400 (480 by default).
The turn events come at once, while the text runs a streaming delay behind the
audio: a turn's last words follow its turn.end by up to that delay, and even the
next turn.start (measured: "What is the capital of Italy?" had its last two words
arrive after the next turn had started). So a turn's final waits the delay plus
100 ms (final_wait_ms, 0.58 s by default), and the text arriving until then is
that turn's; a new turn's speech start is reported at once. When the next turn
starts during that wait, the final comes a little sooner, 0.95 of the delay after
the end: the ending turn's last words came up to 0.9 of the delay after it
(0.43 s at 480 ms), and the new turn's first word comes no sooner than a delay
after it — with the plain wait, that word had been joined to the last turn. Words
later than the wait are dropped, with a warning naming final_wait_ms when they
arrive.
With turn detection the provider reports keeps_stream_across_turns, so the
VoiceChannel keeps one realtime session for the whole conversation instead of
reconnecting after each turn. On a session per turn, a turn started within the
wait was sent half to the session being closed and reached the room in pieces
("What is the capital of Italy?", "Answer", "in one word.").
Measured on 2026-10-21, a spoken question fed in real time through a VoiceChannel:
| Settings | Final after the end of speech |
|---|---|
| Defaults (480 ms delay) | 1.7 s |
aggressive(), streaming_delay_ms=240 |
0.7 s |
The faster settings end a turn on a shorter pause: "What is the capital of Italy? Answer in one word." came out as two turns, and the agent answered both.
Speaker labels¶
diarize=True labels who spoke, in a file and live: the provider reports
supports_diarization, and every final names its speaker in segments
(RFC §12.2.3) — Mistral's speaker_1 arrives as "1", so a VoiceChannel names
the voice "Speaker 1".
stt = VoxtralSTTProvider(VoxtralSTTConfig(
api_key="...",
diarize=True,
turn_detection=None, # live, Voxtral labels speakers or detects turns, not both
speaker_references={"Julie": julie_wav, "Sylvain": sylvain_wav}, # optional, live
))
- File: one segment per run of a speaker, with its start and end (the endpoint returns a sentence or so per segment; consecutive ones of one speaker are joined).
- Live: the session sends a few words at a time with their speaker and no
turn events, so a turn ends when another voice starts, or after a pause
without new words (
diarized_pause_ms, 1.5 s by default). One person's words came up to 2 s apart in hesitant speech and a new speaker's 1.5 s after the last (measured), so a long hesitation can end someone's turn in two. - Channel: a
VoiceChannelcarries the labels in continuous mode (no pipeline VAD), keeping one session across turns; behind a VAD it refuses the provider. - Speaker references name known voices live: a 3 to 15 s recording of each
person (WAV, MP3, FLAC or Opus, mostly speech), at most 8, under the name their
words then carry. A voice the references do not match is numbered after them
("3"). Matching is the model's, and partial: in one measured session a voice
came back under its name for its first words, then as "3". The file endpoint
took references and kept its own labels, so the provider sends them live only.
The channel names a label "Speaker Julie"; an
ON_TRANSCRIPTIONhook that setssender_nameto the label makes it "Julie" (the diarization example does).
examples/voice_voxtral_diarization.py
labels the voices around one microphone (REFERENCES="Julie=julie.wav").
Text-to-speech¶
from roomkit.voice.tts import VoxtralTTSConfig, VoxtralTTSProvider
tts = VoxtralTTSProvider(VoxtralTTSConfig(
api_key="...",
voice="voice_3137", # Manon, French; default voice_478 (Sheila, US English)
expressiveness=2.0, # from 1.0; None keeps the service's default
))
Each text is rendered whole and streamed back as it renders (first audio about
0.4–0.5 s, measured). The audio is 24 kHz mono: the service sends 32-bit float
samples for pcm (not documented), which the provider converts to the 16-bit PCM
the voice path carries. synthesize() returns a 16-bit WAV. Mistral's realtime
speech model, which takes text while it is being written, was not served at launch,
so the channel splits a streamed reply into sentences for this provider.
Voices¶
VoxtralTTSProvider.available_voices() lists the 36 Voxtral 3 presets offline, by
slug, with language and gender: English (US and UK), French (voice_592 Marie,
voice_3137 Manon, voice_589 Lea, voice_3141 Camille), and Arabic, Catalan,
German, Spanish, Hindi, Hungarian, Italian, Japanese, Korean, Dutch, Norwegian,
Polish, Portuguese, Romanian, Russian, Turkish, Ukrainian, Vietnamese and Chinese.
await tts.list_voices() asks the service: the same presets, Voxtral 2's
(gb_jane_neutral, fr_marie_neutral…, still accepted), and the voices saved in the
workspace, whose id is the UUID voice takes.
Emotion and pace tags¶
Voxtral 3 voices follow inline tags in the text: [emotion=happy], [pace=slow],
[pace=fast]. A StripBrackets filter in front of the provider drops every
[...] unless told to keep it, so leave it out (or keep the tags) when the agent
writes them.
Errors¶
What the service refuses or fails raises a ProviderError subclass
(VoxtralSTTError, VoxtralTTSError) carrying Mistral's code and type, and
retryable for a rate limit, a server fault, a realtime connection that drops, or a
speech render that ends before its end. An HTTP transport failure (connection
refused, timeout) comes through as httpx's own exception, as with RoomKit's other
HTTP providers. What the service answers:
| Case | Answer |
|---|---|
| Wrong API key | 401 (the realtime socket refuses the upgrade) |
| Realtime model the key cannot use | 403 with no body |
| Streaming delay outside 80–1200, 2400 | error event, code 3051 (refused before connecting) |
| Unknown voice | 404, code 1902, type invalid_voice |
expressiveness below 1.0 |
422 (refused in the config) |
Example¶
examples/voice_voxtral.py
runs the agent on the local microphone: Voxtral hears you, Mistral Small answers,
Voxtral Speech speaks. It runs the faster turn settings, aggressive() with a
240 ms delay; TURNS=default STREAMING_DELAY_MS=480 gives the service's defaults.
VOICE=voice_3137 VOICE_LANGUAGE=fr makes it French. The model adds emoji to its
answers despite the prompt: tts_filter=StripEmoji() drops them before Voxtral
Speech.
MISTRAL_API_KEY=... uv run --extra mistral-voice --extra mistral \
--extra local-audio --extra webrtc-aec python examples/voice_voxtral.py
Not covered yet¶
- The eager end of a turn (
turn.eager_end,turn.resume), which would let the agent prepare its answer early: RoomKit's STT contract has no such signal yet. - Voice cloning and voice management, and
phoneme_hints. - Mistral's Voice Agents SDK, a whole hosted agent stack on LiveKit: the layer RoomKit itself provides.