Skip to content

Rime: STT, TTS and Prism

Install pip install 'roomkit[rime]' and supply RIME_API_KEY from the environment. The three providers implement RoomKit's existing voice contracts independently:

Provider Use Audio and language
RimeSTTProvider One utterance, with revised partials Mono PCM16 at 16 kHz; English or Spanish
RimeTTSProvider Coda or Mist v3 synthesis, including streamed text input Mono PCM16 at 8/16/24/48 kHz; language depends on model and voice
RimeRealtimeProvider Prism speech-to-speech conversations 16 kHz input, 24 kHz output; English

Transcription

import os

from roomkit.voice.stt import RimeSTTConfig, RimeSTTProvider

stt = RimeSTTProvider(RimeSTTConfig(
    api_key=os.environ["RIME_API_KEY"],
    language="en",                 # en or es; no automatic detection
    mode="written",                # verbatim preserves spoken wording and fillers
    context_terms=["RoomKit"],
))

result = await stt.transcribe(audio)  # AudioFrame, AudioChunk or a WAV data URL
async for result in stt.transcribe_stream(audio_chunks, language="es"):
    # Each partial replaces the previous hypothesis; it is not a text delta.
    update_transcript(result.text, final=result.is_final)

Other PCM rates and stereo input are converted to 16 kHz mono. Remote audio URLs must be fetched separately. Input ends when the iterator finishes or yields an is_final chunk. Only the service's done event becomes a final transcript; a lost connection raises instead of promoting the last partial. The final language is the one the service reports.

Rime does not detect utterance boundaries. With VoiceChannel, configure a pipeline VAD, or use batch_mode=True and call flush_stt() explicitly. Do not use continuous STT mode. There is no diarization or confidence score. Each utterance owns a connection, limited to 120 seconds including processing and 16 MiB of PCM. final_timeout_s (default 10 seconds) bounds the wait after the end marker. Context terms joined with ", " must fit in 1024 UTF-8 bytes.

Synthesis

from roomkit.voice.tts import RimeTTSConfig, RimeTTSProvider

tts = RimeTTSProvider(RimeTTSConfig(
    api_key=os.environ["RIME_API_KEY"],
    model="coda",                  # or mistv3
    voice_id="astra",
    language="en",
    sample_rate=24000,
    time_scale_factor=1.0,          # lower is faster, higher is slower
))

audio = await tts.synthesize("Hello!")  # WAV data URL, duration and transcript
async for chunk in tts.synthesize_stream("Hello!"):
    await play_pcm(chunk)
async for chunk in tts.synthesize_stream_input(llm_text_stream):
    await play_pcm(chunk)

Text fragments retain their spaces and punctuation. Audio arrives while text is still being supplied. Each synthesis has an independent socket and context; closing its iterator cancels only that synthesis. timeout_s bounds the whole synthesis, including time waiting for text.

Choose a voice matching the model and language. available_voices() supplies a starter English voice; await tts.list_voices(language="fr") queries the public catalog for the configured model. For example, Coda supports voice_id="aurelie", language="fr". Dialogue context and cloning are unsupported. Use these providers in a VoiceChannel with your existing backend, VAD and AI channel. examples/voice_rime.py synthesizes a sentence and transcribes it with the same Rime key, without a microphone or separate LLM.

Prism speech-to-speech

from roomkit import RealtimeVoiceChannel
from roomkit.voice.realtime import RimeRealtimeConfig, RimeRealtimeProvider

provider = RimeRealtimeProvider(RimeRealtimeConfig(
    api_key=os.environ["RIME_API_KEY"],
    voice_id="astra",              # astra, aurora, brando, wawona
    eagerness="auto",
    end_of_turn_timeout_ms=5000,
))
channel = RealtimeVoiceChannel(
    "voice", provider=provider, transport=transport,
    input_sample_rate=16000, output_sample_rate=24000,
    system_prompt="Keep your spoken replies brief.",
)

Leave the channel's pipeline VAD unset: it then uses Prism's turn detection. The adapter paces input in 20 ms frames and fills capture gaps with silence. Keep microphone capture active during playback for barge-in; use headphones or acoustic echo cancellation. Buffering is limited to five seconds: supply live audio, not a recording uploaded faster than real time. For a microphone/speaker conversation:

uv sync --extra rime --extra local-audio --extra webrtc-aec --extra console
RIME_API_KEY=... CONSOLE=1 uv run examples/realtime_voice_local_rime.py

CONSOLE=1 enables the same voice dashboard as the other local examples. It is disabled when the variable is unset or 0, and closed on shutdown.

The example enables WebRTC AEC by default through the existing local backend, which resamples the 24 kHz playback reference to the 16 kHz capture rate. AEC=webrtc|speex|0 chooses or disables cancellation; AEC_DELAY_MS sets an optional measured delay. Without AEC, playback mutes the microphone. MUTE_MIC=0|1 overrides this choice; use AEC=0 MUTE_MIC=0 with headphones.

The provider maps transcripts, audio, speech/response boundaries, errors, usage and function calls onto the realtime callbacks. Late user transcripts remain independent of the assistant's reply. Tool results are submitted once; continuation waits for every result's acknowledgement and the parent reply's completion. Cancelling a response does not undo tools already issued.

Text injection preserves intent: user text becomes a typed user turn, system text becomes an application instruction, and assistant text becomes an instruction to speak the quoted line. Silent injection appends history. sent means the service acknowledged the command; a missing acknowledgement returns unknown to avoid inviting a replay of a possibly accepted turn.

reconfigure() updates instructions, tools, voice and interrupt_response on the same connection. provider_config also accepts eagerness, end_of_turn_timeout_ms and truncation; eagerness and end-of-turn timeout are fixed at setup. Temperature and client endpointing are explicitly rejected.

A dropped connection is reported and closed without automatic reconnection that would silently reset the conversation. Context preservation is not advertised because Prism can evict old turns. RoomKit's full_duplex stays false: Prism exposes response boundaries and native function calls.

Verification and lifecycle

uv run pytest tests/voice/stt/test_rime.py tests/voice/tts/test_rime.py tests/voice/realtime/test_rime.py

These contract tests use local WebSocket servers. tests/voice/test_rime_live.py is skipped unless RIME_API_KEY is set; enabling it makes billable service calls. Close providers with await close() when using them directly. Channels manage their lifetime in a RoomKit app.

Protocol references: STT, Coda, Mist v3, Prism.