Multi-Party Conferences: RoomKit on an External SFU with LiveKit

RoomKit 0.38.0 adds multi-party conference support: real meetings — Alice, Bob, and Carol on video-call infrastructure — with a RoomKit agent in the room to transcribe, answer, and moderate. The media plane is an external SFU, and LiveKit is the first production backend. One design rule shapes everything else: RoomKit never sits in the media path between human participants.

Bridge for Calls, Conference for Meetings

RoomKit could already connect several parties with its in-process AudioBridge — by mixing their audio inside RoomKit. That works for a handful of calls RoomKit itself terminates, but it puts the framework in the media path, and that caps how far it scales. The conference channel exists to cross that boundary: when the parties are humans in a meeting, the SFU carries the media and RoomKit steps out of the path. Bridge for a handful of calls RoomKit terminates; conference for meetings.

 alice ──┐                              ┌── RoomKit bot participant
         │        ┌──────────┐          │     ├─ subscribes audio tracks
 bob ────┼───────►│   SFU    │◄─────────┘     │    └─ lane per track → VAD → STT
         │        │ (LiveKit)│                ├─ publishes one AI voice track
 carol ──┘        └──────────┘                └─ observes joins, mutes, quality
                       ▲
              media stays here — RoomKit
              never routes a human's packets

The model, defined normatively in RFC §12.10, comes down to a few principles:

The conference is the room. A conference is 1:1 with a RoomKit Room. Conference participants are Room participants, transcriptions are RoomEvents, and hooks, permissions, and cross-channel broadcast apply with no conference-specific exceptions.
Humans connect directly to the SFU. RoomKit mints access credentials and observes lifecycle through backend events; it never proxies client signaling or media.
RoomKit joins as a bot participant — one server-side connection per conference, subscribing audio tracks and publishing the AI's voice track.
Tracks are the unit of media identity. Each participant's speech arrives on their own track, which is why the conference pipeline has no diarization stage and no server-side echo cancellation — clients cancel their own echo, and the SFU never mixes.
The backend boundary is decoded frames and opaque credentials. SDP, ICE, codecs, and simulcast appear in no framework interface.

Deliberately Not livekit-agents

The integration uses LiveKit as an SFU and nothing else: the server API to administer rooms, mint credentials, and moderate; the realtime SDK to give the framework one bot participant. What is deliberately absent is livekit-agents. RoomKit already owns VAD, speech-to-text, synthesis, turn detection, and interruption — a transport that delivered pre-segmented speech would break the separation the RFC draws, and would hide exactly the class of raw-audio bugs a real backend exists to surface.

Mock First, SFU Later

The conference ships with a full MockConferenceBackend, so you can build and test the whole flow with no SFU and no credentials:

from roomkit import MockConferenceBackend, RoomKit
from roomkit.channels.conference import ConferenceChannel
from roomkit.voice.stt.mock import MockSTTProvider

kit = RoomKit()
conference = ConferenceChannel(
    "conf", backend=MockConferenceBackend(), stt=MockSTTProvider()
)
kit.register_channel(conference)
await kit.create_room("standup")
await kit.attach_channel("standup", "conf")

# The credential a human's client uses to connect to the SFU directly.
await kit.ensure_participant("standup", "conf", "alice", display_name="Alice")
access = await conference.mint_access("standup", "alice")

Moving to a real SFU is a one-line swap:

from roomkit import LiveKitConferenceBackend, LiveKitConfig

backend = LiveKitConferenceBackend(
    LiveKitConfig(url="wss://my-project.livekit.cloud")  # key and secret from env
)

And the whole meeting-assistant loop — speech in, transcription, LLM, voice out — is just attaching an AIChannel to the same room. No conference-specific wiring:

from roomkit.channels.ai import AIChannel
from roomkit.providers.anthropic import AnthropicAIProvider, AnthropicConfig

conference = ConferenceChannel("conf", backend=backend, stt=stt, tts=tts)
ai = AIChannel(
    "ai",
    provider=AnthropicAIProvider(AnthropicConfig(api_key=...)),
    system_prompt="You are the meeting's voice assistant.",
)
kit.register_channel(conference)
kit.register_channel(ai)
await kit.attach_channel("standup", "conf")
await kit.attach_channel("standup", "ai")
# Done. Speak, and the bot answers on its own audio track.

Prefer speech-to-speech over the STT → LLM → TTS pipeline? Configure a realtime provider instead. The conference mixes the participants' audio N→1 for the provider (with 1/√k headroom, in 20 ms windows), holds one provider session per conference, and still runs per-track STT lanes beside it if you want an attributed transcript:

from roomkit import ConferenceRealtimeConfig
from roomkit.providers.gemini import GeminiLiveProvider

conference = ConferenceChannel(
    "conf",
    backend=backend,
    realtime=ConferenceRealtimeConfig(
        provider=GeminiLiveProvider(api_key=...),
        system_prompt="You are the meeting's voice assistant.",
    ),
    stt=stt,  # optional but recommended: the attributed transcript
)

Mechanics Worth Knowing

The lazy join. The bot does not join the SFU room on attach. It joins when there is a reason to: a mint_access() call (the framework's own advance notice that a human is about to connect — by the time the tab opens, the bot is on the participant list), an occupancy probe at attach (for a channel restarted mid-meeting), or an event that needs delivery. And with no needs configured at all — no STT, no TTS, no recording — the bot never joins: a bot kept in a meeting only to watch it is exactly the silent observer the RFC's disclosure rules exist to surface, and RoomKit does not offer that as a mode.

Hot-plugging intelligence. Needs can change mid-meeting: plug_stt(), plug_tts(), plug_recording(), plug_realtime() and their unplug_* counterparts. Plugging STT into a running meeting retroactively subscribes the tracks already published; unplugging the last need takes the bot out of the room.

The asymmetry of visibility. A bot can start as a hidden observer (ConferenceGrants.observer()) — the event bridge without the listening — and be made visible mid-meeting:

# Mid-meeting, the host starts the notetaker, and policy says: visible.
await kit.unmute(room_id, "conf")  # collection opens
await conference.set_bot_grants(ConferenceGrants.for_bot(listens=True))

Hidden → visible happens in place: the SFU announces the bot to already-connected clients. But no SFU interface can un-tell them — so a visible → hidden change always replaces the session, because the announced leave is the only retraction every backend delivers.

Lanes. Each subscribed audio track gets its own lane — a bounded queue and task running frames → [Resampler] → [AGC] → [Denoiser] → VAD → STT. One utterance becomes one RoomEvent, attributed by track identity. Queues are bounded at about two seconds of audio; at saturation the oldest frame is dropped and counted, never silently.

Running LiveKit Locally

The whole thing develops against a local LiveKit in Docker. Two details cost an afternoon to discover, so here they are for free. First, the config — one UDP port instead of the default 50000–60000 range, which macOS publishes very slowly:

port: 7880
rtc:
  tcp_port: 7881
  udp_port: 7882
  node_ip: 127.0.0.1
  use_external_ip: false

docker run --rm -p 7880:7880 -p 7881:7881 -p 7882:7882/udp \
    -e LIVEKIT_CONFIG="$(cat livekit.yaml)" \
    livekit/livekit-server --dev --bind 0.0.0.0

Second, the --bind 0.0.0.0: LiveKit's --dev mode binds to loopback inside the container, where a published port cannot reach it — and the advertised ICE candidate has to be an address on the host side of the NAT, not the container's.

Limits, Honestly

One worker per conference. The one-room-one-conference reservation is enforced by a single RoomKit instance; multi-worker coordination over a shared store is not yet a contract.
Audio only. No vision on conference video tracks, no bot-published video, no video egress recording — yet.
E2EE and intelligence are mutually exclusive. e2ee=True combined with STT or recording is refused at construction — a bot that can read encrypted media would be lying about the encryption.
Recording consent is yours. The framework provides the ON_RECORDING_STARTED consent point; consent flows and encryption at rest are the integrator's responsibility today.
Capabilities report what is wired, not what LiveKit sells. Remote unmute and SIP dial-in are declared only where the deployment actually supports them.

Getting Started

pip install roomkit[livekit]

LIVEKIT_URL=wss://my-project.livekit.cloud \
LIVEKIT_API_KEY=... \
LIVEKIT_API_SECRET=... \
python your_conference.py

The conference guide walks through the full lifecycle — admission, moderation, recording, resilience — and the RFC (§12.10) is the normative spec; the LiveKit backend implements it at conformance Level 3. The MockConferenceBackend means your integration tests never need an SFU. This is the first release of the conference channel — if you put an agent in a real meeting with it, we'd love to hear what happens.