AI Providers¶

AIProvider ¶

Bases: ABC

AI model provider for generating responses.

name `property` ¶

name

Provider name (e.g. 'anthropic', 'openai').

supports_vision `property` ¶

supports_vision

Whether this provider can process images.

supports_streaming `property` ¶

supports_streaming

Whether this provider supports streaming token generation.

supports_structured_streaming `property` ¶

supports_structured_streaming

Whether this provider supports structured streaming with tool calls.

supports_response_schema `property` ¶

supports_response_schema

Whether :meth:generate honours :attr:AIContext.response_schema.

A provider default, not a fact about every model it can reach: a model or an OpenAI-compatible server may still refuse the constraint, which surfaces as an error rather than as prose.

The contract is the implementation's to keep (RFC §6.7): refuse a schema it cannot carry before sending anything, and never return an answer that does not satisfy it. The helpers in :mod:roomkit.providers.ai.response_schema do both; a provider that ignores the field breaks the contract.

supports_deferred_tools `property` ¶

supports_deferred_tools

Whether the model can hold a tool declared but unseen.

Such a tool (AITool.defer_loading) stays out of the model's view and out of the cached prompt prefix until a tool result references it (AIToolResultPart.references), so a tool that has to appear mid-turn does not change the declaration (RFC §6.4). False by default: a provider that cannot declares only what the model may see.

supports_response_schema_with_tools `property` ¶

supports_response_schema_with_tools

Whether a response schema may ride a turn that also carries tools.

The model then either calls tools or answers in the schema, and only the final answer is checked (RFC §6.7). False wherever the constraint is a decoding grammar: it stops the model from calling a tool, and the model invents a schema-valid answer instead.

model_name `abstractmethod` `property` ¶

model_name

Model identifier (e.g. 'claude-opus-5', 'gpt-5.6-sol').

context_window `property` ¶

context_window

Input context window of the active model in tokens, if known.

Resolved offline from the curated :meth:available_models catalog keyed by :attr:model_name — no API key or network. Returns None when the active model is absent from the catalog (custom / local model ids, e.g. an arbitrary vLLM model string), so callers must degrade gracefully rather than assume a window.

available_models `classmethod` ¶

available_models()

Offline metadata for the models roomkit can describe without a key.

This is not the discovery surface. A provider's lineup turns over faster than a release cycle, so a hand-maintained list can never be the authoritative answer to "what does this provider offer" — that is :meth:list_models, which asks the provider. What this list is for is the metadata roomkit needs before any network call exists: it backs :attr:context_window (a sync property, so it cannot await an API) and backfills the sparse ids a live endpoint returns, via :meth:_merge_curated.

A model absent from it is an ordinary outcome, not an error: the caller gets context_window is None and degrades, which is the point — an unknown window is safer than a stale one. The base returns an empty list; providers override it.

list_models `async` ¶

list_models()

Models reported live by the provider's API — the discovery surface.

Always current, and the only answer that reflects the caller's own account (entitlements, regional availability, locally loaded weights). The base implementation falls back to :meth:available_models for providers whose API exposes no models endpoint; the rest override this to query it, backfilling missing metadata via :meth:_merge_curated.

catalog_entry ¶

catalog_entry()

The offline :class:ModelInfo for the active model, if known.

The single place a provider should read its own model's metadata from. A second hardcoded table — a tuple of vision-capable prefixes, say — duplicates what :meth:available_models already states and rots independently of it, which is how a provider ends up reporting a current model as text-only.

Returns None for an id the catalog does not carry (a custom or local model behind base_url, a snapshot newer than this release).

generate `abstractmethod` `async` ¶

generate(context)

Generate an AI response from the given context.

Parameters:

Name	Type	Description	Default
`context`	`AIContext`	Conversation context including messages, system prompt, temperature, and target channel capabilities.	required

Returns:

Type	Description
`AIResponse`	The AI response with content, usage stats, and optional
`AIResponse`	tasks/observations.

generate_stream `async` ¶

generate_stream(context)

Yield text deltas as they arrive. Override for streaming providers.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events (thinking deltas, text deltas, tool calls, done).

A provider whose wire format fragments a tool call's arguments SHOULD also yield :class:StreamToolCallDelta per fragment; it is optional, and one that delivers whole calls yields none.

The default reads what the provider has, so every provider serves the one tool loop unchanged (RFC §6.4): a turn without tools streams through generate_stream where the provider streams text, and any other turn wraps generate(), everything it returned kept. Override for true streaming support.

close `async` ¶

close()

Release resources. Override in subclasses that hold connections.

AIContext ¶

Bases: BaseModel

Context passed to AI provider for generation.

max_tokens `class-attribute` `instance-attribute` ¶

max_tokens = None

Output cap for this turn. None means "not set for this turn", which lets each provider fall back to its own configured max_tokens. A non-None default here would shadow that config and make it dead.

enable_thinking `class-attribute` `instance-attribute` ¶

enable_thinking = None

Turn this turn's reasoning block on or off, for providers that expose the switch. None defers to the provider's own configuration, and then to the model's default.

reasoning_effort `class-attribute` `instance-attribute` ¶

reasoning_effort = None

Reasoning verbosity for this turn, for providers that grade it. none turns reasoning off (RFC §6.7). A provider whose vendor takes an effort passes the value as the vendor spells it; one that maps it to a vendor setting of its own (Gemini's thinking_level, Ollama's think) reads minimal, low, medium, high and xhigh, sent as the nearest level the model takes. None defers to its config.

response_schema `class-attribute` `instance-attribute` ¶

response_schema = None

JSON Schema the answer must satisfy, within the portable subset of :func:~roomkit.providers.ai.json_schema.check_portable_schema (checked here, on construction and on assignment). generate() then returns one JSON document in content or raises :class:~roomkit.providers.ai.response_schema.ResponseSchemaError; a provider never ignores it. See :attr:AIProvider.supports_response_schema.

model_post_init ¶

model_post_init(__context)

Protect metadata even when Pydantic's validation was bypassed.

model_copy ¶

model_copy(*, update=None, deep=False)

Preserve secret wrapping and the schema check that update= bypasses.

AIMessage ¶

Bases: BaseModel

A message in the AI conversation context.

AITextPart ¶

Bases: BaseModel

Text part of a multimodal message.

AIImagePart ¶

Bases: BaseModel

Image part of a multimodal message.

AITool ¶

Bases: BaseModel

Tool definition for function calling.

A name no vendor accepts (empty, or a character other than a letter, a digit, _, ., : or -) is refused here; a name some vendors refuse is refused by their provider when it declares the tool (RFC §6.7).

AIToolCall ¶

Bases: BaseModel

A tool call from the AI response.

served `class-attribute` `instance-attribute` ¶

served = None

Set by a provider that already ran the call: the channel reports its outcome rather than serving it (RFC §9.3).

partial `class-attribute` `instance-attribute` ¶

partial = False

The call's arguments do not read as an object: the tool loop does not run it and tells the model why (RFC §6.4).

garbled `class-attribute` `instance-attribute` ¶

garbled = False

The model wrote this partial call's arguments unreadable: the response was not cut short over them. A partial call that is not garbled was cut (the output cap or the context window, a content filter or a refusal, a stream that ended without a stop reason), and the model reads which (RFC §6.4).

ServedCall ¶

Bases: BaseModel

What a provider that runs its own tools (an MCP gateway, a sandbox, a proxy) reports of a call it already ran: its result, and whether it failed. Only the provider sets it; the model's arguments never do, so a model that writes a _result key gets an argument like any other, and its call is judged by the gate (RFC §9.3).

AIResponse ¶

Bases: BaseModel

Response from an AI provider.

thinking_parts `class-attribute` `instance-attribute` ¶

thinking_parts = None

The reasoning blocks to replay, in the order the vendor sent them, each with its signature or redacted data (RFC §6.4); None for a provider whose reasoning has no blocks. A block the response cut before its signature is not among them. thinking stays all the reasoning text, thinking_signature the last signature.

ModelInfo ¶

Bases: BaseModel

Metadata describing a single model offered by an AI provider.

Both the curated catalog (:meth:AIProvider.available_models) and the live API query (:meth:AIProvider.list_models) return these. Only id is guaranteed; the remaining fields are best-effort and may be None when the source does not report them.

Attributes:

Name	Type	Description
`id`	`str`	Exact model identifier accepted by the provider's API (e.g. `"claude-sonnet-4-20250514"`, `"gpt-4o"`).
`display_name`	`str \| None`	Human-friendly name (e.g. `"Claude Sonnet 4"`).
`context_window`	`int \| None`	Input context window in tokens, if known.
`supports_vision`	`bool \| None`	Whether the model accepts image input, if known.
`deprecated`	`bool`	Whether the provider marks the model deprecated.
`capabilities`	`list[str]`	Capability tags. In a live listing, the tags the source reports (e.g. Ollama's `"completion"`, `"embedding"`, `"vision"`, `"tools"`), or, where it reports no model type at all, the public tags of the provider's catalog (PolarGrid); and `"transcription"` or `"speech"` on a speech-to-text or text-to-speech model (:mod:`roomkit.providers.ai.model_tags`). A curated catalog (`available_models()`) may carry provider-specific tags here too, internal routing flags among them (`chat_tools_refused`, `deferred_tools`): only `transcription`, `speech` and `image_gen` are defined once, with one meaning across providers. Empty when the source reports none — consumers treat empty as "unknown, allow everywhere" rather than "none".
`pricing`	`ModelPricing \| None`	Vendor list price for this model, if published. It lives here, beside the id, because a lineup and its price list turn over together: kept apart, adding a model leaves its price behind and the consumer bills nothing. `None` where no per-token list price exists — locally pulled open weights, a private edge, a retired id the vendor stopped quoting.

ModelPricing ¶

Bases: BaseModel

List price of one model, per million tokens, as its vendor published it.

Rates mirror the keys roomkit itself reports in usage (:attr:AIResponse.usage) — input, output, cache reads, cache writes — so a consumer can price a response without inventing a mapping. What is not here is deliberate: per-client negotiated rates, discounts and currency conversion belong to whoever bills, not to a shared catalog.

A rate is volatile in a way a model id is not, hence :attr:verified: the entry states what the vendor published on that date, and a consumer can decide for itself when that is too old to trust.

Attributes:

Name	Type	Description
`input_per_million`	`float`	Price of a million uncached input tokens.
`output_per_million`	`float`	Price of a million output tokens.
`cache_read_per_million`	`float \| None`	Price of a million tokens re-read from the prompt cache. `None` means this catalog represents no separate per-token charge for that counter. Catalogs must explicitly repeat the input rate when cache reads are billed as ordinary input.
`cache_write_per_million`	`float \| None`	Price of a million tokens written to the prompt cache — Anthropic's 5-minute write premium (1.25x input), which is the TTL roomkit's `ephemeral` markers request. `None` where a write is not billed per token. Google cache storage, for example, is billed by time and cannot be represented here.
`image_input_per_million`	`float \| None`	Price of a million image input tokens, where the vendor quotes them apart from text — a reference image handed to an image model, say. `None` where the catalog represents no separate charge, which is every conversational model here: they bill a vision token as an ordinary input token.
`image_output_per_million`	`float \| None`	Price of a million generated-image tokens. An image model is billed per token like any other, only with the picture counted on its own meter and at a rate an order of magnitude above the text one — which is why it is a field and not an approximation folded into `output_per_million`. `None` for a model that generates no images.
`long_context_threshold_tokens`	`int \| None`	Total input-token threshold above which the model's published long-context multipliers apply. `None` for models with flat pricing.
`long_context_input_multiplier`	`float`	Multiplier applied to all represented input and cache charges above the long-context threshold.
`long_context_output_multiplier`	`float`	Multiplier applied to output charges above the long-context threshold.
`currency`	`str`	ISO 4217 code the rates are quoted in. Every vendor roomkit ships a catalog for publishes in USD.
`verified`	`date`	Date the rates were read from the vendor's own price list.

cost_for ¶

cost_for(usage)

Price a single response's usage dict, in :attr:currency.

Reads the keys roomkit's providers report — input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens, plus input_image_tokens and output_image_tokens from an image generation (:class:~roomkit.providers.image.base.ImageProvider) — and ignores anything else, so a provider reporting extra counters neither breaks nor inflates the total. Missing keys count as zero.

Every counter is disjoint: a token is charged under exactly one of them. Providers that receive image tokens nested inside a total subtract them before reporting, so summing here bills each token once. reasoning_tokens is not one of them: it is the thinking share of output_tokens, a detail this ignores (RFC §6).

A counter with no corresponding rate is omitted: None means the catalog does not represent a separate per-token charge for it. When the response crosses a published long-context threshold, the configured input and output multipliers are applied automatically.

Parameters:

Name	Type	Description	Default
`usage`	`Mapping[str, int]`	A response's token counters, as reported by the provider.	required

Returns:

Type	Description
`float`	The cost of that response, in :attr:`currency`.

ModelPricing is also exported from the package root. Its rates are finite and non-negative, its multipliers finite and positive, and cost_for() accepts only non-negative integer token counters. Invalid accounting input raises instead of producing a negative or non-finite cost.

MockAIProvider ¶

MockAIProvider(responses=None, *, vision=False, ai_responses=None, streaming=False, tool_call_delta_chunks=0, response_schema=False, response_schema_with_tools=False)

Bases: AIProvider

Round-robin response provider for tests.

With response_schema=True it honours :attr:AIContext.response_schema the way a real provider must (RFC §6.7), streaming included: the scripted answer has to be a JSON document satisfying it, and an :class:AIResponse scripted with finish_reason="refusal", or an ending that ran out of room ("length", "model_length", any :func:~roomkit.providers.ai.tool_calls.is_truncation reads), raises the matching :class:~roomkit.providers.ai.response_schema.ResponseSchemaError.

available_models `classmethod` ¶

available_models()

Fixed two-entry catalog for exercising model-discovery code.

generate_stream `async` ¶

generate_stream(context)

Yield the scripted text as a single delta.

generate_structured_stream ¶

generate_structured_stream(context)

Yield the scripted response as events.

A mock that does not stream is read the way every such provider is, through the default adapter over generate().

Model capability tags¶

A conversational provider's list_models() also lists the speech models its vendor serves. They carry transcription (speech-to-text) or speech (text-to-speech) in ModelInfo.capabilities, read from what the vendor reports or from the model's name, so a model picker can keep them out of a chat list. A curated catalog's own capabilities stay out of a live listing.

model_tags ¶

The capability tags a model listing reads off what a model is.

A chat provider's live listing also surfaces its speech models, and a caller picking a chat model must be able to set them apart. The vendors name them consistently (measured 2026-10-02 on OpenAI, Gemini and Mistral): a speech-to-text model's id says transcribe or whisper, a text-to-speech model's says tts. A model that converses in audio (gpt-audio, gpt-realtime, voxtral-small, Gemini's native audio) is neither and stays untagged.

TRANSCRIPTION_CAPABILITY `module-attribute` ¶

TRANSCRIPTION_CAPABILITY = 'transcription'

ModelInfo.capabilities tag of a speech-to-text model.

SPEECH_CAPABILITY `module-attribute` ¶

SPEECH_CAPABILITY = 'speech'

ModelInfo.capabilities tag of a text-to-speech model.

speech_tags ¶

speech_tags(model_id)

The speech tag model_id names itself by, or none.

with_speech_tags ¶

with_speech_tags(models)

models, each one the listing left untagged given its speech tag.

Response schema¶

AIContext.response_schema constrains the answer of generate() to a JSON Schema within a portable subset. See the Structured Output guide.

check_portable_schema ¶

check_portable_schema(schema)

Refuse a response schema outside the portable subset.

The root must be an object. Every object lists properties, names each of them in required and sets additionalProperties to false; every array has items; only a string may carry enum. title and description are allowed anywhere.

Raises:

Type	Description
`ValueError`	naming the path of the first subschema that is not portable.

schema_mismatch ¶

schema_mismatch(schema, value, path='$')

Where value departs from a portable schema, or None when it fits.

value is a parsed JSON document and schema one that passed :func:check_portable_schema. An object must carry exactly its properties, an integer is a whole number, and a boolean is never taken for a number.

ResponseSchemaError ¶

ResponseSchemaError(message, *, reason, provider='')

Bases: ProviderError

A response schema was set and no answer satisfying it can be returned.

Never retryable: the same request fails the same way.

Attributes:

Name	Type	Description
`reason`	`ResponseSchemaFailure`	`"unsupported"` when the call cannot carry a schema: the provider does not support one, the turn also has tools, or a streaming method received it. Raised before any request is sent. `"refusal"` when the model declined to answer. `"truncated"` when the output cap or the context window cut the document. `"invalid_json"` when the text is not a JSON document satisfying the schema, from a server that accepted the constraint and did not apply it.

Image parts¶

Every provider turns an AIImagePart into what its API takes through one reader: media type from the header, then the part's mime_type, then image/png; a payload an encoder wrapped or left unpadded is repaired; a corrupt one is refused before the request leaves, as a non-retryable ProviderError that names the cause.

image_part_payload ¶

image_part_payload(part, *, provider)

The media type and the decoded bytes of a data: URI image part.

:func:~roomkit.providers.utils.parse_data_uri with the part's own mime_type as the fallback, and its ValueError surfaced as the non-retryable :class:ProviderError an AI provider's caller expects: a caller error, named as such, raised before the request leaves — never a retry or a fallback, since the same URI would fail again. Only for a data: URI; a remote URL is the provider's to forward.

image_part_base64 ¶

image_part_base64(part, *, provider)

The media type and the canonical base64 payload of a data: URI image part.

For an API that takes the two apart — Anthropic's source block, Ollama's images list. Same reading as :func:image_part_payload, re-encoded from the validated bytes.

image_part_uri ¶

image_part_uri(part, *, provider)

The URI an OpenAI-shaped request forwards for an image part.

A remote URL passes through untouched. A data: URI goes through :func:image_part_payload and is rebuilt canonically, so a header without a media type reaches the vendor with the part's, and a payload an encoder wrapped reaches it on one line.

Handing the loop a tool call¶

Every provider hands the tool loop the same thing for the same call (RFC §6.4): arguments as a mapping, never an error; an id no other call of the response carries; and partial=True on a call whose arguments do not read as an object, which the loop answers without running it, whatever stop reason the response gave. The model reads why: garbled=True says the model wrote the arguments unreadable; a partial call that is not garbled was cut (the output cap or the context window, a content filter or a refusal, a stream that stopped without a stop reason). Only a response's last call can be cut: a call another followed was closed by it. A custom provider builds its calls through these rules rather than its own. A realtime provider hands on_tool_call the result of readable_arguments: the mapping, or the model's text for a call whose arguments do not read, which the channel refuses (RFC §12.4). A provider whose wire says whether the response cut a call reads it with realtime_call_arguments instead, which marks a call the cut stopped as CutArguments (RFC §6.4).

tool_arguments ¶

tool_arguments(raw)

A call's arguments as a mapping, never an error.

A mapping passes through. No arguments (nothing, blank text, JSON null) are {}. Text that parses to a JSON object is that object; anything else (invalid JSON, an array, a fragment the output cap cut) is kept whole under raw, and the call is partial (:func:unreadable_arguments).

readable_arguments ¶

readable_arguments(raw)

A call's arguments as a mapping, or the text the model wrote when they do not read as one: how a realtime provider hands a call to on_tool_call, so the channel refuses it rather than run a tool on a mapping that only passes for arguments (RFC §12.4).

realtime_call_arguments ¶

realtime_call_arguments(raw, *, cut)

A realtime call's arguments as its provider hands them to on_tool_call, the response cut over the call or not: a mapping when the call may run, else the model's text, a :class:CutArguments when the cut stopped it (:func:partial_when, the rule every provider applies).

CutArguments ¶

Bases: str

The argument text of a realtime call its response cut before they were whole: the model's text, which the channel refuses as cut off (RFC §6.4, §12.4), where text that merely does not read is refused as unreadable.

unreadable_arguments ¶

unreadable_arguments(raw)

Whether a call's arguments do not read as an object, which makes the call partial: it never runs, whatever the provider and whatever stop reason the response gave (RFC §6.4).

call_partial ¶

call_partial(raw, finish_reason, *, last=True)

Whether a call must not run: its arguments do not read, or it is the response's last call and the response ended on something that stops a call mid-arguments (running out of room, a content filter, a refusal) or on nothing at all, a stream that stopped without a stop reason, before argument text that reads arrived (RFC §6.4).

call_cut ¶

call_cut(raw, finish_reason, *, last=True)

Whether the response was cut short over a call's arguments: it is the response's last call, the response ended on something that stops a call mid-arguments, and they did not arrive whole (:func:call_partial). The call is partial and cut.

call_garbled ¶

call_garbled(raw, finish_reason, *, last=True)

Whether the model wrote a call's arguments unreadable: they do not read, and the response did not cut them short (:func:call_cut). The call is partial and garbled.

CallIds ¶

CallIds()

Hands each call of one response its id.

The server's id when it gave one no earlier call of the response took, a minted one otherwise: results, eviction and cancellation all find a call by its id, so two calls must never share one.

partial_call_error ¶

partial_call_error(name, *, garbled)

What the model reads for a partial call: nothing ran, and why.

tool_call_of ¶

tool_call_of(event)

The tool call a streamed one is, every field it carries kept.

Declaring tools¶

A provider declares the turn's tools in its vendor's format (RFC §6.7). A tool without parameters is declared as an object with none, which every vendor accepts: Anthropic refuses an empty map, and Anthropic and Mistral a declaration without a schema. Tool names differ by vendor, so a provider checks its vendor's rule before the request and raises a ProviderError naming the tool and the rule; a provider in front of a server it does not know (a custom URL: base_url, Mistral's server_url) checks none.

declared_parameters ¶

declared_parameters(parameters)

The schema a tool is declared with: its own, its root an object, or for a tool that takes no parameters an object with none, which every vendor accepts.

An empty map is not one: Anthropic refuses it, and Anthropic and Mistral refuse a declaration without a schema (measured 2026-10-02). A root without a type is an object's, said so for every vendor: Anthropic and OpenAI refuse it untyped (measured 2026-10-03).

chat_tool_declarations ¶

chat_tool_declarations(tools)

The tools as Chat Completions declares them, the shape every OpenAI-compatible server reads.

ToolNameRule `dataclass` ¶

ToolNameRule(vendor, pattern)

The tool names one vendor accepts.

A provider checks the turn's tools against its vendor's rule before the request, so a name the vendor would refuse fails at once with an error naming the tool and the rule, not mid-turn with the vendor's 400.

pattern `instance-attribute` ¶

pattern

The names the vendor accepts, as a regular expression matched whole.

check ¶

check(names)

Raise a non-retryable :class:ProviderError for the first name the vendor does not accept.

Rendering a conversation as Chat Completions messages¶

OpenAI and its derivatives, Mistral and PolarGrid render a conversation through one builder. What a provider renders differently is its ChatDialect: where a model's earlier reasoning goes (inline <think> tags, a message field such as Cerebras's reasoning, or nowhere), whether a tool message names its tool, whether text goes flat. A derivative of OpenAIAIProvider whose service renders differently sets _chat_dialect.

ChatDialect `dataclass` ¶

ChatDialect(thinking_field=None, drops_thinking=False, names_tool_results=False, flattens_text=False, round_thinking_required=False)

What one provider renders differently from OpenAI's own endpoint.

thinking_field `class-attribute` `instance-attribute` ¶

thinking_field = None

The assistant message field a model's earlier reasoning rides (Cerebras reasoning); None sends it inline, as a <think> block.

drops_thinking `class-attribute` `instance-attribute` ¶

drops_thinking = False

Send no earlier reasoning at all: a model that regenerates its own each turn echoes any it is fed back (Qwen behind PolarGrid).

names_tool_results `class-attribute` `instance-attribute` ¶

names_tool_results = False

Each tool message names its tool (Mistral, PolarGrid).

flattens_text `class-attribute` `instance-attribute` ¶

flattens_text = False

A message of text parts goes as one string, and an empty message is not sent; only one carrying an image goes as parts (PolarGrid).

round_thinking_required `class-attribute` `instance-attribute` ¶

round_thinking_required = False

Every round that called tools carries the thinking field, empty when the round did not reason: DeepSeek in thinking mode refuses a round of the turn in progress without its reasoning_content, unless it issued the round's call ids itself a moment ago.

chat_messages ¶

chat_messages(messages, system_prompt, dialect, *, provider)

The conversation as dialect sends it, the system prompt first.

provider names the vendor in the error an image it cannot send raises.

is_truncation ¶

is_truncation(finish_reason)

Whether a response ended by running out of room: its output cap, or its context window filling up mid-answer.

Compared case-insensitively so Gemini's MAX_TOKENS and Anthropic's max_tokens are one entry rather than two.

Explicit models and modern request profiling¶

model= is required by both OpenAIConfig and AnthropicConfig, so a RoomKit upgrade cannot silently change cost, latency, or model behavior. For the model the caller selects, OpenAIConfig profiles first-party gpt-5/o-series models to use max_completion_tokens and omit a custom temperature. AnthropicConfig profiles modern Claude reasoning models to use adaptive thinking and omit temperature. This prevents selected modern model ids from being paired with parameters those models reject.

Explicit compatibility flags always win. A custom base_url also keeps the conservative legacy defaults, since OpenAI-compatible and Anthropic-compatible proxies may implement the older request shape.

The OpenAI provider uses Chat Completions. On OpenAI's own endpoint, what a turn carrying function tools sends of reasoning_effort comes from the model catalogue: the turn's effort for the reasoning models before GPT-5.4, and "none" explicitly from GPT-5.4 on, where it is the only value accepted alongside tools and leaving it out answers 400 on the models that default higher. A model the catalogue does not tag, a custom base_url and Azure send none on such a turn, and log one warning per provider when an effort is left out, unless supports_reasoning_effort_with_tools on the config says whether the server takes it there; a model the catalogue tags follows its tag whatever the field says. Tool-free turns use the turn's effort, else OpenAIConfig.reasoning_effort, else the model default. See Reasoning settings on a turn with tools.

The catalogue also says what Chat Completions refuses a model, and the request fails before it is sent, with a non-retryable ProviderError: GPT-6 Astra and GPT-6.1 Sol take no function tools there (the Responses API serves them with tools), and the -pro models are not chat models there. A custom base_url leaves it to the server.

Per-Room AI Configuration¶

AI channel settings can be overridden per-room using binding metadata:

# Default AI channel
ai = AIChannel("ai", provider=anthropic, system_prompt="Default assistant")
kit.register_channel(ai)

# Override per room
kit.attach_channel(room_id, "ai", metadata={
    "system_prompt": "You are a customer support agent for Acme Corp.",
    "temperature": 0.3,  # More deterministic
    "max_tokens": 2048,
})

Silent Observer Pattern (Meeting Notes)¶

# Attach AI as note-taker
kit.attach_channel(meeting_room_id, "ai", metadata={
    "system_prompt": """You are a meeting note-taker.
    Listen to the conversation silently.
    When someone says 'meeting ended', compile and send a summary.""",
})

# Mute so AI listens but doesn't respond
await kit.mute(meeting_room_id, "ai")

# Later, unmute to let AI send summary
await kit.unmute(meeting_room_id, "ai")

Tools/Function Calling¶

Tools can be passed via binding metadata for function calling:

kit.attach_channel(room_id, "ai", metadata={
    "tools": [
        {
            "name": "search_knowledge_base",
            "description": "Search the company knowledge base",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string"}
                },
                "required": ["query"]
            }
        }
    ]
})

Tool calls are returned in AIResponse.tool_calls:

response = await provider.generate(context)
for tool_call in response.tool_calls:
    print(f"Tool: {tool_call.name}, Args: {tool_call.arguments}")

Gemini Provider¶

GeminiAIProvider ¶

GeminiAIProvider(config)

Bases: AIProvider

AI provider using the Google Gemini API.

supports_vision `property` ¶

supports_vision

All Gemini models support vision.

supports_response_schema `property` ¶

supports_response_schema

Controlled generation, through response_json_schema.

supports_response_schema_with_tools `property` ¶

supports_response_schema_with_tools

Function calling and response_json_schema share a request (verified live on gemini-3.8-flash, 2026-09-27).

available_models `classmethod` ¶

available_models()

Curated, offline catalog of Gemini models.

list_models `async` ¶

list_models()

List generate-content models the Gemini API currently exposes.

Serves both surfaces this provider family speaks to, which name their models differently and describe them unequally:

Developer API (AI Studio): models/gemini-3.5-flash, and each entry declares supported_actions.
Vertex (:class:~roomkit.providers.gemini.vertex.GeminiVertexProvider): publishers/google/models/gemini-2.5-flash, with no supported_actions and no metadata at all.

So the id is the last path segment rather than one stripped prefix — a prefixed id matches nothing in the curated catalog, which silently emptied the metadata, and would be written to a caller's config as a model name the API then rejects. And where no action is declared, the listing mixes in models this call cannot serve, which the family and embedding checks drop without hiding a model too new to be curated.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events from the Gemini streaming API.

A response schema is checked before the done event (RFC §6.7).

generate `async` ¶

generate(context)

Generate by consuming the structured stream.

generate_stream `async` ¶

generate_stream(context)

Yield text deltas as they arrive from the Gemini API.

close `async` ¶

close()

Close the SDK and the httpx client it was given.

GeminiConfig ¶

Bases: BaseModel

Google Gemini AI provider configuration.

thinking_level `class-attribute` `instance-attribute` ¶

thinking_level = None

Thinking level for Gemini 3 models: minimal, low, medium, high. gemini-3.8-flash, the default, refuses minimal with a 400: low is its lowest level.

timeout `class-attribute` `instance-attribute` ¶

timeout = 60.0

Read budget in seconds: how long the first chunk, and each one after it, may take. Generation streams, so a stalled answer fails here instead of holding the turn open.

connect_timeout `class-attribute` `instance-attribute` ¶

connect_timeout = 5.0

TCP connect timeout in seconds, kept apart from timeout so a host that no longer accepts connections is given up on in seconds rather than after the read budget. The default across RoomKit's providers.

GeminiVertexProvider ¶

GeminiVertexProvider(config)

Bases: GeminiAIProvider

Gemini provider backed by Vertex AI in a specific Google Cloud region.

Subclasses :class:GeminiAIProvider — only client construction (Vertex mode, and an identity that is not an API key) and the billing labels on each request differ. All generation, streaming, thinking, and model discovery are inherited.

GeminiVertexConfig ¶

Bases: GeminiConfig

Gemini-on-Vertex configuration.

Subclasses :class:GeminiConfig, inheriting every generation field (model, max_tokens, thinking_level) so the two cannot drift. There is no API key on Vertex: the caller is authenticated either by an explicit service-account key or, failing that, by Application Default Credentials — the standard Google chain (gcloud auth application-default login, GOOGLE_APPLICATION_CREDENTIALS, workload identity).

api_key `class-attribute` `instance-attribute` ¶

api_key = None

Optional and unused on Vertex — the identity comes from service_account_json or from ADC, never from a key on the request.

project `instance-attribute` ¶

project

Google Cloud project id that hosts the Vertex AI API.

location `instance-attribute` ¶

location

Vertex region — required, no default. Pin it to keep data in-region for residency (e.g. "northamerica-northeast1" for Montréal, "europe-west1"). A default like "global" could route out of region and defeat the whole point, so the choice is made explicit.

service_account_json `class-attribute` `instance-attribute` ¶

service_account_json = None

A service-account key file's contents, as JSON, authenticating as that account instead of as the process.

ADC answers "who is this machine", which is the wrong question wherever one deployment serves several projects: the ambient identity belongs to whoever runs the server, so a caller naming someone else's project gets PERMISSION_DENIED no matter what it puts in project. Passing a key here makes the identity travel with the configuration, which is what lets one process serve one project per tenant.

None keeps the ADC chain, which stays right for a single-project deployment and for local development.

impersonate_service_account `class-attribute` `instance-attribute` ¶

impersonate_service_account = None

Email of a service account to borrow instead of holding its key.

The identity a caller needs and the secret it holds are separate problems, and organizations increasingly forbid the second: Google enforces constraints/iam.disableServiceAccountKeyCreation by default on recent organizations, so a project owner cannot hand out a key even when willing. Here the owner instead grants this deployment's own identity roles/iam.serviceAccountTokenCreator on one of their service accounts, and Vertex is called as that account with short-lived tokens nobody ever downloads — revocable from their side, in one command, without telling us.

Combines with the fields above rather than replacing them: the borrowing identity is service_account_json when set, otherwise ADC.

labels `class-attribute` `instance-attribute` ¶

labels = None

Per-request labels for Google's billing report, e.g. {"tenant": "acme"}.

Vertex attaches them to every generateContent call and Cloud Billing groups the charges by them, which is how one project's Gemini spend is attributed to the tenants or partners it serves. Metadata only: no quota, no limit, no effect on the answer. The report runs 24 to 48 hours behind, so it is never a source of truth for what a caller consumed — meter that from the usage each response reports.

Constant for the life of the provider, like project: a provider serves one channel, a channel one agent, an agent one tenant. Google's label rules are enforced here so a bad label fails at configuration rather than on the first request: at most 64 labels; keys 1 to 63 characters starting with a lowercase letter, values 0 to 63; lowercase letters, digits, _ and - only (international characters allowed). The Gemini Developer API refuses the field outright, which is why it lives on this config alone.

Usage¶

from roomkit.providers.gemini.ai import GeminiAIProvider
from roomkit.providers.gemini.config import GeminiConfig

config = GeminiConfig(api_key="your-api-key")
provider = GeminiAIProvider(config)

# Use with AIChannel
ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[gemini]

A tool result whose call was refused, failed, blocked, served by nothing or cancelled goes to Gemini (and Vertex) under the error key of its function response, any other under result; the text the model reads is the same either way. Gemini Live does not flag it yet.

Schema Cleaning¶

Gemini rejects extra JSON Schema fields common in MCP/OpenAPI tool definitions. RoomKit auto-cleans schemas when building FunctionDeclaration objects:

clean_gemini_schema ¶

clean_gemini_schema(schema)

Recursively prepare a JSON Schema for Gemini FunctionDeclaration.

Performs four passes per node:

Fold what Gemini cannot say. A type list and a union become one type: {"anyOf": [{...}, {"type": "null"}]} and {"type": [..., "null"]} become {..., "nullable": true}. A wider anyOf / oneOf keeps its first non-null branch, an allOf gathers the fields of all of them. A union that only narrows its node (see :func:_refines_node) is not folded; the node keeps the fields its branches add (see :func:_keep_node).
Spell out the shape JSON Schema leaves implied: the type an untyped node's properties or items imply, items on an array, and no key the type cannot carry (see :func:_complete_shape).
Describe a non-string enum, which Gemini cannot hold (see :func:_describe_enum).
Strip unknown keys. Anything not in :data:_GEMINI_ALLOWED_KEYS (e.g. $schema, additionalProperties, default, title) is dropped at every nesting level, and required keeps only the properties that survived.

Parameters:

Name	Type	Description	Default
`schema`	`dict[str, Any] \| None`	A JSON Schema dict (e.g. tool parameter schema), or None.	required

Returns:

Type	Description
`dict[str, Any] \| None`	A cleaned copy, or None if input was None.

Meta Muse Spark Provider¶

MetaAIProvider ¶

MetaAIProvider(config, *, transport=None)

Bases: OpenAIAIProvider

AI provider on Meta's Muse Spark and Muse Wonder, through its OpenAI-compatible Chat Completions.

Subclasses :class:~roomkit.providers.openai.ai.OpenAIAIProvider: message building, tool calls, streaming, usage (reasoning and cached tokens included) and error mapping are inherited. What is Meta's own:

Muse Spark and Muse Wonder always reason, so reasoning_effort rides every request, tool turns included — it is the only lever over the cost and latency of a turn. "none" is refused by the service and sent as "minimal" (3.0 s against 4.6 s at "low" on a one-line answer, 2026-09-27).
/v1/models lists the account's image, speech and segmentation models too; :meth:list_models keeps the chat ones.

Example::

provider = MetaAIProvider(MetaConfig(api_key="...", reasoning_effort="low"))

supports_vision `property` ¶

supports_vision

Every Muse chat model reads images, a newer id than the catalog too.

available_models `classmethod` ¶

available_models()

Curated, offline catalog of the Muse chat models.

list_models `async` ¶

list_models()

The chat models the account can use, with the catalog's metadata.

MetaConfig ¶

Bases: OpenAIConfig

Meta Muse Spark chat provider configuration.

The Meta Model API serves an OpenAI-compatible Chat Completions API at https://api.meta.ai/v1, so this subclasses :class:OpenAIConfig and inherits every request field the inherited provider reads (reasoning_effort, extra_body …). Only the endpoint, the model and two defaults change.

reasoning_effort takes minimal, low, medium, high or xhigh, and Muse Wonder max too. Every Muse chat model always reasons: "none" is refused by the service and sent as "minimal", its lightest effort.

The -contributor model ids cost a fraction of the standard ones because Meta trains its models on their prompts and completions. They are never a default; choose one knowingly.

base_url `class-attribute` `instance-attribute` ¶

base_url = 'https://api.meta.ai/v1'

Meta Model API endpoint. Override only to point at a proxy.

model `class-attribute` `instance-attribute` ¶

model = 'muse-spark-1.3'

Muse Spark or Muse Wonder model id — see :mod:roomkit.providers.meta.models.

use_max_completion_tokens `class-attribute` `instance-attribute` ¶

use_max_completion_tokens = True

Meta documents max_completion_tokens for Chat Completions.

include_stream_usage `class-attribute` `instance-attribute` ¶

include_stream_usage = True

Meta reports usage on the last streamed chunk, reasoning tokens included.

Muse Spark cannot turn reasoning off: reasoning_effort="none" is sent as "minimal". See the Meta Model API guide.

vLLM Provider (Local LLM)¶

VLLMConfig ¶

Bases: BaseModel

Configuration for a local vLLM server.

vLLM exposes an OpenAI-compatible API, so this config is translated into an OpenAIConfig by :func:create_vllm_provider.

Attributes:

Name	Type	Description
`model`	`str`	Model name loaded by the vLLM server (required).
`base_url`	`str`	Base URL of the vLLM OpenAI-compatible endpoint.
`api_key`	`SecretStr`	Bearer token sent as `Authorization: Bearer <key>`. Matches `vllm serve --api-key`; default `"none"` for the common no-auth local server.
`max_tokens`	`int`	Maximum tokens in the response.
`timeout`	`float`	HTTP request timeout in seconds. Increase for vLLM servers that load models lazily on first request.
`connect_timeout`	`float`	TCP connect timeout in seconds, kept apart from `timeout` so a host that no longer accepts connections is given up on in seconds rather than after the read budget.
`headers`	`dict[str, str] \| None`	Extra HTTP headers on every request — for a reverse proxy that needs custom headers, or a non-Bearer `Authorization` scheme. Maps to `OpenAIConfig.default_headers`.
`top_p`	`float \| None`	Nucleus sampling cutoff. `None` leaves the server's default.
`top_k`	`int \| None`	Top-k sampling cutoff — a vLLM extension, not an OpenAI field. `None` leaves the server's default.
`min_p`	`float \| None`	Minimum-probability sampling cutoff — a vLLM extension. `None` leaves the server's default.
`presence_penalty`	`float \| None`	Penalty on tokens already present in the output. The knob Qwen's own guidance raises (to `1.5`) for non-thinking mode, where the failure it addresses is degenerate repetition. `None` leaves the server's default.
`repetition_penalty`	`float \| None`	Multiplicative repetition penalty — a vLLM extension, distinct from `presence_penalty`. `None` leaves the server's default.
`extra_body`	`dict[str, Any] \| None`	Extra JSON fields merged into every request body — the route for vLLM params this config does not model (`guided_json`/`guided_choice` guided decoding, and any sampler added by a newer server). Maps to `OpenAIConfig.extra_body`, and an entry here wins over the typed fields above.
`enable_thinking`	`bool \| None`	Turn the model's reasoning block on or off. `None` leaves the model's own default, which for current Qwen builds is on at the most verbose effort — reasoning then competes with the answer for `max_tokens` and can consume the whole budget, leaving an empty `content`. Set `False` for tool loops that only need the final answer.
`reasoning_effort`	`str \| None`	Reasoning verbosity when thinking is on — `"low"`, `"medium"` or `"xhigh"`. Accepted values depend on the served model's chat template; vLLM raises a template error on an unknown one.

max_retries `class-attribute` `instance-attribute` ¶

max_retries = 0

SDK-level retry count. Default 0 because RoomKit's RetryPolicy handles retries at the right layer with proper backoff and fallback.

include_stream_usage `class-attribute` `instance-attribute` ¶

include_stream_usage = True

Request token usage in streaming responses; without it a streamed turn reports no usage. Set False for a server that rejects stream_options.

headers `class-attribute` `instance-attribute` ¶

headers = None

Extra HTTP headers sent on every request (proxy headers, non-Bearer auth). None sends only the SDK defaults.

top_p `class-attribute` `instance-attribute` ¶

top_p = None

Nucleus sampling cutoff. None leaves the server's default.

top_k `class-attribute` `instance-attribute` ¶

top_k = None

Top-k sampling cutoff (vLLM extension). None leaves the default.

min_p `class-attribute` `instance-attribute` ¶

min_p = None

Minimum-probability cutoff (vLLM extension). None leaves the default.

presence_penalty `class-attribute` `instance-attribute` ¶

presence_penalty = None

Penalty on already-present tokens. None leaves the default.

repetition_penalty `class-attribute` `instance-attribute` ¶

repetition_penalty = None

Multiplicative repetition penalty (vLLM extension). None leaves the default.

extra_body `class-attribute` `instance-attribute` ¶

extra_body = None

Extra request-body fields for vLLM params this config does not model (guided decoding, a newer server's sampler). None sends a vanilla body; an entry here wins over the typed sampling fields.

enable_thinking `class-attribute` `instance-attribute` ¶

enable_thinking = None

Reasoning block on/off. None leaves the model's own default.

reasoning_effort `class-attribute` `instance-attribute` ¶

reasoning_effort = None

Reasoning verbosity when thinking is on ("low"/"medium"/"xhigh").

supports_response_schema `class-attribute` `instance-attribute` ¶

supports_response_schema = None

Whether the server honours a json_schema response format. None keeps the default (vLLM applies it through guided decoding); set False for a server that does not.

supports_response_schema_with_tools `class-attribute` `instance-attribute` ¶

supports_response_schema_with_tools = None

Whether a schema may ride a turn with tools. None keeps it off: guided decoding forces the output into the schema, so the model cannot call a tool under it.

sampling_body ¶

sampling_body()

The request-body fields implied by the sampling settings.

All of them ride the body rather than the SDK's named parameters: top_k, min_p and repetition_penalty are vLLM extensions the OpenAI SDK has no argument for, and top_p/presence_penalty are read from the same body by the server, so routing the five through one place keeps the split invisible to the caller.

Only the knobs actually set are emitted — None means "the server decides", which is not the same as sending its documented default and is the only honest answer for a server whose model we cannot see. Tested with is not None so an explicit 0 survives: min_p=0 and presence_penalty=0 are meaningful values, not absences.

chat_template_kwargs ¶

chat_template_kwargs()

The chat_template_kwargs implied by the reasoning settings.

vLLM renders the model's own chat template server-side, so reasoning is steered through template kwargs rather than a sampling parameter. Empty when neither knob is set, so a vanilla body stays vanilla.

create_vllm_provider ¶

create_vllm_provider(config, *, transport=None)

Create an OpenAI-compatible AI provider pointed at a local vLLM server.

The returned provider is an :class:OpenAIAIProvider subclass: identical on the wire, but reporting model metadata for your server rather than OpenAI's hosted catalog. The openai SDK is imported lazily, when the provider is instantiated.

Note

available_models() is empty by design — nothing offline can know what a local server loaded. Use :meth:~OpenAIAIProvider.list_models, which queries the server's /v1/models endpoint.

Parameters:

Name	Type	Description	Default
`config`	`VLLMConfig`	vLLM connection settings.	required
`transport`	`AsyncBaseTransport \| None`	Carries every request, as for :class:`OpenAIAIProvider`.	`None`

Returns:

Type	Description
`OpenAIAIProvider`	A provider configured for the local vLLM server.

Usage¶

from roomkit.providers.vllm import create_vllm_provider, VLLMConfig
from roomkit.channels.ai import AIChannel

# Configure connection to a local vLLM server
config = VLLMConfig(
    model="meta-llama/Llama-3.1-8B-Instruct",
    base_url="http://localhost:8000/v1",
)

# Factory returns an OpenAIAIProvider pointed at your vLLM server
provider = create_vllm_provider(config)

# Use with AIChannel like any other AI provider
ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[vllm]

llama.cpp Provider (Local LLM, nothing to run beside it)¶

RoomKit downloads the llama.cpp build for the machine and the model, runs llama-server, and stops it on close(). See the llama.cpp guide.

LlamaCppConfig ¶

Bases: BaseModel

A local model that RoomKit runs itself, through llama.cpp's llama-server.

Only model is required. On first use the provider downloads the llama.cpp build for this machine (checked against a pinned SHA-256) and the model, starts llama-server on a free local port, and stops it when the provider is closed.

Attributes:

Name	Type	Description
`model`	`str`	The GGUF model to run: a Hugging Face `repo:quant` reference (`"unsloth/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M"`), which `llama-server` downloads and caches itself, or the path of a local `.gguf` file.
`context_size`	`int`	Context window in tokens (`-c`). Tool definitions and tool results count against it.
`gpu_layers`	`int \| None`	Layers offloaded to the GPU (`-ngl`). `None` lets llama.cpp fit as many as the GPU holds; `0` runs on the CPU only.
`max_tokens`	`int`	Maximum tokens in one response.
`enable_thinking`	`bool \| None`	Turn a reasoning model's thinking on or off through its chat template. `None` keeps the model's default.
`binary`	`str \| None`	A `llama-server` to use instead of the pinned download (your own build, a distribution package): a path, or a name looked up on the `PATH`. A `llama-server` merely present on the machine is never picked up on its own.
`variant`	`str \| None`	Force a download variant (`"linux-x64-cpu"`, `"linux-x64-cuda-13"`, `"macos-arm64"`…) instead of the one detected for this machine.
`cache_dir`	`str \| None`	Where downloaded llama.cpp builds are kept. Defaults to `~/.cache/roomkit/llama.cpp`.
`port`	`int \| None`	Local port for `llama-server`. `None` picks a free one.
`startup_timeout`	`float`	Seconds to wait for the server to be ready. The first start downloads the model, so the default is generous.
`timeout`	`float`	HTTP timeout of one request, in seconds.
`extra_args`	`list[str]`	More `llama-server` arguments, passed as given (`["--threads", "8"]`).

LlamaCppAIProvider ¶

LlamaCppAIProvider(config)

Bases: _VLLMProvider

Run a GGUF model locally through llama.cpp, with nothing to install or start.

The first request (or :meth:start) downloads the pinned llama.cpp build for this machine and the model, then starts llama-server on a free local port; :meth:close stops it. Everything else — tool calls in the model's native format (--jinja), streaming, enable_thinking — is the OpenAI-compatible path RoomKit already uses for vLLM.

Example::

from roomkit.providers.llamacpp import LlamaCppAIProvider, LlamaCppConfig

ai = LlamaCppAIProvider(
    LlamaCppConfig(model="unsloth/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M")
)

base_url `property` ¶

base_url

The local OpenAI-compatible endpoint of the managed llama-server.

start `async` ¶

start()

Download what is missing and start the server now, not on the first request.

Call it at startup so the first user turn does not wait for the model to load. Safe to call more than once.

close `async` ¶

close()

Close the HTTP client and stop llama-server.

Usage¶

from roomkit.providers.llamacpp import LlamaCppAIProvider, LlamaCppConfig

provider = LlamaCppAIProvider(
    LlamaCppConfig(model="unsloth/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M")
)
await provider.start()  # optional: load the model before the first request

Install with: pip install roomkit[llamacpp]

Anthropic Provider¶

AnthropicAIProvider ¶

AnthropicAIProvider(config)

Bases: AIProvider

AI provider using the Anthropic Messages API.

supports_deferred_tools `property` ¶

supports_deferred_tools

Read from the catalogue: a model it does not carry is assumed not to.

Never behind another base_url: a proxy or gateway gets the request shape it always got, as the configuration's other shape defaults leave it.

supports_vision `property` ¶

supports_vision

Whether the configured Claude model accepts image input.

Read from the offline catalog, which states it per model, rather than from a prefix table that has to be remembered on every release — the prefix form silently reported the whole 4.5-and-later lineup as text-only, dropping images before they reached the wire.

supports_response_schema `property` ¶

supports_response_schema

Structured outputs, through output_config.format.

A model older than the feature answers 400, surfaced as a :class:ProviderError.

supports_response_schema_with_tools `property` ¶

supports_response_schema_with_tools

output_config.format and tools share a request: the model calls tools or answers in the schema.

available_models `classmethod` ¶

available_models()

Curated, offline catalog of Claude models.

list_models `async` ¶

list_models()

List models the Anthropic API currently exposes for this key.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events from the Anthropic Messages streaming API.

When extended thinking is enabled, yields StreamThinkingDelta events before text deltas. A response schema is checked before the done event (RFC §6.7).

generate `async` ¶

generate(context)

Generate by consuming the structured stream.

generate_stream `async` ¶

generate_stream(context)

Yield text deltas as they arrive from the Anthropic Messages API.

close `async` ¶

close()

Close the configured client and every per-request one still cached.

AnthropicConfig ¶

Bases: BaseModel

Anthropic AI provider configuration.

model `instance-attribute` ¶

model

Model identifier. Required so upgrading RoomKit cannot silently change a caller's model, cost, latency, or behavior.

timeout `class-attribute` `instance-attribute` ¶

timeout = 60.0

Request timeout in seconds (default 60s).

connect_timeout `class-attribute` `instance-attribute` ¶

connect_timeout = 5.0

TCP connect timeout in seconds, kept apart from timeout so a host that no longer accepts connections is given up on in seconds rather than after the read budget. The SDK's own default.

max_retries `class-attribute` `instance-attribute` ¶

max_retries = 0

SDK-level retry count. Default 0 because RoomKit's RetryPolicy handles retries at the right layer with proper backoff and fallback; the SDK's own retries would multiply its attempts (RMK-509).

base_url `class-attribute` `instance-attribute` ¶

base_url = None

Override the base URL (e.g., for Claude Code sandbox proxy).

The SDK appends /v1/messages to this, so a value that already ends in that path is dropped down to its parent: Microsoft Foundry documents the Claude surface as the whole <resource>/anthropic/v1/messages URL, and pasting it verbatim would otherwise post to /v1/messages/v1/messages. A bare trailing /v1 is deliberately left alone — unlike the full path it is not unambiguously wrong, and a gateway may route on it.

extra_headers `class-attribute` `instance-attribute` ¶

extra_headers = None

Extra headers sent with every request (e.g., X-Tenant-ID).

enable_prompt_caching `class-attribute` `instance-attribute` ¶

enable_prompt_caching = True

Apply Anthropic prompt caching (explicit cache_control markers) to the stable request prefix — tools, system prompt, and the conversation suffix. Every tool-loop round re-sends the full context; without markers it is billed at the full input rate on every round, with them the prefix re-reads at the cached rate (10%). Disable for proxies that reject cache_control blocks.

use_adaptive_thinking `class-attribute` `instance-attribute` ¶

use_adaptive_thinking = False

Send extended thinking as {"type": "adaptive"} instead of the deprecated {"type": "enabled", "budget_tokens": N}. Anthropic's newer models reject budget_tokens with HTTP 400. Official modern models are profiled automatically; an explicit value or custom base_url is left untouched for compatibility with proxies and older deployments.

supports_custom_temperature `class-attribute` `instance-attribute` ¶

supports_custom_temperature = True

When False, temperature is omitted from requests. Anthropic's modern reasoning models removed the sampling parameters and reject temperature with HTTP 400. Official modern models are profiled automatically unless this field is explicitly set.

model_post_init ¶

model_post_init(__context)

Apply safe defaults for Anthropic's modern first-party models.

A tool result whose call was refused, failed, blocked, served by nothing or cancelled goes to Anthropic with is_error set on its tool_result block; the text the model reads is the same as on any other provider.

Usage¶

from roomkit.providers.anthropic.ai import AnthropicAIProvider
from roomkit.providers.anthropic.config import AnthropicConfig
from roomkit.channels.ai import AIChannel

config = AnthropicConfig(api_key="your-api-key", model="claude-opus-5")
provider = AnthropicAIProvider(config)

ai_channel = AIChannel("ai", provider=provider)

Per-request credentials¶

The configured key remains the default. In a multi-tenant host where a caller uses their own Anthropic subscription, a BEFORE_AI_GENERATION hook can select that credential for one turn without rebuilding the shared provider:

from roomkit import HookResult, HookTrigger
from roomkit.providers.ai import API_KEY_METADATA_KEY

@kit.hook(HookTrigger.BEFORE_AI_GENERATION)
async def select_anthropic_key(event, ctx):
    key = await tenant_secrets.anthropic_key(ctx.room.metadata["tenant_id"])
    event.ai_context.metadata[API_KEY_METADATA_KEY] = key
    return HookResult.allow()

RoomKit stores the value as a Pydantic secret, so rendering or serializing the context redacts it. An absent, empty, or non-string value — or one equal to the configured key — falls back to AnthropicConfig.api_key and its shared client.

Per-key clients are cached in a pool bounded by a soft limit: only entries whose last turn has finished are evicted and closed. A burst of distinct credentials may hold the pool briefly above that limit, and it trims itself as those turns end, because closing a client underneath an in-flight stream would break a response that has nothing to do with the new caller.

Install with: pip install roomkit[anthropic]

OpenAI Provider¶

OpenAIAIProvider ¶

OpenAIAIProvider(config, *, transport=None)

Bases: AIProvider

AI provider using the OpenAI Chat Completions API.

transport carries every request the SDK makes, inside the SDK's own default client, which keeps following redirects (each hop through the transport) and applies the per-request timeout: the seam for an outbound policy that judges the address actually dialled, or a MockTransport in a test. The transport owns its connection pool and limits, and environment proxies are not read (httpx reads none beside a transport). It is closed with the provider. None leaves the SDK its default client.

supports_vision `property` ¶

supports_vision

Whether the configured model accepts image input.

Read from the offline catalog, which states it per model, rather than from a prefix table that has to be remembered on every release — the prefix form predated GPT-5 entirely and silently reported the whole current lineup as text-only, dropping images before they reached the wire.

supports_response_schema `property` ¶

supports_response_schema

The config's answer when it gives one, else this endpoint's default.

An OpenAI-compatible server decides for itself whether it applies a json_schema response format, so the host that knows the server states it on the config rather than the class guessing.

supports_response_schema_with_tools `property` ¶

supports_response_schema_with_tools

OpenAI's own endpoint takes a strict json_schema format beside function tools. Behind a base_url it depends on the server, and a grammar-constrained one (vLLM, llama.cpp) cannot call a tool under the constraint, so it is off unless the config says otherwise.

available_models `classmethod` ¶

available_models()

Curated, offline catalog of OpenAI chat/multimodal models.

list_models `async` ¶

list_models()

List every model id the configured endpoint exposes.

The OpenAI /v1/models response carries only ids — metadata for known chat models is backfilled from the curated catalog. The raw list also includes non-chat models (embeddings, audio); they pass through unfiltered since the endpoint reports no capability field.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events with <think> tag parsing.

Text inside <think>...</think> is yielded as :class:StreamThinkingDelta; everything else as :class:StreamTextDelta. Tool calls are collected from the final chunks and yielded as :class:StreamToolCall. A response schema is checked before the done event (RFC §6.7).

generate_stream `async` ¶

generate_stream(context)

Yield text deltas (thinking content filtered out).

close `async` ¶

close()

Close the underlying HTTP client.

OpenAIConfig ¶

Bases: BaseModel

OpenAI AI provider configuration.

Attributes:

Name	Type	Description
`api_key`	`SecretStr`	API key for authentication.
`base_url`	`str \| None`	Custom base URL for OpenAI-compatible APIs (e.g., Ollama, LM Studio, Azure OpenAI, or other providers). If None, uses the default OpenAI API.
`model`	`str`	Model identifier to use.
`max_tokens`	`int`	Maximum tokens in the response.

model `instance-attribute` ¶

model

Model identifier. Required so upgrading RoomKit cannot silently change a caller's model, cost, latency, or behavior.

timeout `class-attribute` `instance-attribute` ¶

timeout = 30.0

HTTP request timeout in seconds. Override for servers that need longer (e.g. Ollama cold-starting a model on first request).

connect_timeout `class-attribute` `instance-attribute` ¶

connect_timeout = 5.0

TCP connect timeout in seconds, kept apart from timeout so a host that no longer accepts connections is given up on in seconds rather than after the read budget. The SDK's own default.

max_retries `class-attribute` `instance-attribute` ¶

max_retries = 0

SDK-level retry count. Default 0 because RoomKit's RetryPolicy handles retries at the right layer with proper backoff and fallback.

include_stream_usage `class-attribute` `instance-attribute` ¶

include_stream_usage = True

Request token usage in streaming responses via stream_options.include_usage; it is included in the final :class:StreamDone event. Every tool round streams, so without it a turn reports no usage and prices at zero. Set False for an OpenAI-compatible server that rejects stream_options.

use_max_completion_tokens `class-attribute` `instance-attribute` ¶

use_max_completion_tokens = False

Send the output cap as max_completion_tokens instead of the deprecated max_tokens. OpenAI's newer models (o-series, gpt-5, gpt-4.1) reject max_tokens outright. Leave False for OpenAI-compatible servers (vLLM, LM Studio, older Azure deployments) that only understand max_tokens. Official modern models are profiled automatically unless this field is explicitly set.

supports_custom_temperature `class-attribute` `instance-attribute` ¶

supports_custom_temperature = True

When False, temperature is omitted from requests. OpenAI's reasoning models (o-series, gpt-5) accept only the default temperature=1 and reject any other value with HTTP 400.

reasoning_effort `class-attribute` `instance-attribute` ¶

reasoning_effort = None

Reasoning depth for OpenAI reasoning models (o-series, gpt-5): "none" | "low" | "medium" | "high" | "xhigh" | "max" (availability varies by model). Controls how long the model reasons (quality vs latency/cost); the reasoning trace itself stays hidden in the Chat Completions API. None = the model's default. The turn's own effort outranks this one. On a turn with tools the model catalogue decides what is sent (RFC §6.7): the effort up to GPT-5.2, "none" instead from GPT-5.4 on, and nothing at all for a model it does not tag or one behind base_url, unless supports_reasoning_effort_with_tools says otherwise. Only configure this for reasoning models — others reject the parameter.

default_headers `class-attribute` `instance-attribute` ¶

default_headers = None

Extra HTTP headers sent on every request, passed to the SDK's default_headers. Use for an OpenAI-compatible endpoint behind a reverse proxy that needs custom headers, or a non-Bearer Authorization scheme (e.g. Basic). None sends only the SDK's own headers; the api_key Bearer token is unaffected.

extra_body `class-attribute` `instance-attribute` ¶

extra_body = None

Extra JSON fields merged into every Chat Completions request body via the SDK's extra_body. The route for server-specific params the OpenAI schema omits — e.g. vLLM guided decoding (guided_json/guided_choice) and extra sampling (top_k, repetition_penalty, min_p). None sends a vanilla body.

supports_response_schema `class-attribute` `instance-attribute` ¶

supports_response_schema = None

Whether the server honours a json_schema response format (RFC §6.7). None keeps the provider's default; set it for a server behind base_url that differs, so a turn carrying a response schema is refused up front instead of answered in prose.

supports_response_schema_with_tools `class-attribute` `instance-attribute` ¶

supports_response_schema_with_tools = None

Whether the server takes a json_schema response format beside function tools and still lets the model call them. None keeps the provider's default (on for OpenAI's own endpoint, off behind a base_url).

supports_reasoning_effort_with_tools `class-attribute` `instance-attribute` ¶

supports_reasoning_effort_with_tools = None

Whether the server takes reasoning_effort beside function tools (RFC §6.7), for a model the provider cannot know: one behind a base_url, or on OpenAI's own endpoint one the catalogue does not tag. True sends the turn's effort on a turn with tools as on any other; False leaves it out. None leaves it out too, with a warning logged once when an effort is left out. A model the catalogue tags follows its tag whatever this says. Read by the OpenAI and LiteLLM providers; the configs of the derivatives that know their vendor's rule (Meta, Cerebras, xAI, OpenRouter, DeepSeek, Qwen) refuse it.

model_post_init ¶

model_post_init(__context)

Apply safe defaults for modern models on OpenAI's own endpoint.

Usage¶

from roomkit.providers.openai.ai import OpenAIAIProvider
from roomkit.providers.openai.config import OpenAIConfig
from roomkit.channels.ai import AIChannel

config = OpenAIConfig(api_key="your-api-key", model="gpt-5.6-sol")
provider = OpenAIAIProvider(config)

ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[openai]

Cerebras Provider¶

CerebrasAIProvider ¶

CerebrasAIProvider(config, *, transport=None)

Bases: OpenAIAIProvider

Cerebras chat, reasoning and tool calling, including streaming.

Reuses OpenAI's async client and RoomKit's response decoder, error mapping, token accounting and live /v1/models discovery. Cerebras-specific request parameters and historical reasoning are shaped here.

Example::

provider = CerebrasAIProvider(
    CerebrasConfig(api_key="...", model="gpt-oss-120b")
)

name `property` ¶

name

Stable provider name in streaming and non-streaming telemetry.

supports_vision `property` ¶

supports_vision

Report the configured model's capability; unknown ids default False.

available_models `classmethod` ¶

available_models()

Offline metadata; use list_models() for account availability.

CerebrasConfig ¶

Bases: OpenAIConfig

Configuration for Cerebras's OpenAI-compatible Chat Completions API.

Inherits connection settings, sampling, retries and extra_body from :class:OpenAIConfig. model is required; upgrading RoomKit never silently selects another model. Install with roomkit[cerebras].

base_url `class-attribute` `instance-attribute` ¶

base_url = 'https://api.cerebras.ai/v1'

Cerebras endpoint. May be overridden for a dedicated endpoint or proxy.

use_max_completion_tokens `class-attribute` `instance-attribute` ¶

use_max_completion_tokens = True

The output cap includes both reasoning and final-answer tokens.

include_stream_usage `class-attribute` `instance-attribute` ¶

include_stream_usage = False

Cerebras sends usage in the final chunk without stream_options. Leave False for Cerebras; the inherited decoder still collects usage.

reasoning_effort `class-attribute` `instance-attribute` ¶

reasoning_effort = None

Reasoning effort, also sent on tool turns. Supported values depend on the model: GPT OSS accepts low/medium/high; Qwen 3.8 and Gemma 4 also accept none. A per-turn AIContext.reasoning_effort takes precedence. Token-based thinking_budget is not mapped to an effort level.

reasoning_format `class-attribute` `instance-attribute` ¶

reasoning_format = 'parsed'

Use parsed reasoning to keep thinking separate from user-visible text. Raw output cannot always be separated (GPT OSS concatenates it without delimiters). Supported formats vary by model; None omits the parameter.

clear_thinking `class-attribute` `instance-attribute` ¶

clear_thinking = None

Whether to remove historical reasoning before prompting Qwen 3.8. None leaves the server default. Only set for models supporting this field.

Install with pip install "roomkit[cerebras]". The provider uses the shared OpenAI-compatible async transport and supports text, streaming, reasoning, function calls and image input on vision-capable models.

import os

from roomkit import AIChannel, CerebrasAIProvider, CerebrasConfig

provider = CerebrasAIProvider(CerebrasConfig(
    api_key=os.environ["CEREBRAS_API_KEY"],
    model="gpt-oss-120b",
    reasoning_effort="low",
))
ai = AIChannel("assistant", provider=provider)

Choose model explicitly. await provider.list_models() queries the models available to the account; CerebrasAIProvider.available_models() supplies offline capabilities, context limits and dated prices. The snapshot includes GPT OSS 120B, Qwen 3.8 27B and Gemma 4 31B; Gemma may require a dedicated endpoint. Qwen uses a conservative 65,536-token limit covering the trial tier. Its model card advertises a larger window on paid tiers. Unknown model ids remain usable, with unknown context size and vision disabled.

reasoning_effort remains active when tools are supplied; a per-turn AIContext.reasoning_effort overrides the provider setting. GPT OSS accepts low, medium or high. Qwen 3.8 also accepts none to disable reasoning. thinking_budget is not translated into an effort level. Available values depend on the selected model, as described in the Cerebras reasoning guide.

reasoning_format="parsed" is the default, keeping thinking separate from answer text. Historical AIThinkingPart values are sent in the assistant's reasoning field, including across tool rounds. clear_thinking is optional and should only be set for a model supporting it. Selecting raw can mix reasoning into visible text; GPT OSS supplies no separator in that mode.

The output cap uses max_completion_tokens. Token usage is collected from Cerebras's final streaming chunk without requesting stream_options. Cache reads are reported separately and priced at the ordinary input rate. Errors and latency metrics identify the provider as cerebras; SDK retries default to zero so RoomKit's retry policy remains in control.

See examples/cerebras_ai.py for a complete conversation.

Mistral Provider¶

MistralAIProvider ¶

MistralAIProvider(config)

Bases: AIProvider

AI provider using the Mistral AI API.

Supports streaming, tool calling, vision (multimodal models), and <think> tag parsing for reasoning models.

supports_response_schema `property` ¶

supports_response_schema

Custom structured outputs, through a strict json_schema format.

available_models `classmethod` ¶

available_models()

Curated, offline catalog of Mistral chat/multimodal models.

list_models `async` ¶

list_models()

List models the Mistral API currently exposes for this key.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events with <think> tag parsing.

Text inside <think>...</think> is yielded as :class:StreamThinkingDelta; everything else as :class:StreamTextDelta. Tool calls are accumulated from deltas and yielded as :class:StreamToolCall. A response schema is checked before the done event (RFC §6.7).

generate `async` ¶

generate(context)

Generate by consuming the structured stream.

generate_stream `async` ¶

generate_stream(context)

Yield text deltas as they arrive from the Mistral API.

close `async` ¶

close()

Close the HTTP client the SDK opened.

The SDK has no close(): leaving its async context is what closes the client it created (RMK-651).

MistralConfig ¶

Bases: BaseModel

Mistral AI provider configuration.

Attributes:

Name	Type	Description
`api_key`	`SecretStr`	Mistral API key for authentication.
`model`	`str`	Model identifier (e.g. `'mistral-large-latest'`, `'pixtral-large-latest'`).
`max_tokens`	`int`	Maximum tokens in the response.
`server_url`	`str \| None`	Custom base URL for Mistral-compatible APIs. If `None`, uses the default Mistral endpoint.
`reasoning_effort`	`str \| None`	Reasoning effort for models that expose it (`mistral-small-latest`, `mistral-medium-3-5`): `"high"` streams a reasoning trace before the answer, `"none"` omits it. `None` leaves it to the model's default (Magistral models always reason). Overridden per-turn by `AIContext.thinking_budget`.

Usage¶

from roomkit.providers.mistral.ai import MistralAIProvider
from roomkit.providers.mistral.config import MistralConfig
from roomkit.channels.ai import AIChannel

config = MistralConfig(api_key="your-api-key")
provider = MistralAIProvider(config)

ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[mistral]

Azure Provider¶

AzureAIProvider ¶

AzureAIProvider(config, *, transport=None)

Bases: OpenAIAIProvider

AI provider using Azure AI Studio's OpenAI-compatible Chat Completions API.

Subclasses :class:OpenAIAIProvider — only client initialisation and provider name differ. All message building, tool handling, response parsing, and streaming are inherited.

transport carries every request, as for :class:OpenAIAIProvider.

supports_vision `property` ¶

supports_vision

True: the configured model is a deployment name its operator chose.

The parent's fallback prefixes, OpenAI's own model names, say nothing of it and reported a deployment named vision-prod as text-only, dropping images before they reached the wire. Passed through, a text-only deployment answers with an error (RFC §6.7).

available_models `classmethod` ¶

available_models()

Azure exposes user-named deployments, not a fixed model catalog.

Deployment names are chosen per Azure resource, so there is no meaningful offline list — use :meth:list_models for the live set.

AzureAIConfig ¶

Bases: BaseModel

Azure AI Studio provider configuration.

Uses the OpenAI-compatible Chat Completions API exposed by Azure AI Foundry deployments (DeepSeek, GPT-4o, Mistral, etc.).

Attributes:

Name	Type	Description
`api_key`	`SecretStr`	Azure API key for authentication.
`azure_endpoint`	`str`	Azure AI Foundry project endpoint URL.
`api_version`	`str`	Azure API version string.
`model`	`str`	Deployment name (no default — user must specify).
`max_tokens`	`int`	Maximum tokens in the response.
`timeout`	`float`	HTTP request timeout in seconds.
`connect_timeout`	`float`	TCP connect timeout in seconds, kept apart from `timeout` so a host that no longer accepts connections is given up on in seconds rather than after the read budget.

max_retries `class-attribute` `instance-attribute` ¶

max_retries = 0

SDK-level retry count. Default 0 because RoomKit's RetryPolicy handles retries at the right layer with proper backoff and fallback.

include_stream_usage `class-attribute` `instance-attribute` ¶

include_stream_usage = True

Request token usage in streaming responses; without it a streamed turn reports no usage. Set False for a deployment that rejects stream_options.

use_max_completion_tokens `class-attribute` `instance-attribute` ¶

use_max_completion_tokens = False

Send the output cap as max_completion_tokens rather than the deprecated max_tokens. Required by newer Azure-hosted OpenAI models; leave False for deployments that only understand max_tokens.

supports_custom_temperature `class-attribute` `instance-attribute` ¶

supports_custom_temperature = True

When False, temperature is omitted — reasoning deployments accept only the default and reject any other value with HTTP 400.

reasoning_effort `class-attribute` `instance-attribute` ¶

reasoning_effort = None

Reasoning depth for reasoning deployments ("low"/"medium"/ "high"); None uses the model default. Sent on a turn without tools and left out on a turn with tools: a deployment name does not say which model it serves, nor so what that model accepts alongside function tools (RFC §6.7), unless supports_reasoning_effort_with_tools says so. The turn's own effort outranks this one.

extra_body `class-attribute` `instance-attribute` ¶

extra_body = None

Extra JSON fields merged into every request body via the SDK's extra_body — for deployment-specific params the OpenAI schema omits. None sends a vanilla body.

supports_response_schema `class-attribute` `instance-attribute` ¶

supports_response_schema = None

Whether the deployment honours a json_schema response format (RFC §6.7). None keeps the provider's default; set it for a deployment whose model or API version differs, so a turn carrying a response schema is refused up front instead of answered in prose.

supports_response_schema_with_tools `class-attribute` `instance-attribute` ¶

supports_response_schema_with_tools = None

Whether the server takes a json_schema response format beside function tools and still lets the model call them. None keeps the provider's default, on for Azure OpenAI deployments.

supports_reasoning_effort_with_tools `class-attribute` `instance-attribute` ¶

supports_reasoning_effort_with_tools = None

Whether the deployment takes reasoning_effort beside function tools (RFC §6.7). True sends the turn's effort on a turn with tools as on any other (an o-series or GPT-5 deployment before GPT-5.4); False leaves it out. None leaves it out too, since a deployment name does not say which model it serves, and logs a warning once when an effort is left out.

Usage¶

from roomkit.providers.azure.ai import AzureAIProvider
from roomkit.providers.azure.config import AzureAIConfig
from roomkit.channels.ai import AIChannel

config = AzureAIConfig(
    azure_endpoint="https://your-resource.openai.azure.com/",
    api_key="your-api-key",
    deployment="your-deployment-name",
)
provider = AzureAIProvider(config)

ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[azure]

Ollama Provider (Local / Cloud LLM)¶

Native provider for Ollama. Calls /api/chat directly, so the think parameter and streamed reasoning work without <think> tag parsing. See AI Thinking — Native Ollama provider for thinking, authentication, and sampling options (temperature, num_ctx, top_p, top_k, min_p, keep_alive).

OllamaAIProvider ¶

OllamaAIProvider(config)

Bases: AIProvider

AI provider using Ollama's native API via the ollama-python SDK.

supports_response_schema `property` ¶

supports_response_schema

Structured outputs, through format holding the schema.

available_models `classmethod` ¶

available_models()

Curated, offline snapshot of popular public Ollama models.

list_models `async` ¶

list_models()

List models installed on the configured Ollama server.

Reads /api/tags for the installed set, then probes /api/show per model (bounded parallel fan-out) to attach capabilities — the tags the picker uses to keep completion-only models out of an embeddings channel and vice-versa. A per-model probe failure yields no capabilities for that model (older servers don't ship the field); consumers treat empty as "unknown, allow everywhere".

generate_stream `async` ¶

generate_stream(context)

Yield text deltas (thinking content filtered out).

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events with thinking streamed separately.

Ollama's native streaming emits one chunk per token-ish, each with message.thinking and/or message.content deltas plus an optional final message.tool_calls. We pass these straight through as the corresponding StreamThinkingDelta, StreamTextDelta, and StreamToolCall events — no tag parsing, no field reordering. A response schema is checked before the done event (RFC §6.7).

close `async` ¶

close()

Release the underlying httpx client.

OllamaConfig ¶

Bases: BaseModel

Ollama AI provider configuration.

Wraps the native Ollama /api/chat endpoint via the ollama-python SDK. Prefer this over the OpenAI-compatible shim when the model is a reasoning model (DeepSeek-R1, Qwen 3 thinking variants, etc.) because Ollama exposes the think parameter and streams the reasoning content as a separate thinking field — both ignored by the OpenAI-compat endpoint.

Attributes:

Name	Type	Description
`host`	`str`	Base URL of the Ollama server. Default points at the local daemon. The native API lives under `/api` on the same host; the SDK appends the path.
`model`	`str`	Model identifier to use (e.g. `"qwen3:8b"`, `"llama3.2"`, `"deepseek-r1:7b"`).
`max_tokens`	`int \| None`	Maximum tokens to generate in the response. Maps to Ollama's `options.num_predict`. `None` lets the server pick its default.
`timeout`	`float`	HTTP request timeout in seconds. Long default because local models cold-start on first request and reasoning models can take 30-60s before the first token.
`connect_timeout`	`float`	TCP connect timeout in seconds, kept apart from `timeout` so a host that no longer accepts connections is given up on in seconds rather than after the read budget.
`max_retries`	`int`	SDK-level retry count. Default 0 because RoomKit's RetryPolicy handles retries at the right layer with proper backoff and fallback.
`think`	`bool \| ThinkEffort \| None`	Whether and how hard the model should reason before answering. `None` (default) means "use the model's default" — reasoning models think, others don't. `True`/`False` force thinking on or off as a boolean. One of `"low"`, `"medium"`, `"high"` selects an effort level for models that take one (gpt-oss); a model that takes none refuses it (Ollama 0.17). The turn outranks it on what it states (RFC §6.7): `thinking_budget` (`0` off, above `0` on), `enable_thinking` and a `reasoning_effort` of `"none"` turn thinking off or on, and a `reasoning_effort` replaces a configured level. A turn's level is sent only when this is a level, the one sign the model takes levels.
`keep_alive`	`str \| int \| None`	How long the model stays loaded in memory after the request. Maps to Ollama's `keep_alive` parameter. A duration string with a unit (`"5m"`, `"30s"`) or an integer number of seconds — `-1` keeps the model loaded indefinitely, `0` unloads it immediately. A unit-less numeric string (e.g. `"-1"` from a text field) is coerced to an int, because Ollama parses a string as a Go duration and rejects one without a unit. `None` uses the server default (5 minutes).
`num_ctx`	`int \| None`	Context window size. Maps to `options.num_ctx`. `None` uses the model's default (typically 2048 — bump for long contexts).
`top_p`	`float \| None`	Nucleus sampling cutoff. Maps to `options.top_p`. `None` uses the model's default.
`top_k`	`int \| None`	Top-k sampling cutoff. Maps to `options.top_k`. `None` uses the model's default.
`min_p`	`float \| None`	Minimum-probability sampling cutoff. Maps to `options.min_p`. `None` uses the model's default.
`api_key`	`SecretStr \| None`	Bearer token for a protected Ollama endpoint — Ollama Cloud/Turbo, or a self-hosted server behind a reverse proxy that checks `Authorization: Bearer`. Sent as the `Authorization` header. `None` (default) leaves auth to the SDK, which still falls back to the `OLLAMA_API_KEY` environment variable when it is set. Prefer this field when the key comes from a secret manager rather than the process environment.
`headers`	`dict[str, str] \| None`	Extra HTTP headers attached to every request — for a reverse proxy that needs custom headers, or a non-Bearer `Authorization` scheme (e.g. Basic). `api_key` takes precedence over an `Authorization` entry supplied here. `None` (default) sends only the SDK's own headers.

Usage¶

from roomkit.providers.ollama import OllamaAIProvider, OllamaConfig
from roomkit.channels.ai import AIChannel

config = OllamaConfig(host="http://localhost:11434", model="llama3.2")
provider = OllamaAIProvider(config)

ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[ollama]

A tool's nested parameters reach the model: a nested object's properties and required, and an anyOf. ollama-python drops them from a declaration before the request leaves (ollama/ollama-python#724; measured on 0.6.2 and 0.6.3), although the Ollama server reads them. Until a release sends them, a request that declares tools goes through the SDK's own request method with the declarations as given (roomkit.providers.ollama.sdk_patch). The server itself drops a $ref and the constraints (minLength and the like), so a schema meant for Ollama spells those out in its descriptions.

Streaming¶

StreamEvent `module-attribute` ¶

StreamEvent = StreamThinkingDelta | StreamTextDelta | StreamToolCallDelta | StreamToolCall | StreamDone

StreamTextDelta ¶

Bases: BaseModel

A text delta from a streaming AI response.

StreamThinkingDelta ¶

Bases: BaseModel

A thinking/reasoning delta from a streaming AI response.

Emitted before text deltas when the model is reasoning. A delta may carry only a signature (with empty thinking): Anthropic streams the thinking block's opaque signature separately, and it must be preserved so the block can be echoed back in history without a 400.

block `class-attribute` `instance-attribute` ¶

block = None

The reasoning block the delta belongs to, for a vendor whose reasoning comes in blocks each with its own signature (Anthropic's content block index); None for one whose reasoning is a single block per round.

redacted `class-attribute` `instance-attribute` ¶

redacted = None

A block the vendor redacted: its opaque data, replayed as received.

StreamToolCall ¶

Bases: BaseModel

A complete tool call extracted from a streaming AI response.

served `class-attribute` `instance-attribute` ¶

served = None

As :attr:AIToolCall.served: the provider already ran the call.

partial `class-attribute` `instance-attribute` ¶

partial = False

As :attr:AIToolCall.partial: the call's arguments do not read.

garbled `class-attribute` `instance-attribute` ¶

garbled = False

As :attr:AIToolCall.garbled: written unreadable, not cut.

StreamDone ¶

Bases: BaseModel

Signals the end of a streaming AI response.

Response Parts¶

AIThinkingPart ¶

Bases: BaseModel

AI reasoning/thinking block in conversation history.

Used to preserve thinking blocks across tool-loop turns (required by providers like Anthropic that mandate round-trip fidelity).

Attributes:

Name	Type	Description
`thinking`	`str`	The reasoning text produced by the model.
`signature`	`str \| None`	Provider-specific opaque token for caching/validation (e.g. Anthropic's thinking block signature).
`redacted`	`str \| None`	The opaque data of a block the vendor redacted (Anthropic's `redacted_thinking`), replayed as received; `thinking` is then empty.

AIToolCallPart ¶

Bases: BaseModel

Assistant's tool call in conversation history.

AIToolResultPart ¶

Bases: BaseModel

Tool execution result in conversation history.

result is a plain string for text results, or a list of content parts (text and/or image) when a tool returns multimodal output — e.g. an edge tool that returns a screenshot. Providers that support image tool results (Anthropic) render the parts as content blocks; the rest flatten via as_text().

as_text ¶

as_text()

Flatten the result to plain text for providers without image support.

A string result is returned unchanged; a list concatenates its text parts and substitutes a [image] placeholder for each image part.

split_for_message ¶

split_for_message()

Split the result into the tool-message text and its image parts.

Unlike Anthropic — whose Messages API accepts image blocks inside a tool_result — most providers reject image content in a tool / function-response message; the image has to ride on a following user message instead. This returns the text to keep on the tool message (text parts joined, or a "[see image below]" placeholder when the result was image-only, so the tool-call/result pairing stays non-empty and valid) together with the image parts to render natively elsewhere.

A string result yields (result, []) and a text-only list yields (joined_text, []) — the no-op path that keeps every existing text tool byte-for-byte unchanged. Only a list carrying an image populates the second element and triggers a provider's synthetic-image path.

ProviderError ¶

ProviderError(message, *, retryable=False, provider='', status_code=None, context_overflow=None)

Bases: Exception

Error from an AI provider SDK call.

Attributes:

Name	Type	Description
`retryable`		Whether the caller should retry the request.
`provider`		Name of the provider that raised the error.
`status_code`		HTTP status code from the provider, if available.
`context_overflow`		Did the request exceed the model's context window? Tri-state. `True` and `False` are a structural classification (measurement, an error code) and are believed as stated, in both directions. `None` means nobody classified, and the message wording decides as a fallback — an envelope may rewrap the provider's prose, and prose must never override an explicit answer.

str ¶

__str__()

The provider's message, led by the provider and the status it answered: cerebras (402): Payment required…. An OpenAI-compatible provider is reached through the OpenAI SDK, whose message names no provider, so a log line or a traceback otherwise says nothing of which one failed. A message that already names its provider is left as it is.

AI Providers¶

AIProvider ¶

name property ¶

supports_vision property ¶

supports_streaming property ¶

supports_structured_streaming property ¶

supports_response_schema property ¶

supports_deferred_tools property ¶

supports_response_schema_with_tools property ¶

model_name abstractmethod property ¶

context_window property ¶

available_models classmethod ¶

list_models async ¶

catalog_entry ¶

generate abstractmethod async ¶

generate_stream async ¶

generate_structured_stream async ¶

close async ¶

AIContext ¶

max_tokens class-attribute instance-attribute ¶

enable_thinking class-attribute instance-attribute ¶

reasoning_effort class-attribute instance-attribute ¶

response_schema class-attribute instance-attribute ¶

model_post_init ¶

model_copy ¶

AIMessage ¶

AITextPart ¶

AIImagePart ¶

AITool ¶

AIToolCall ¶

served class-attribute instance-attribute ¶

partial class-attribute instance-attribute ¶

garbled class-attribute instance-attribute ¶

ServedCall ¶

AIResponse ¶

thinking_parts class-attribute instance-attribute ¶

ModelInfo ¶

ModelPricing ¶

cost_for ¶

MockAIProvider ¶

available_models classmethod ¶

generate_stream async ¶

generate_structured_stream ¶

Model capability tags¶

model_tags ¶

TRANSCRIPTION_CAPABILITY module-attribute ¶

SPEECH_CAPABILITY module-attribute ¶

speech_tags ¶

with_speech_tags ¶

Response schema¶

check_portable_schema ¶

schema_mismatch ¶

ResponseSchemaError ¶

Image parts¶

image_part_payload ¶

image_part_base64 ¶

image_part_uri ¶

Handing the loop a tool call¶

tool_arguments ¶

readable_arguments ¶

realtime_call_arguments ¶

CutArguments ¶

unreadable_arguments ¶

call_partial ¶

call_cut ¶

call_garbled ¶

CallIds ¶

partial_call_error ¶

tool_call_of ¶

Declaring tools¶

declared_parameters ¶

chat_tool_declarations ¶

ToolNameRule dataclass ¶

pattern instance-attribute ¶

check ¶

Rendering a conversation as Chat Completions messages¶

ChatDialect dataclass ¶

thinking_field class-attribute instance-attribute ¶

drops_thinking class-attribute instance-attribute ¶

names_tool_results class-attribute instance-attribute ¶

name `property` ¶

supports_vision `property` ¶

supports_streaming `property` ¶

supports_structured_streaming `property` ¶

supports_response_schema `property` ¶

supports_deferred_tools `property` ¶

supports_response_schema_with_tools `property` ¶

model_name `abstractmethod` `property` ¶

context_window `property` ¶

available_models `classmethod` ¶

list_models `async` ¶

generate `abstractmethod` `async` ¶

generate_stream `async` ¶

generate_structured_stream `async` ¶

close `async` ¶

max_tokens `class-attribute` `instance-attribute` ¶

enable_thinking `class-attribute` `instance-attribute` ¶

reasoning_effort `class-attribute` `instance-attribute` ¶

response_schema `class-attribute` `instance-attribute` ¶

served `class-attribute` `instance-attribute` ¶

partial `class-attribute` `instance-attribute` ¶

garbled `class-attribute` `instance-attribute` ¶

thinking_parts `class-attribute` `instance-attribute` ¶

available_models `classmethod` ¶

generate_stream `async` ¶

TRANSCRIPTION_CAPABILITY `module-attribute` ¶

SPEECH_CAPABILITY `module-attribute` ¶

ToolNameRule `dataclass` ¶

pattern `instance-attribute` ¶

ChatDialect `dataclass` ¶

thinking_field `class-attribute` `instance-attribute` ¶

drops_thinking `class-attribute` `instance-attribute` ¶

names_tool_results `class-attribute` `instance-attribute` ¶

flattens_text `class-attribute` `instance-attribute` ¶

round_thinking_required `class-attribute` `instance-attribute` ¶

supports_vision `property` ¶

supports_response_schema `property` ¶

supports_response_schema_with_tools `property` ¶

available_models `classmethod` ¶

list_models `async` ¶

generate_structured_stream `async` ¶

generate `async` ¶

generate_stream `async` ¶

close `async` ¶

thinking_level `class-attribute` `instance-attribute` ¶

timeout `class-attribute` `instance-attribute` ¶

connect_timeout `class-attribute` `instance-attribute` ¶

api_key `class-attribute` `instance-attribute` ¶

project `instance-attribute` ¶

location `instance-attribute` ¶

service_account_json `class-attribute` `instance-attribute` ¶

impersonate_service_account `class-attribute` `instance-attribute` ¶

labels `class-attribute` `instance-attribute` ¶

supports_vision `property` ¶

available_models `classmethod` ¶

list_models `async` ¶

base_url `class-attribute` `instance-attribute` ¶

model `class-attribute` `instance-attribute` ¶

use_max_completion_tokens `class-attribute` `instance-attribute` ¶

include_stream_usage `class-attribute` `instance-attribute` ¶

max_retries `class-attribute` `instance-attribute` ¶

include_stream_usage `class-attribute` `instance-attribute` ¶

headers `class-attribute` `instance-attribute` ¶

top_p `class-attribute` `instance-attribute` ¶

top_k `class-attribute` `instance-attribute` ¶

min_p `class-attribute` `instance-attribute` ¶

presence_penalty `class-attribute` `instance-attribute` ¶

repetition_penalty `class-attribute` `instance-attribute` ¶

extra_body `class-attribute` `instance-attribute` ¶

enable_thinking `class-attribute` `instance-attribute` ¶

reasoning_effort `class-attribute` `instance-attribute` ¶

supports_response_schema `class-attribute` `instance-attribute` ¶

supports_response_schema_with_tools `class-attribute` `instance-attribute` ¶

base_url `property` ¶

start `async` ¶

close `async` ¶