AI Providers¶

AIProvider ¶

Bases: ABC

AI model provider for generating responses.

name `property` ¶

name

Provider name (e.g. 'anthropic', 'openai').

supports_vision `property` ¶

supports_vision

Whether this provider can process images.

supports_streaming `property` ¶

supports_streaming

Whether this provider supports streaming token generation.

supports_structured_streaming `property` ¶

supports_structured_streaming

Whether this provider supports structured streaming with tool calls.

model_name `abstractmethod` `property` ¶

model_name

Model identifier (e.g. 'claude-opus-5', 'gpt-5.6-sol').

context_window `property` ¶

context_window

Input context window of the active model in tokens, if known.

Resolved offline from the curated :meth:available_models catalog keyed by :attr:model_name — no API key or network. Returns None when the active model is absent from the catalog (custom / local model ids, e.g. an arbitrary vLLM model string), so callers must degrade gracefully rather than assume a window.

available_models `classmethod` ¶

available_models()

Offline metadata for the models roomkit can describe without a key.

This is not the discovery surface. A provider's lineup turns over faster than a release cycle, so a hand-maintained list can never be the authoritative answer to "what does this provider offer" — that is :meth:list_models, which asks the provider. What this list is for is the metadata roomkit needs before any network call exists: it backs :attr:context_window (a sync property, so it cannot await an API) and backfills the sparse ids a live endpoint returns, via :meth:_merge_curated.

A model absent from it is an ordinary outcome, not an error: the caller gets context_window is None and degrades, which is the point — an unknown window is safer than a stale one. The base returns an empty list; providers override it.

list_models `async` ¶

list_models()

Models reported live by the provider's API — the discovery surface.

Always current, and the only answer that reflects the caller's own account (entitlements, regional availability, locally loaded weights). The base implementation falls back to :meth:available_models for providers whose API exposes no models endpoint; the rest override this to query it, backfilling missing metadata via :meth:_merge_curated.

catalog_entry ¶

catalog_entry()

The offline :class:ModelInfo for the active model, if known.

The single place a provider should read its own model's metadata from. A second hardcoded table — a tuple of vision-capable prefixes, say — duplicates what :meth:available_models already states and rots independently of it, which is how a provider ends up reporting a current model as text-only.

Returns None for an id the catalog does not carry (a custom or local model behind base_url, a snapshot newer than this release).

generate `abstractmethod` `async` ¶

generate(context)

Generate an AI response from the given context.

Parameters:

Name	Type	Description	Default
`context`	`AIContext`	Conversation context including messages, system prompt, temperature, and target channel capabilities.	required

Returns:

Type	Description
`AIResponse`	The AI response with content, usage stats, and optional
`AIResponse`	tasks/observations.

generate_stream `async` ¶

generate_stream(context)

Yield text deltas as they arrive. Override for streaming providers.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events (thinking deltas, text deltas, tool calls, done).

A provider whose wire format fragments a tool call's arguments SHOULD also yield :class:StreamToolCallDelta per fragment; it is optional, and one that delivers whole calls yields none.

Default implementation wraps generate() so every provider works without changes. Override for true streaming support.

close `async` ¶

close()

Release resources. Override in subclasses that hold connections.

AIContext ¶

Bases: BaseModel

Context passed to AI provider for generation.

max_tokens `class-attribute` `instance-attribute` ¶

max_tokens = None

Output cap for this turn. None means "not set for this turn", which lets each provider fall back to its own configured max_tokens. A non-None default here would shadow that config and make it dead.

enable_thinking `class-attribute` `instance-attribute` ¶

enable_thinking = None

Turn this turn's reasoning block on or off, for providers that expose the switch. None defers to the provider's own configuration, and then to the model's default.

reasoning_effort `class-attribute` `instance-attribute` ¶

reasoning_effort = None

Reasoning verbosity for this turn, for providers that grade it. Accepted values are the provider's own; None defers to its config.

model_post_init ¶

model_post_init(__context)

Protect metadata even when Pydantic's validation was bypassed.

model_copy ¶

model_copy(*, update=None, deep=False)

Preserve secret wrapping when model_copy(update=...) bypasses validation.

AIMessage ¶

Bases: BaseModel

A message in the AI conversation context.

AITextPart ¶

Bases: BaseModel

Text part of a multimodal message.

AIImagePart ¶

Bases: BaseModel

Image part of a multimodal message.

AITool ¶

Bases: BaseModel

Tool definition for function calling.

AIToolCall ¶

Bases: BaseModel

A tool call from the AI response.

AIResponse ¶

Bases: BaseModel

Response from an AI provider.

ModelInfo ¶

Bases: BaseModel

Metadata describing a single model offered by an AI provider.

Both the curated catalog (:meth:AIProvider.available_models) and the live API query (:meth:AIProvider.list_models) return these. Only id is guaranteed; the remaining fields are best-effort and may be None when the source does not report them.

Attributes:

Name	Type	Description
`id`	`str`	Exact model identifier accepted by the provider's API (e.g. `"claude-sonnet-4-20250514"`, `"gpt-4o"`).
`display_name`	`str \| None`	Human-friendly name (e.g. `"Claude Sonnet 4"`).
`context_window`	`int \| None`	Input context window in tokens, if known.
`supports_vision`	`bool \| None`	Whether the model accepts image input, if known.
`deprecated`	`bool`	Whether the provider marks the model deprecated.
`capabilities`	`list[str]`	Provider-reported capability tags (e.g. Ollama's `"completion"`, `"embedding"`, `"vision"`, `"tools"`). Empty when the source does not report them — consumers treat empty as "unknown, allow everywhere" rather than "none".
`pricing`	`ModelPricing \| None`	Vendor list price for this model, if published. It lives here, beside the id, because a lineup and its price list turn over together: kept apart, adding a model leaves its price behind and the consumer bills nothing. `None` where no per-token list price exists — locally pulled open weights, a private edge, a retired id the vendor stopped quoting.

ModelPricing ¶

Bases: BaseModel

List price of one model, per million tokens, as its vendor published it.

Rates mirror the keys roomkit itself reports in usage (:attr:AIResponse.usage) — input, output, cache reads, cache writes — so a consumer can price a response without inventing a mapping. What is not here is deliberate: per-client negotiated rates, discounts and currency conversion belong to whoever bills, not to a shared catalog.

A rate is volatile in a way a model id is not, hence :attr:verified: the entry states what the vendor published on that date, and a consumer can decide for itself when that is too old to trust.

Attributes:

Name	Type	Description
`input_per_million`	`float`	Price of a million uncached input tokens.
`output_per_million`	`float`	Price of a million output tokens.
`cache_read_per_million`	`float \| None`	Price of a million tokens re-read from the prompt cache. `None` means this catalog represents no separate per-token charge for that counter. Catalogs must explicitly repeat the input rate when cache reads are billed as ordinary input.
`cache_write_per_million`	`float \| None`	Price of a million tokens written to the prompt cache — Anthropic's 5-minute write premium (1.25x input), which is the TTL roomkit's `ephemeral` markers request. `None` where a write is not billed per token. Google cache storage, for example, is billed by time and cannot be represented here.
`image_input_per_million`	`float \| None`	Price of a million image input tokens, where the vendor quotes them apart from text — a reference image handed to an image model, say. `None` where the catalog represents no separate charge, which is every conversational model here: they bill a vision token as an ordinary input token.
`image_output_per_million`	`float \| None`	Price of a million generated-image tokens. An image model is billed per token like any other, only with the picture counted on its own meter and at a rate an order of magnitude above the text one — which is why it is a field and not an approximation folded into `output_per_million`. `None` for a model that generates no images.
`long_context_threshold_tokens`	`int \| None`	Total input-token threshold above which the model's published long-context multipliers apply. `None` for models with flat pricing.
`long_context_input_multiplier`	`float`	Multiplier applied to all represented input and cache charges above the long-context threshold.
`long_context_output_multiplier`	`float`	Multiplier applied to output charges above the long-context threshold.
`currency`	`str`	ISO 4217 code the rates are quoted in. Every vendor roomkit ships a catalog for publishes in USD.
`verified`	`date`	Date the rates were read from the vendor's own price list.

cost_for ¶

cost_for(usage)

Price a single response's usage dict, in :attr:currency.

Reads the keys roomkit's providers report — input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens, plus input_image_tokens and output_image_tokens from an image generation (:class:~roomkit.providers.image.base.ImageProvider) — and ignores anything else, so a provider reporting extra counters neither breaks nor inflates the total. Missing keys count as zero.

Every counter is disjoint: a token is charged under exactly one of them. Providers that receive image tokens nested inside a total subtract them before reporting, so summing here bills each token once.

A counter with no corresponding rate is omitted: None means the catalog does not represent a separate per-token charge for it. When the response crosses a published long-context threshold, the configured input and output multipliers are applied automatically.

Parameters:

Name	Type	Description	Default
`usage`	`Mapping[str, int]`	A response's token counters, as reported by the provider.	required

Returns:

Type	Description
`float`	The cost of that response, in :attr:`currency`.

ModelPricing is also exported from the package root. Its rates are finite and non-negative, its multipliers finite and positive, and cost_for() accepts only non-negative integer token counters. Invalid accounting input raises instead of producing a negative or non-finite cost.

MockAIProvider ¶

MockAIProvider(responses=None, *, vision=False, ai_responses=None, streaming=False, tool_call_delta_chunks=0)

Bases: AIProvider

Round-robin response provider for tests.

available_models `classmethod` ¶

available_models()

Fixed two-entry catalog for exercising model-discovery code.

generate_stream `async` ¶

generate_stream(context)

Yield text from generate() as a single delta.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events from generate() result.

Image parts¶

Every provider turns an AIImagePart into what its API takes through one reader: media type from the header, then the part's mime_type, then image/png; a payload an encoder wrapped or left unpadded is repaired; a corrupt one is refused before the request leaves, as a non-retryable ProviderError that names the cause.

image_part_payload ¶

image_part_payload(part, *, provider)

The media type and the decoded bytes of a data: URI image part.

:func:~roomkit.providers.utils.parse_data_uri with the part's own mime_type as the fallback, and its ValueError surfaced as the non-retryable :class:ProviderError an AI provider's caller expects: a caller error, named as such, raised before the request leaves — never a retry or a fallback, since the same URI would fail again. Only for a data: URI; a remote URL is the provider's to forward.

image_part_base64 ¶

image_part_base64(part, *, provider)

The media type and the canonical base64 payload of a data: URI image part.

For an API that takes the two apart — Anthropic's source block, Ollama's images list. Same reading as :func:image_part_payload, re-encoded from the validated bytes.

image_part_uri ¶

image_part_uri(part, *, provider)

The URI an OpenAI-shaped request forwards for an image part.

A remote URL passes through untouched. A data: URI goes through :func:image_part_payload and is rebuilt canonically, so a header without a media type reaches the vendor with the part's, and a payload an encoder wrapped reaches it on one line.

Explicit models and modern request profiling¶

model= is required by both OpenAIConfig and AnthropicConfig, so a RoomKit upgrade cannot silently change cost, latency, or model behavior. For the model the caller selects, OpenAIConfig profiles first-party gpt-5/o-series models to use max_completion_tokens and omit a custom temperature. AnthropicConfig profiles modern Claude reasoning models to use adaptive thinking and omit temperature. This prevents selected modern model ids from being paired with parameters those models reject.

Explicit compatibility flags always win. A custom base_url also keeps the conservative legacy defaults, since OpenAI-compatible and Anthropic-compatible proxies may implement the older request shape.

The OpenAI provider currently uses Chat Completions. On OpenAI's own endpoint, a GPT-5.6 turn carrying function tools therefore sends reasoning_effort="none" explicitly: omission would select the model family's medium default, which is incompatible with function tools on that endpoint. Tool-free turns still use OpenAIConfig.reasoning_effort, or the model default when it is unset. Custom base_url deployments are never force-profiled this way.

Per-Room AI Configuration¶

AI channel settings can be overridden per-room using binding metadata:

# Default AI channel
ai = AIChannel("ai", provider=anthropic, system_prompt="Default assistant")
kit.register_channel(ai)

# Override per room
kit.attach_channel(room_id, "ai", metadata={
    "system_prompt": "You are a customer support agent for Acme Corp.",
    "temperature": 0.3,  # More deterministic
    "max_tokens": 2048,
})

Silent Observer Pattern (Meeting Notes)¶

# Attach AI as note-taker
kit.attach_channel(meeting_room_id, "ai", metadata={
    "system_prompt": """You are a meeting note-taker.
    Listen to the conversation silently.
    When someone says 'meeting ended', compile and send a summary.""",
})

# Mute so AI listens but doesn't respond
await kit.mute(meeting_room_id, "ai")

# Later, unmute to let AI send summary
await kit.unmute(meeting_room_id, "ai")

Tools/Function Calling¶

Tools can be passed via binding metadata for function calling:

kit.attach_channel(room_id, "ai", metadata={
    "tools": [
        {
            "name": "search_knowledge_base",
            "description": "Search the company knowledge base",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string"}
                },
                "required": ["query"]
            }
        }
    ]
})

Tool calls are returned in AIResponse.tool_calls:

response = await provider.generate(context)
for tool_call in response.tool_calls:
    print(f"Tool: {tool_call.name}, Args: {tool_call.arguments}")

Gemini Provider¶

GeminiAIProvider ¶

GeminiAIProvider(config)

Bases: AIProvider

AI provider using the Google Gemini API.

supports_vision `property` ¶

supports_vision

All Gemini models support vision.

available_models `classmethod` ¶

available_models()

Curated, offline catalog of Gemini models.

list_models `async` ¶

list_models()

List generate-content models the Gemini API currently exposes.

Serves both surfaces this provider family speaks to, which name their models differently and describe them unequally:

Developer API (AI Studio): models/gemini-3.5-flash, and each entry declares supported_actions.
Vertex (:class:~roomkit.providers.gemini.vertex.GeminiVertexProvider): publishers/google/models/gemini-2.5-flash, with no supported_actions and no metadata at all.

So the id is the last path segment rather than one stripped prefix — a prefixed id matches nothing in the curated catalog, which silently emptied the metadata, and would be written to a caller's config as a model name the API then rejects. And where no action is declared, the listing mixes in models this call cannot serve, which the family and embedding checks drop without hiding a model too new to be curated.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events from the Gemini streaming API.

generate `async` ¶

generate(context)

Generate by consuming the structured stream.

generate_stream `async` ¶

generate_stream(context)

Yield text deltas as they arrive from the Gemini API.

close `async` ¶

close()

Close the SDK and the httpx client it was given.

GeminiConfig ¶

Bases: BaseModel

Google Gemini AI provider configuration.

thinking_level `class-attribute` `instance-attribute` ¶

thinking_level = None

Thinking level for Gemini 3.1 models: minimal, low, medium, high.

timeout `class-attribute` `instance-attribute` ¶

timeout = 60.0

Read budget in seconds: how long the first chunk, and each one after it, may take. Generation streams, so a stalled answer fails here instead of holding the turn open.

connect_timeout `class-attribute` `instance-attribute` ¶

connect_timeout = 5.0

TCP connect timeout in seconds, kept apart from timeout so a host that no longer accepts connections is given up on in seconds rather than after the read budget. The default across RoomKit's providers.

GeminiVertexProvider ¶

GeminiVertexProvider(config)

Bases: GeminiAIProvider

Gemini provider backed by Vertex AI in a specific Google Cloud region.

Subclasses :class:GeminiAIProvider — only client construction (Vertex mode, and an identity that is not an API key) and the billing labels on each request differ. All generation, streaming, thinking, and model discovery are inherited.

GeminiVertexConfig ¶

Bases: GeminiConfig

Gemini-on-Vertex configuration.

Subclasses :class:GeminiConfig, inheriting every generation field (model, max_tokens, temperature, thinking_level) so the two cannot drift. There is no API key on Vertex: the caller is authenticated either by an explicit service-account key or, failing that, by Application Default Credentials — the standard Google chain (gcloud auth application-default login, GOOGLE_APPLICATION_CREDENTIALS, workload identity).

api_key `class-attribute` `instance-attribute` ¶

api_key = None

Optional and unused on Vertex — the identity comes from service_account_json or from ADC, never from a key on the request.

project `instance-attribute` ¶

project

Google Cloud project id that hosts the Vertex AI API.

location `instance-attribute` ¶

location

Vertex region — required, no default. Pin it to keep data in-region for residency (e.g. "northamerica-northeast1" for Montréal, "europe-west1"). A default like "global" could route out of region and defeat the whole point, so the choice is made explicit.

service_account_json `class-attribute` `instance-attribute` ¶

service_account_json = None

A service-account key file's contents, as JSON, authenticating as that account instead of as the process.

ADC answers "who is this machine", which is the wrong question wherever one deployment serves several projects: the ambient identity belongs to whoever runs the server, so a caller naming someone else's project gets PERMISSION_DENIED no matter what it puts in project. Passing a key here makes the identity travel with the configuration, which is what lets one process serve one project per tenant.

None keeps the ADC chain, which stays right for a single-project deployment and for local development.

impersonate_service_account `class-attribute` `instance-attribute` ¶

impersonate_service_account = None

Email of a service account to borrow instead of holding its key.

The identity a caller needs and the secret it holds are separate problems, and organizations increasingly forbid the second: Google enforces constraints/iam.disableServiceAccountKeyCreation by default on recent organizations, so a project owner cannot hand out a key even when willing. Here the owner instead grants this deployment's own identity roles/iam.serviceAccountTokenCreator on one of their service accounts, and Vertex is called as that account with short-lived tokens nobody ever downloads — revocable from their side, in one command, without telling us.

Combines with the fields above rather than replacing them: the borrowing identity is service_account_json when set, otherwise ADC.

labels `class-attribute` `instance-attribute` ¶

labels = None

Per-request labels for Google's billing report, e.g. {"tenant": "acme"}.

Vertex attaches them to every generateContent call and Cloud Billing groups the charges by them, which is how one project's Gemini spend is attributed to the tenants or partners it serves. Metadata only: no quota, no limit, no effect on the answer. The report runs 24 to 48 hours behind, so it is never a source of truth for what a caller consumed — meter that from the usage each response reports.

Constant for the life of the provider, like project: a provider serves one channel, a channel one agent, an agent one tenant. Google's label rules are enforced here so a bad label fails at configuration rather than on the first request: at most 64 labels; keys 1 to 63 characters starting with a lowercase letter, values 0 to 63; lowercase letters, digits, _ and - only (international characters allowed). The Gemini Developer API refuses the field outright, which is why it lives on this config alone.

Usage¶

from roomkit.providers.gemini.ai import GeminiAIProvider
from roomkit.providers.gemini.config import GeminiConfig

config = GeminiConfig(api_key="your-api-key")
provider = GeminiAIProvider(config)

# Use with AIChannel
ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[gemini]

Schema Cleaning¶

Gemini rejects extra JSON Schema fields common in MCP/OpenAPI tool definitions. RoomKit auto-cleans schemas when building FunctionDeclaration objects:

clean_gemini_schema ¶

clean_gemini_schema(schema)

Recursively prepare a JSON Schema for Gemini FunctionDeclaration.

Performs two passes per node:

Collapse Pydantic-style Optionals. {"anyOf": [{...}, {"type": "null"}]} becomes {..., "nullable": true}. oneOf / allOf are handled the same way for symmetry. When the union has multiple non-null branches, we keep the first one and add nullable if any branch was null — Gemini's schema dialect does not support full union types.
Strip unknown keys. Anything not in :data:_GEMINI_ALLOWED_KEYS (e.g. $schema, additionalProperties, default, title) is dropped at every nesting level.

Parameters:

Name	Type	Description	Default
`schema`	`dict[str, Any] \| None`	A JSON Schema dict (e.g. tool parameter schema), or None.	required

Returns:

Type	Description
`dict[str, Any] \| None`	A cleaned copy, or None if input was None.

vLLM Provider (Local LLM)¶

VLLMConfig ¶

Bases: BaseModel

Configuration for a local vLLM server.

vLLM exposes an OpenAI-compatible API, so this config is translated into an OpenAIConfig by :func:create_vllm_provider.

Attributes:

Name	Type	Description
`model`	`str`	Model name loaded by the vLLM server (required).
`base_url`	`str`	Base URL of the vLLM OpenAI-compatible endpoint.
`api_key`	`SecretStr`	Bearer token sent as `Authorization: Bearer <key>`. Matches `vllm serve --api-key`; default `"none"` for the common no-auth local server.
`max_tokens`	`int`	Maximum tokens in the response.
`temperature`	`float`	Sampling temperature.
`timeout`	`float`	HTTP request timeout in seconds. Increase for vLLM servers that load models lazily on first request.
`connect_timeout`	`float`	TCP connect timeout in seconds, kept apart from `timeout` so a host that no longer accepts connections is given up on in seconds rather than after the read budget.
`headers`	`dict[str, str] \| None`	Extra HTTP headers on every request — for a reverse proxy that needs custom headers, or a non-Bearer `Authorization` scheme. Maps to `OpenAIConfig.default_headers`.
`top_p`	`float \| None`	Nucleus sampling cutoff. `None` leaves the server's default.
`top_k`	`int \| None`	Top-k sampling cutoff — a vLLM extension, not an OpenAI field. `None` leaves the server's default.
`min_p`	`float \| None`	Minimum-probability sampling cutoff — a vLLM extension. `None` leaves the server's default.
`presence_penalty`	`float \| None`	Penalty on tokens already present in the output. The knob Qwen's own guidance raises (to `1.5`) for non-thinking mode, where the failure it addresses is degenerate repetition. `None` leaves the server's default.
`repetition_penalty`	`float \| None`	Multiplicative repetition penalty — a vLLM extension, distinct from `presence_penalty`. `None` leaves the server's default.
`extra_body`	`dict[str, Any] \| None`	Extra JSON fields merged into every request body — the route for vLLM params this config does not model (`guided_json`/`guided_choice` guided decoding, and any sampler added by a newer server). Maps to `OpenAIConfig.extra_body`, and an entry here wins over the typed fields above.
`enable_thinking`	`bool \| None`	Turn the model's reasoning block on or off. `None` leaves the model's own default, which for current Qwen builds is on at the most verbose effort — reasoning then competes with the answer for `max_tokens` and can consume the whole budget, leaving an empty `content`. Set `False` for tool loops that only need the final answer.
`reasoning_effort`	`str \| None`	Reasoning verbosity when thinking is on — `"low"`, `"medium"` or `"xhigh"`. Accepted values depend on the served model's chat template; vLLM raises a template error on an unknown one.

max_retries `class-attribute` `instance-attribute` ¶

max_retries = 0

SDK-level retry count. Default 0 because RoomKit's RetryPolicy handles retries at the right layer with proper backoff and fallback.

include_stream_usage `class-attribute` `instance-attribute` ¶

include_stream_usage = False

When True, request token usage in streaming responses.

headers `class-attribute` `instance-attribute` ¶

headers = None

Extra HTTP headers sent on every request (proxy headers, non-Bearer auth). None sends only the SDK defaults.

top_p `class-attribute` `instance-attribute` ¶

top_p = None

Nucleus sampling cutoff. None leaves the server's default.

top_k `class-attribute` `instance-attribute` ¶

top_k = None

Top-k sampling cutoff (vLLM extension). None leaves the default.

min_p `class-attribute` `instance-attribute` ¶

min_p = None

Minimum-probability cutoff (vLLM extension). None leaves the default.

presence_penalty `class-attribute` `instance-attribute` ¶

presence_penalty = None

Penalty on already-present tokens. None leaves the default.

repetition_penalty `class-attribute` `instance-attribute` ¶

repetition_penalty = None

Multiplicative repetition penalty (vLLM extension). None leaves the default.

extra_body `class-attribute` `instance-attribute` ¶

extra_body = None

Extra request-body fields for vLLM params this config does not model (guided decoding, a newer server's sampler). None sends a vanilla body; an entry here wins over the typed sampling fields.

enable_thinking `class-attribute` `instance-attribute` ¶

enable_thinking = None

Reasoning block on/off. None leaves the model's own default.

reasoning_effort `class-attribute` `instance-attribute` ¶

reasoning_effort = None

Reasoning verbosity when thinking is on ("low"/"medium"/"xhigh").

sampling_body ¶

sampling_body()

The request-body fields implied by the sampling settings.

All of them ride the body rather than the SDK's named parameters: top_k, min_p and repetition_penalty are vLLM extensions the OpenAI SDK has no argument for, and top_p/presence_penalty are read from the same body by the server, so routing the five through one place keeps the split invisible to the caller.

Only the knobs actually set are emitted — None means "the server decides", which is not the same as sending its documented default and is the only honest answer for a server whose model we cannot see. Tested with is not None so an explicit 0 survives: min_p=0 and presence_penalty=0 are meaningful values, not absences.

chat_template_kwargs ¶

chat_template_kwargs()

The chat_template_kwargs implied by the reasoning settings.

vLLM renders the model's own chat template server-side, so reasoning is steered through template kwargs rather than a sampling parameter. Empty when neither knob is set, so a vanilla body stays vanilla.

create_vllm_provider ¶

create_vllm_provider(config)

Create an OpenAI-compatible AI provider pointed at a local vLLM server.

The returned provider is an :class:OpenAIAIProvider subclass: identical on the wire, but reporting model metadata for your server rather than OpenAI's hosted catalog. The openai SDK is imported lazily, when the provider is instantiated.

Note

available_models() is empty by design — nothing offline can know what a local server loaded. Use :meth:~OpenAIAIProvider.list_models, which queries the server's /v1/models endpoint.

Parameters:

Name	Type	Description	Default
`config`	`VLLMConfig`	vLLM connection settings.	required

Returns:

Type	Description
`OpenAIAIProvider`	A provider configured for the local vLLM server.

Usage¶

from roomkit.providers.vllm import create_vllm_provider, VLLMConfig
from roomkit.channels.ai import AIChannel

# Configure connection to a local vLLM server
config = VLLMConfig(
    model="meta-llama/Llama-3.1-8B-Instruct",
    base_url="http://localhost:8000/v1",
)

# Factory returns an OpenAIAIProvider pointed at your vLLM server
provider = create_vllm_provider(config)

# Use with AIChannel like any other AI provider
ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[vllm]

Anthropic Provider¶

AnthropicAIProvider ¶

AnthropicAIProvider(config)

Bases: AIProvider

AI provider using the Anthropic Messages API.

supports_vision `property` ¶

supports_vision

Whether the configured Claude model accepts image input.

Read from the offline catalog, which states it per model, rather than from a prefix table that has to be remembered on every release — the prefix form silently reported the whole 4.5-and-later lineup as text-only, dropping images before they reached the wire.

available_models `classmethod` ¶

available_models()

Curated, offline catalog of Claude models.

list_models `async` ¶

list_models()

List models the Anthropic API currently exposes for this key.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events from the Anthropic Messages streaming API.

When extended thinking is enabled, yields StreamThinkingDelta events before text deltas.

generate `async` ¶

generate(context)

Generate by consuming the structured stream.

generate_stream `async` ¶

generate_stream(context)

Yield text deltas as they arrive from the Anthropic Messages API.

close `async` ¶

close()

Close the configured client and every per-request one still cached.

AnthropicConfig ¶

Bases: BaseModel

Anthropic AI provider configuration.

model `instance-attribute` ¶

model

Model identifier. Required so upgrading RoomKit cannot silently change a caller's model, cost, latency, or behavior.

timeout `class-attribute` `instance-attribute` ¶

timeout = 60.0

Request timeout in seconds (default 60s).

connect_timeout `class-attribute` `instance-attribute` ¶

connect_timeout = 5.0

TCP connect timeout in seconds, kept apart from timeout so a host that no longer accepts connections is given up on in seconds rather than after the read budget. The SDK's own default.

base_url `class-attribute` `instance-attribute` ¶

base_url = None

Override the base URL (e.g., for Claude Code sandbox proxy).

The SDK appends /v1/messages to this, so a value that already ends in that path is dropped down to its parent: Microsoft Foundry documents the Claude surface as the whole <resource>/anthropic/v1/messages URL, and pasting it verbatim would otherwise post to /v1/messages/v1/messages. A bare trailing /v1 is deliberately left alone — unlike the full path it is not unambiguously wrong, and a gateway may route on it.

extra_headers `class-attribute` `instance-attribute` ¶

extra_headers = None

Extra headers sent with every request (e.g., X-Tenant-ID).

enable_prompt_caching `class-attribute` `instance-attribute` ¶

enable_prompt_caching = True

Apply Anthropic prompt caching (explicit cache_control markers) to the stable request prefix — tools, system prompt, and the conversation suffix. Every tool-loop round re-sends the full context; without markers it is billed at the full input rate on every round, with them the prefix re-reads at the cached rate (10%). Disable for proxies that reject cache_control blocks.

use_adaptive_thinking `class-attribute` `instance-attribute` ¶

use_adaptive_thinking = False

Send extended thinking as {"type": "adaptive"} instead of the deprecated {"type": "enabled", "budget_tokens": N}. Anthropic's newer models reject budget_tokens with HTTP 400. Official modern models are profiled automatically; an explicit value or custom base_url is left untouched for compatibility with proxies and older deployments.

supports_custom_temperature `class-attribute` `instance-attribute` ¶

supports_custom_temperature = True

When False, temperature is omitted from requests. Anthropic's modern reasoning models removed the sampling parameters and reject temperature with HTTP 400. Official modern models are profiled automatically unless this field is explicitly set.

model_post_init ¶

model_post_init(__context)

Apply safe defaults for Anthropic's modern first-party models.

Usage¶

from roomkit.providers.anthropic.ai import AnthropicAIProvider
from roomkit.providers.anthropic.config import AnthropicConfig
from roomkit.channels.ai import AIChannel

config = AnthropicConfig(api_key="your-api-key", model="claude-opus-5")
provider = AnthropicAIProvider(config)

ai_channel = AIChannel("ai", provider=provider)

Per-request credentials¶

The configured key remains the default. In a multi-tenant host where a caller uses their own Anthropic subscription, a BEFORE_AI_GENERATION hook can select that credential for one turn without rebuilding the shared provider:

from roomkit import HookResult, HookTrigger
from roomkit.providers.ai import API_KEY_METADATA_KEY

@kit.hook(HookTrigger.BEFORE_AI_GENERATION)
async def select_anthropic_key(event, ctx):
    key = await tenant_secrets.anthropic_key(ctx.room.metadata["tenant_id"])
    event.ai_context.metadata[API_KEY_METADATA_KEY] = key
    return HookResult.allow()

RoomKit stores the value as a Pydantic secret, so rendering or serializing the context redacts it. An absent, empty, or non-string value — or one equal to the configured key — falls back to AnthropicConfig.api_key and its shared client.

Per-key clients are cached in a pool bounded by a soft limit: only entries whose last turn has finished are evicted and closed. A burst of distinct credentials may hold the pool briefly above that limit, and it trims itself as those turns end, because closing a client underneath an in-flight stream would break a response that has nothing to do with the new caller.

Install with: pip install roomkit[anthropic]

OpenAI Provider¶

OpenAIAIProvider ¶

OpenAIAIProvider(config)

Bases: AIProvider

AI provider using the OpenAI Chat Completions API.

supports_vision `property` ¶

supports_vision

Whether the configured model accepts image input.

Read from the offline catalog, which states it per model, rather than from a prefix table that has to be remembered on every release — the prefix form predated GPT-5 entirely and silently reported the whole current lineup as text-only, dropping images before they reached the wire.

available_models `classmethod` ¶

available_models()

Curated, offline catalog of OpenAI chat/multimodal models.

list_models `async` ¶

list_models()

List every model id the configured endpoint exposes.

The OpenAI /v1/models response carries only ids — metadata for known chat models is backfilled from the curated catalog. The raw list also includes non-chat models (embeddings, audio); they pass through unfiltered since the endpoint reports no capability field.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events with <think> tag parsing.

Text inside <think>...</think> is yielded as :class:StreamThinkingDelta; everything else as :class:StreamTextDelta. Tool calls are collected from the final chunks and yielded as :class:StreamToolCall.

generate_stream `async` ¶

generate_stream(context)

Yield text deltas (thinking content filtered out).

close `async` ¶

close()

Close the underlying HTTP client.

OpenAIConfig ¶

Bases: BaseModel

OpenAI AI provider configuration.

Attributes:

Name	Type	Description
`api_key`	`SecretStr`	API key for authentication.
`base_url`	`str \| None`	Custom base URL for OpenAI-compatible APIs (e.g., Ollama, LM Studio, Azure OpenAI, or other providers). If None, uses the default OpenAI API.
`model`	`str`	Model identifier to use.
`max_tokens`	`int`	Maximum tokens in the response.
`temperature`	`float`	Sampling temperature.

model `instance-attribute` ¶

model

Model identifier. Required so upgrading RoomKit cannot silently change a caller's model, cost, latency, or behavior.

timeout `class-attribute` `instance-attribute` ¶

timeout = 30.0

HTTP request timeout in seconds. Override for servers that need longer (e.g. Ollama cold-starting a model on first request).

connect_timeout `class-attribute` `instance-attribute` ¶

connect_timeout = 5.0

TCP connect timeout in seconds, kept apart from timeout so a host that no longer accepts connections is given up on in seconds rather than after the read budget. The SDK's own default.

max_retries `class-attribute` `instance-attribute` ¶

max_retries = 0

SDK-level retry count. Default 0 because RoomKit's RetryPolicy handles retries at the right layer with proper backoff and fallback.

include_stream_usage `class-attribute` `instance-attribute` ¶

include_stream_usage = False

When True, request token usage in streaming responses via stream_options.include_usage. The usage is included in the final :class:StreamDone event.

use_max_completion_tokens `class-attribute` `instance-attribute` ¶

use_max_completion_tokens = False

Send the output cap as max_completion_tokens instead of the deprecated max_tokens. OpenAI's newer models (o-series, gpt-5, gpt-4.1) reject max_tokens outright. Leave False for OpenAI-compatible servers (vLLM, LM Studio, older Azure deployments) that only understand max_tokens. Official modern models are profiled automatically unless this field is explicitly set.

supports_custom_temperature `class-attribute` `instance-attribute` ¶

supports_custom_temperature = True

When False, temperature is omitted from requests. OpenAI's reasoning models (o-series, gpt-5) accept only the default temperature=1 and reject any other value with HTTP 400.

reasoning_effort `class-attribute` `instance-attribute` ¶

reasoning_effort = None

Reasoning depth for OpenAI reasoning models (o-series, gpt-5): "none" | "low" | "medium" | "high" | "xhigh" | "max" (availability varies by model). Controls how long the model reasons (quality vs latency/cost); the reasoning trace itself stays hidden in the Chat Completions API. None = the model's default. GPT-5.6 tool turns on OpenAI's endpoint use "none" because Chat Completions function tools reject that family at higher effective efforts. Only configure this for reasoning models — others reject the parameter.

default_headers `class-attribute` `instance-attribute` ¶

default_headers = None

Extra HTTP headers sent on every request, passed to the SDK's default_headers. Use for an OpenAI-compatible endpoint behind a reverse proxy that needs custom headers, or a non-Bearer Authorization scheme (e.g. Basic). None sends only the SDK's own headers; the api_key Bearer token is unaffected.

extra_body `class-attribute` `instance-attribute` ¶

extra_body = None

Extra JSON fields merged into every Chat Completions request body via the SDK's extra_body. The route for server-specific params the OpenAI schema omits — e.g. vLLM guided decoding (guided_json/guided_choice) and extra sampling (top_k, repetition_penalty, min_p). None sends a vanilla body.

model_post_init ¶

model_post_init(__context)

Apply safe defaults for modern models on OpenAI's own endpoint.

Usage¶

from roomkit.providers.openai.ai import OpenAIAIProvider
from roomkit.providers.openai.config import OpenAIConfig
from roomkit.channels.ai import AIChannel

config = OpenAIConfig(api_key="your-api-key", model="gpt-5.6-sol")
provider = OpenAIAIProvider(config)

ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[openai]

Cerebras Provider¶

CerebrasAIProvider ¶

CerebrasAIProvider(config)

Bases: OpenAIAIProvider

Cerebras chat, reasoning and tool calling, including streaming.

Reuses OpenAI's async client and RoomKit's response decoder, error mapping, token accounting and live /v1/models discovery. Cerebras-specific request parameters and historical reasoning are shaped here.

Example::

provider = CerebrasAIProvider(
    CerebrasConfig(api_key="...", model="gpt-oss-120b")
)

name `property` ¶

name

Stable provider name in streaming and non-streaming telemetry.

supports_vision `property` ¶

supports_vision

Report the configured model's capability; unknown ids default False.

available_models `classmethod` ¶

available_models()

Offline metadata; use list_models() for account availability.

CerebrasConfig ¶

Bases: OpenAIConfig

Configuration for Cerebras's OpenAI-compatible Chat Completions API.

Inherits connection settings, sampling, retries and extra_body from :class:OpenAIConfig. model is required; upgrading RoomKit never silently selects another model. Install with roomkit[cerebras].

base_url `class-attribute` `instance-attribute` ¶

base_url = 'https://api.cerebras.ai/v1'

Cerebras endpoint. May be overridden for a dedicated endpoint or proxy.

use_max_completion_tokens `class-attribute` `instance-attribute` ¶

use_max_completion_tokens = True

The output cap includes both reasoning and final-answer tokens.

include_stream_usage `class-attribute` `instance-attribute` ¶

include_stream_usage = False

Cerebras sends usage in the final chunk without stream_options. Leave False for Cerebras; the inherited decoder still collects usage.

reasoning_effort `class-attribute` `instance-attribute` ¶

reasoning_effort = None

Reasoning effort, also sent on tool turns. Supported values depend on the model: GPT OSS accepts low/medium/high; Qwen 3.8 and Gemma 4 also accept none. A per-turn AIContext.reasoning_effort takes precedence. Token-based thinking_budget is not mapped to an effort level.

reasoning_format `class-attribute` `instance-attribute` ¶

reasoning_format = 'parsed'

Use parsed reasoning to keep thinking separate from user-visible text. Raw output cannot always be separated (GPT OSS concatenates it without delimiters). Supported formats vary by model; None omits the parameter.

clear_thinking `class-attribute` `instance-attribute` ¶

clear_thinking = None

Whether to remove historical reasoning before prompting Qwen 3.8. None leaves the server default. Only set for models supporting this field.

Install with pip install "roomkit[cerebras]". The provider uses the shared OpenAI-compatible async transport and supports text, streaming, reasoning, function calls and image input on vision-capable models.

import os

from roomkit import AIChannel, CerebrasAIProvider, CerebrasConfig

provider = CerebrasAIProvider(CerebrasConfig(
    api_key=os.environ["CEREBRAS_API_KEY"],
    model="gpt-oss-120b",
    reasoning_effort="low",
))
ai = AIChannel("assistant", provider=provider)

Choose model explicitly. await provider.list_models() queries the models available to the account; CerebrasAIProvider.available_models() supplies offline capabilities, context limits and dated prices. The snapshot includes GPT OSS 120B, Qwen 3.8 27B and Gemma 4 31B; Gemma may require a dedicated endpoint. Qwen uses a conservative 65,536-token limit covering the trial tier. Its model card advertises a larger window on paid tiers. Unknown model ids remain usable, with unknown context size and vision disabled.

reasoning_effort remains active when tools are supplied; a per-turn AIContext.reasoning_effort overrides the provider setting. GPT OSS accepts low, medium or high. Qwen 3.8 also accepts none to disable reasoning. thinking_budget is not translated into an effort level. Available values depend on the selected model, as described in the Cerebras reasoning guide.

reasoning_format="parsed" is the default, keeping thinking separate from answer text. Historical AIThinkingPart values are sent in the assistant's reasoning field, including across tool rounds. clear_thinking is optional and should only be set for a model supporting it. Selecting raw can mix reasoning into visible text; GPT OSS supplies no separator in that mode.

The output cap uses max_completion_tokens. Token usage is collected from Cerebras's final streaming chunk without requesting stream_options. Cache reads are reported separately and priced at the ordinary input rate. Errors and latency metrics identify the provider as cerebras; SDK retries default to zero so RoomKit's retry policy remains in control.

See examples/cerebras_ai.py for a complete conversation.

Mistral Provider¶

MistralAIProvider ¶

MistralAIProvider(config)

Bases: AIProvider

AI provider using the Mistral AI API.

Supports streaming, tool calling, vision (multimodal models), and <think> tag parsing for reasoning models.

available_models `classmethod` ¶

available_models()

Curated, offline catalog of Mistral chat/multimodal models.

list_models `async` ¶

list_models()

List models the Mistral API currently exposes for this key.

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events with <think> tag parsing.

Text inside <think>...</think> is yielded as :class:StreamThinkingDelta; everything else as :class:StreamTextDelta. Tool calls are accumulated from deltas and yielded as :class:StreamToolCall.

generate `async` ¶

generate(context)

Generate by consuming the structured stream.

generate_stream `async` ¶

generate_stream(context)

Yield text deltas as they arrive from the Mistral API.

close `async` ¶

close()

Close the underlying HTTP client.

MistralConfig ¶

Bases: BaseModel

Mistral AI provider configuration.

Attributes:

Name	Type	Description
`api_key`	`SecretStr`	Mistral API key for authentication.
`model`	`str`	Model identifier (e.g. `'mistral-large-latest'`, `'pixtral-large-latest'`).
`max_tokens`	`int`	Maximum tokens in the response.
`temperature`	`float`	Sampling temperature.
`server_url`	`str \| None`	Custom base URL for Mistral-compatible APIs. If `None`, uses the default Mistral endpoint.
`reasoning_effort`	`str \| None`	Reasoning effort for models that expose it (`mistral-small-latest`, `mistral-medium-3-5`): `"high"` streams a reasoning trace before the answer, `"none"` omits it. `None` leaves it to the model's default (Magistral models always reason). Overridden per-turn by `AIContext.thinking_budget`.

Usage¶

from roomkit.providers.mistral.ai import MistralAIProvider
from roomkit.providers.mistral.config import MistralConfig
from roomkit.channels.ai import AIChannel

config = MistralConfig(api_key="your-api-key")
provider = MistralAIProvider(config)

ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[mistral]

Azure Provider¶

AzureAIProvider ¶

AzureAIProvider(config)

Bases: OpenAIAIProvider

AI provider using Azure AI Studio's OpenAI-compatible Chat Completions API.

Subclasses :class:OpenAIAIProvider — only client initialisation and provider name differ. All message building, tool handling, response parsing, and streaming are inherited.

available_models `classmethod` ¶

available_models()

Azure exposes user-named deployments, not a fixed model catalog.

Deployment names are chosen per Azure resource, so there is no meaningful offline list — use :meth:list_models for the live set.

AzureAIConfig ¶

Bases: BaseModel

Azure AI Studio provider configuration.

Uses the OpenAI-compatible Chat Completions API exposed by Azure AI Foundry deployments (DeepSeek, GPT-4o, Mistral, etc.).

Attributes:

Name	Type	Description
`api_key`	`SecretStr`	Azure API key for authentication.
`azure_endpoint`	`str`	Azure AI Foundry project endpoint URL.
`api_version`	`str`	Azure API version string.
`model`	`str`	Deployment name (no default — user must specify).
`max_tokens`	`int`	Maximum tokens in the response.
`temperature`	`float`	Sampling temperature.
`timeout`	`float`	HTTP request timeout in seconds.
`connect_timeout`	`float`	TCP connect timeout in seconds, kept apart from `timeout` so a host that no longer accepts connections is given up on in seconds rather than after the read budget.

max_retries `class-attribute` `instance-attribute` ¶

max_retries = 0

SDK-level retry count. Default 0 because RoomKit's RetryPolicy handles retries at the right layer with proper backoff and fallback.

include_stream_usage `class-attribute` `instance-attribute` ¶

include_stream_usage = False

When True, request token usage in streaming responses.

use_max_completion_tokens `class-attribute` `instance-attribute` ¶

use_max_completion_tokens = False

Send the output cap as max_completion_tokens rather than the deprecated max_tokens. Required by newer Azure-hosted OpenAI models; leave False for deployments that only understand max_tokens.

supports_custom_temperature `class-attribute` `instance-attribute` ¶

supports_custom_temperature = True

When False, temperature is omitted — reasoning deployments accept only the default and reject any other value with HTTP 400.

reasoning_effort `class-attribute` `instance-attribute` ¶

reasoning_effort = None

Reasoning depth for reasoning deployments ("low"/"medium"/ "high"); None uses the model default. Only sent for models that accept it.

extra_body `class-attribute` `instance-attribute` ¶

extra_body = None

Extra JSON fields merged into every request body via the SDK's extra_body — for deployment-specific params the OpenAI schema omits. None sends a vanilla body.

Usage¶

from roomkit.providers.azure.ai import AzureAIProvider
from roomkit.providers.azure.config import AzureAIConfig
from roomkit.channels.ai import AIChannel

config = AzureAIConfig(
    azure_endpoint="https://your-resource.openai.azure.com/",
    api_key="your-api-key",
    deployment="your-deployment-name",
)
provider = AzureAIProvider(config)

ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[azure]

Ollama Provider (Local / Cloud LLM)¶

Native provider for Ollama. Calls /api/chat directly, so the think parameter and streamed reasoning work without <think> tag parsing. See AI Thinking — Native Ollama provider for thinking, authentication, and sampling options (temperature, num_ctx, top_p, top_k, min_p, keep_alive).

OllamaAIProvider ¶

OllamaAIProvider(config)

Bases: AIProvider

AI provider using Ollama's native API via the ollama-python SDK.

available_models `classmethod` ¶

available_models()

Curated, offline snapshot of popular public Ollama models.

list_models `async` ¶

list_models()

List models installed on the configured Ollama server.

Reads /api/tags for the installed set, then probes /api/show per model (bounded parallel fan-out) to attach capabilities — the tags the picker uses to keep completion-only models out of an embeddings channel and vice-versa. A per-model probe failure yields no capabilities for that model (older servers don't ship the field); consumers treat empty as "unknown, allow everywhere".

generate_stream `async` ¶

generate_stream(context)

Yield text deltas (thinking content filtered out).

generate_structured_stream `async` ¶

generate_structured_stream(context)

Yield structured events with thinking streamed separately.

Ollama's native streaming emits one chunk per token-ish, each with message.thinking and/or message.content deltas plus an optional final message.tool_calls. We pass these straight through as the corresponding StreamThinkingDelta, StreamTextDelta, and StreamToolCall events — no tag parsing, no field reordering.

close `async` ¶

close()

Release the underlying httpx client.

OllamaConfig ¶

Bases: BaseModel

Ollama AI provider configuration.

Wraps the native Ollama /api/chat endpoint via the ollama-python SDK. Prefer this over the OpenAI-compatible shim when the model is a reasoning model (DeepSeek-R1, Qwen 3 thinking variants, etc.) because Ollama exposes the think parameter and streams the reasoning content as a separate thinking field — both ignored by the OpenAI-compat endpoint.

Attributes:

Name	Type	Description
`host`	`str`	Base URL of the Ollama server. Default points at the local daemon. The native API lives under `/api` on the same host; the SDK appends the path.
`model`	`str`	Model identifier to use (e.g. `"qwen3:8b"`, `"llama3.2"`, `"deepseek-r1:7b"`).
`max_tokens`	`int \| None`	Maximum tokens to generate in the response. Maps to Ollama's `options.num_predict`. `None` lets the server pick its default.
`temperature`	`float`	Sampling temperature. Maps to `options.temperature`.
`timeout`	`float`	HTTP request timeout in seconds. Long default because local models cold-start on first request and reasoning models can take 30-60s before the first token.
`connect_timeout`	`float`	TCP connect timeout in seconds, kept apart from `timeout` so a host that no longer accepts connections is given up on in seconds rather than after the read budget.
`max_retries`	`int`	SDK-level retry count. Default 0 because RoomKit's RetryPolicy handles retries at the right layer with proper backoff and fallback.
`think`	`bool \| ThinkEffort \| None`	Whether and how hard the model should reason before answering. `None` (default) means "use the model's default" — reasoning models think, others don't. `True`/`False` force thinking on or off as a boolean. One of `"low"`, `"medium"`, `"high"` selects an effort level for models that support it (Ollama 0.7+ on reasoning-capable models like gpt-oss and deepseek-r1). Effort strings pass straight through to the Ollama API; unsupported models silently downgrade to boolean behavior. `AIContext.thinking_budget` overrides this at request time: `None`/`0` → `think=False`, `>0` → uses this config value if it's a string, otherwise `think=True`.
`keep_alive`	`str \| int \| None`	How long the model stays loaded in memory after the request. Maps to Ollama's `keep_alive` parameter. A duration string with a unit (`"5m"`, `"30s"`) or an integer number of seconds — `-1` keeps the model loaded indefinitely, `0` unloads it immediately. A unit-less numeric string (e.g. `"-1"` from a text field) is coerced to an int, because Ollama parses a string as a Go duration and rejects one without a unit. `None` uses the server default (5 minutes).
`num_ctx`	`int \| None`	Context window size. Maps to `options.num_ctx`. `None` uses the model's default (typically 2048 — bump for long contexts).
`top_p`	`float \| None`	Nucleus sampling cutoff. Maps to `options.top_p`. `None` uses the model's default.
`top_k`	`int \| None`	Top-k sampling cutoff. Maps to `options.top_k`. `None` uses the model's default.
`min_p`	`float \| None`	Minimum-probability sampling cutoff. Maps to `options.min_p`. `None` uses the model's default.
`api_key`	`SecretStr \| None`	Bearer token for a protected Ollama endpoint — Ollama Cloud/Turbo, or a self-hosted server behind a reverse proxy that checks `Authorization: Bearer`. Sent as the `Authorization` header. `None` (default) leaves auth to the SDK, which still falls back to the `OLLAMA_API_KEY` environment variable when it is set. Prefer this field when the key comes from a secret manager rather than the process environment.
`headers`	`dict[str, str] \| None`	Extra HTTP headers attached to every request — for a reverse proxy that needs custom headers, or a non-Bearer `Authorization` scheme (e.g. Basic). `api_key` takes precedence over an `Authorization` entry supplied here. `None` (default) sends only the SDK's own headers.

Usage¶

from roomkit.providers.ollama import OllamaAIProvider, OllamaConfig
from roomkit.channels.ai import AIChannel

config = OllamaConfig(host="http://localhost:11434", model="llama3.2")
provider = OllamaAIProvider(config)

ai_channel = AIChannel("ai", provider=provider)

Install with: pip install roomkit[ollama]

Streaming¶

StreamEvent `module-attribute` ¶

StreamEvent = StreamThinkingDelta | StreamTextDelta | StreamToolCallDelta | StreamToolCall | StreamDone

StreamTextDelta ¶

Bases: BaseModel

A text delta from a streaming AI response.

StreamThinkingDelta ¶

Bases: BaseModel

A thinking/reasoning delta from a streaming AI response.

Emitted before text deltas when the model is reasoning. A delta may carry only a signature (with empty thinking): Anthropic streams the thinking block's opaque signature separately, and it must be preserved so the block can be echoed back in history without a 400.

StreamToolCall ¶

Bases: BaseModel

A complete tool call extracted from a streaming AI response.

StreamDone ¶

Bases: BaseModel

Signals the end of a streaming AI response.

Response Parts¶

AIThinkingPart ¶

Bases: BaseModel

AI reasoning/thinking block in conversation history.

Used to preserve thinking blocks across tool-loop turns (required by providers like Anthropic that mandate round-trip fidelity).

Attributes:

Name	Type	Description
`thinking`	`str`	The reasoning text produced by the model.
`signature`	`str \| None`	Provider-specific opaque token for caching/validation (e.g. Anthropic's thinking block signature).

AIToolCallPart ¶

Bases: BaseModel

Assistant's tool call in conversation history.

AIToolResultPart ¶

Bases: BaseModel

Tool execution result in conversation history.

result is a plain string for text results, or a list of content parts (text and/or image) when a tool returns multimodal output — e.g. an edge tool that returns a screenshot. Providers that support image tool results (Anthropic) render the parts as content blocks; the rest flatten via as_text().

as_text ¶

as_text()

Flatten the result to plain text for providers without image support.

A string result is returned unchanged; a list concatenates its text parts and substitutes a [image] placeholder for each image part.

split_for_message ¶

split_for_message()

Split the result into the tool-message text and its image parts.

Unlike Anthropic — whose Messages API accepts image blocks inside a tool_result — most providers reject image content in a tool / function-response message; the image has to ride on a following user message instead. This returns the text to keep on the tool message (text parts joined, or a "[see image below]" placeholder when the result was image-only, so the tool-call/result pairing stays non-empty and valid) together with the image parts to render natively elsewhere.

A string result yields (result, []) and a text-only list yields (joined_text, []) — the no-op path that keeps every existing text tool byte-for-byte unchanged. Only a list carrying an image populates the second element and triggers a provider's synthetic-image path.

ProviderError ¶

ProviderError(message, *, retryable=False, provider='', status_code=None, context_overflow=None)

Bases: Exception

Error from an AI provider SDK call.

Attributes:

Name	Type	Description
`retryable`		Whether the caller should retry the request.
`provider`		Name of the provider that raised the error.
`status_code`		HTTP status code from the provider, if available.
`context_overflow`		Did the request exceed the model's context window? Tri-state. `True` and `False` are a structural classification (measurement, an error code) and are believed as stated, in both directions. `None` means nobody classified, and the message wording decides as a fallback — an envelope may rewrap the provider's prose, and prose must never override an explicit answer.

AI Providers¶

AIProvider ¶

name property ¶

supports_vision property ¶

supports_streaming property ¶

supports_structured_streaming property ¶

model_name abstractmethod property ¶

context_window property ¶

available_models classmethod ¶

list_models async ¶

catalog_entry ¶

generate abstractmethod async ¶

generate_stream async ¶

generate_structured_stream async ¶

close async ¶

AIContext ¶

max_tokens class-attribute instance-attribute ¶

enable_thinking class-attribute instance-attribute ¶

reasoning_effort class-attribute instance-attribute ¶

model_post_init ¶

model_copy ¶

AIMessage ¶

AITextPart ¶

AIImagePart ¶

AITool ¶

AIToolCall ¶

AIResponse ¶

ModelInfo ¶

ModelPricing ¶

cost_for ¶

MockAIProvider ¶

available_models classmethod ¶

generate_stream async ¶

generate_structured_stream async ¶

Image parts¶

image_part_payload ¶

image_part_base64 ¶

image_part_uri ¶

Explicit models and modern request profiling¶

Per-Room AI Configuration¶

Silent Observer Pattern (Meeting Notes)¶

Tools/Function Calling¶

Gemini Provider¶

GeminiAIProvider ¶

supports_vision property ¶

available_models classmethod ¶

list_models async ¶

generate_structured_stream async ¶

generate async ¶

generate_stream async ¶

close async ¶

GeminiConfig ¶

thinking_level class-attribute instance-attribute ¶

timeout class-attribute instance-attribute ¶

connect_timeout class-attribute instance-attribute ¶

GeminiVertexProvider ¶

GeminiVertexConfig ¶

api_key class-attribute instance-attribute ¶

project instance-attribute ¶

location instance-attribute ¶

service_account_json class-attribute instance-attribute ¶

impersonate_service_account class-attribute instance-attribute ¶

labels class-attribute instance-attribute ¶

Usage¶

Schema Cleaning¶

clean_gemini_schema ¶

vLLM Provider (Local LLM)¶

VLLMConfig ¶

max_retries class-attribute instance-attribute ¶

include_stream_usage class-attribute instance-attribute ¶

headers class-attribute instance-attribute ¶

top_p class-attribute instance-attribute ¶

top_k class-attribute instance-attribute ¶

min_p class-attribute instance-attribute ¶

presence_penalty class-attribute instance-attribute ¶

repetition_penalty class-attribute instance-attribute ¶

extra_body class-attribute instance-attribute ¶

enable_thinking class-attribute instance-attribute ¶

reasoning_effort class-attribute instance-attribute ¶

sampling_body ¶

name `property` ¶

supports_vision `property` ¶

supports_streaming `property` ¶

supports_structured_streaming `property` ¶

model_name `abstractmethod` `property` ¶

context_window `property` ¶

available_models `classmethod` ¶

list_models `async` ¶

generate `abstractmethod` `async` ¶

generate_stream `async` ¶

generate_structured_stream `async` ¶

close `async` ¶

max_tokens `class-attribute` `instance-attribute` ¶

enable_thinking `class-attribute` `instance-attribute` ¶

reasoning_effort `class-attribute` `instance-attribute` ¶

available_models `classmethod` ¶

generate_stream `async` ¶

generate_structured_stream `async` ¶

supports_vision `property` ¶

available_models `classmethod` ¶

list_models `async` ¶

generate_structured_stream `async` ¶

generate `async` ¶

generate_stream `async` ¶

close `async` ¶

thinking_level `class-attribute` `instance-attribute` ¶

timeout `class-attribute` `instance-attribute` ¶

connect_timeout `class-attribute` `instance-attribute` ¶

api_key `class-attribute` `instance-attribute` ¶

project `instance-attribute` ¶

location `instance-attribute` ¶

service_account_json `class-attribute` `instance-attribute` ¶

impersonate_service_account `class-attribute` `instance-attribute` ¶

labels `class-attribute` `instance-attribute` ¶

max_retries `class-attribute` `instance-attribute` ¶

include_stream_usage `class-attribute` `instance-attribute` ¶

headers `class-attribute` `instance-attribute` ¶

top_p `class-attribute` `instance-attribute` ¶

top_k `class-attribute` `instance-attribute` ¶

min_p `class-attribute` `instance-attribute` ¶

presence_penalty `class-attribute` `instance-attribute` ¶

repetition_penalty `class-attribute` `instance-attribute` ¶

extra_body `class-attribute` `instance-attribute` ¶

enable_thinking `class-attribute` `instance-attribute` ¶

reasoning_effort `class-attribute` `instance-attribute` ¶

supports_vision `property` ¶

available_models `classmethod` ¶

list_models `async` ¶

generate_structured_stream `async` ¶

generate `async` ¶

generate_stream `async` ¶

close `async` ¶

model `instance-attribute` ¶

timeout `class-attribute` `instance-attribute` ¶

connect_timeout `class-attribute` `instance-attribute` ¶

base_url `class-attribute` `instance-attribute` ¶

extra_headers `class-attribute` `instance-attribute` ¶

enable_prompt_caching `class-attribute` `instance-attribute` ¶

use_adaptive_thinking `class-attribute` `instance-attribute` ¶

supports_custom_temperature `class-attribute` `instance-attribute` ¶

supports_vision `property` ¶

available_models `classmethod` ¶

list_models `async` ¶

generate_structured_stream `async` ¶

generate_stream `async` ¶

close `async` ¶

model `instance-attribute` ¶

timeout `class-attribute` `instance-attribute` ¶

connect_timeout `class-attribute` `instance-attribute` ¶

max_retries `class-attribute` `instance-attribute` ¶

include_stream_usage `class-attribute` `instance-attribute` ¶

use_max_completion_tokens `class-attribute` `instance-attribute` ¶

supports_custom_temperature `class-attribute` `instance-attribute` ¶

reasoning_effort `class-attribute` `instance-attribute` ¶

default_headers `class-attribute` `instance-attribute` ¶

extra_body `class-attribute` `instance-attribute` ¶

name `property` ¶

supports_vision `property` ¶

available_models `classmethod` ¶

base_url `class-attribute` `instance-attribute` ¶