Skip to main content

Providers API

The client.providers namespace exposes provider discovery, adapter selection, and the base LLMProvider contract used by every backend.

SDK proxy

crp.sdk.proxies._ProvidersProxy

LLM provider registry proxy (SPEC-008).

Registers providers, returns the current default provider, and lists supported provider names.

register(provider)

Register provider on the orchestrator if possible.

Parameters:

Name	Type	Description	Default
provider	Any	An LLMProvider instance.	required

default()

Return the orchestrator's current default provider.

list_supported()

Return the list of known/supported provider names.

LLM Provider base class

crp.providers.base.LLMProvider

Bases: ABC

Abstract interface for LLM backends.

Implementations: OpenAIAdapter, AnthropicAdapter, LlamaCppAdapter, CustomProvider.

Subclasses must implement the three core abstract methods: generate_chat, count_tokens, and context_window_size. The remaining methods provide optional metadata, cost estimates, and tool or streaming support.

max_output_tokens property

Optional provider-reported max output tokens.

Returns:

Type	Description
int | None	Maximum output tokens, or None if not known.

model_name property

Human-readable model identifier for diagnostics.

Returns:

Type	Description
str	Class name by default; subclasses may override with a specific
str	model identifier.

is_thinking_model property

Return True if the model produces reasoning_content alongside output.

Thinking models (qwen3, deepseek-r1, o1, etc.) spend a significant portion of tokens on internal reasoning, requiring a larger generation reserve to avoid starving the final content output.

Returns:

Type	Description
bool	True when the provider represents a thinking model.

generate_chat(messages, **kwargs) abstractmethod

Generate text completion from a message array.

Parameters:

Name	Type	Description	Default
messages	list[dict[str, str]]	List of chat messages, e.g. [{"role": "system", "content": "..."}, ...].	required
**kwargs	object	Provider-specific overrides (max_tokens, temperature, etc.)	{}

Returns:

Type	Description
tuple[str, str]	(output_text, finish_reason) where finish_reason is one of: - "length" - model hit generation reserve limit (physical wall) - "stop" - model produced natural EOS - "error" - provider returned an error

count_tokens(text) abstractmethod

Count tokens in text using this provider's exact tokenizer.

MUST return accurate per-model counts - no estimates.

Parameters:

Name	Type	Description	Default
text	str	Text to tokenise.	required

Returns:

Type	Description
int	Number of tokens in text.

context_window_size() abstractmethod

Return the model's physical context window size in tokens.

E.g. 128_000 for Claude 3 Opus, 8_192 for GPT-3.5.

Returns:

Type	Description
int	Context window size in tokens.

cost_per_1k_tokens()

Return (input_cost_per_1k, output_cost_per_1k) in USD.

Returns:

Type	Description
float	(input_cost, output_cost) per 1,000 tokens. Defaults to
float	(0.0, 0.0) for local models or unknown pricing.

supports_tools()

Return True if this provider supports function/tool calling.

Override in subclasses that implement generate_chat_with_tools(). The orchestrator uses this to decide between push (envelope) and pull (tool-mediated) context relay.

Returns:

Type	Description
bool	True when tool-mediated dispatch is supported.

generate_chat_with_tools(messages, tools, **kwargs)

Generate with tool/function calling support.

Parameters:

Name	Type	Description	Default
messages	list[dict[str, object]]	Chat messages (may include tool role messages).	required
tools	list[dict[str, object]]	Tool definitions in OpenAI-compatible format.	required
**kwargs	object	Provider-specific overrides.	{}

Returns:

Name	Type	Description
	str	(output_text, finish_reason, tool_calls, raw_assistant_message)
where	str
	list[dict[str, object]] | None	output_text: Generated text content (may be empty if tool_calls)
	dict[str, object] | None	finish_reason: "stop", "length", "tool_calls", or "error"
	tuple[str, str, list[dict[str, object]] | None, dict[str, object] | None]	tool_calls: List of tool call dicts if finish_reason=="tool_calls"
	tuple[str, str, list[dict[str, object]] | None, dict[str, object] | None]	raw_assistant_message: The full assistant message dict (for appending to conversation history in the tool loop)

Raises:

Type	Description
NotImplementedError	By default; subclasses must override this method when supports_tools() returns True.

generate_chat_stream(messages, **kwargs)

Stream token chunks from the LLM.

Yields individual token chunks as strings. The return value (accessible via StopIteration.value) is the finish_reason ("stop" or "length").

Default implementation falls back to generate_chat() and yields the full output as a single chunk. Override in subclasses for real streaming.

Parameters:

Name	Type	Description	Default
messages	list[dict[str, str]]	Chat messages.	required
**kwargs	object	Provider-specific overrides.	{}

Yields:

Type	Description
str	Token chunks as strings.

Returns:

Type	Description
str	The finish reason string.