LLM Providers¶

CRP is provider-agnostic. It works with any LLM through a standard adapter interface.

Auto-Detection¶

When no provider is specified, CRP checks (in order):

OPENAI_API_KEY environment variable → OpenAI
ANTHROPIC_API_KEY environment variable → Anthropic
Ollama server at localhost:11434 → Ollama
Model name pattern matching (e.g., gpt-* → OpenAI, claude-* → Anthropic)

# Auto-detect from environment
client = crp.Client()

# Auto-detect from model name
client = crp.Client(model="gpt-4o-mini")
client = crp.Client(model="claude-sonnet-4-20250514")

OpenAI¶

from crp.providers import OpenAIAdapter

provider = OpenAIAdapter(model="gpt-4o-mini")
client = crp.Client(provider=provider)

Supports: GPT-4o, GPT-4o-mini, GPT-4, o1, o3, and all OpenAI chat models. Also works with Azure OpenAI via the openai SDK's Azure configuration.

Anthropic¶

from crp.providers import AnthropicAdapter

provider = AnthropicAdapter(model="claude-sonnet-4-20250514")
client = crp.Client(provider=provider)

Supports: Claude Opus, Claude Sonnet, Claude Haiku, and all Anthropic chat models.

Ollama (Local)¶

from crp.providers import OllamaAdapter

provider = OllamaAdapter(model="llama3.1")
client = crp.Client(provider=provider)

Requires Ollama running locally. No API key needed. Supports any model available in your Ollama installation.

llama.cpp¶

from crp.providers import LlamaCppAdapter

provider = LlamaCppAdapter(model_path="/path/to/model.gguf")
client = crp.Client(provider=provider)

Direct integration with llama.cpp for maximum control over local inference.

Custom Provider¶

Build your own provider for any LLM backend:

from crp.providers import CustomProvider

def my_generate(messages):
    # Call your LLM API
    response = my_api.chat(messages)
    return response.text, response.finish_reason

def my_tokenizer(text):
    return len(text.split())  # Simple word-count tokenizer

provider = CustomProvider(
    generate_fn=my_generate,
    count_tokens_fn=my_tokenizer,
    context_size=128_000,
)
client = crp.Client(provider=provider)

Provider Interface¶

All providers implement the LLMProvider abstract base class:

Method	Required	Description
`generate_chat(messages, **kwargs)`	Yes	Generate a response. Returns `(output, finish_reason)`
`count_tokens(text)`	Yes	Count tokens in text
`context_window_size()`	Yes	Return max context window in tokens
`supports_tools()`	No	Whether the provider supports tool/function calling
`generate_chat_stream(messages, **kwargs)`	No	Streaming generation
`cost_per_1k_tokens()`	No	Returns `(input_cost, output_cost)` per 1K tokens