crp.resources¶
Auto-generated reference for the crp.resources subpackage.
resources¶
crp.resources ¶
Resource management - allocation, monitoring, model registry, cost model.
AdaptiveAllocator ¶
Dynamically tunes CRP pipeline based on real-time overhead and pressure.
The allocator observes per-window overhead ratios and resource pressure, then adjusts throughput parameters and prompt efficiency hints to keep overhead below the configured cap.
Core principle: ML extraction stages (GLiNER, UIE, Discourse) are NEVER disabled - they are the protocol's core intelligence. Under pressure, the system reduces fact counts and increases batch sizes (graceful throttling) rather than cutting analytical capability.
Usage::
alloc = AdaptiveAllocator(
resource_manager=rm,
overhead_manager=om,
overhead_cap_pct=15.0,
)
# Before each dispatch - get recommended profiles
ext_profile = alloc.extraction_profile()
env_profile = alloc.envelope_profile()
efficiency = alloc.prompt_efficiency()
# After each dispatch - record overhead
alloc.record_window(total_ms=1200, llm_ms=1050,
envelope_ms=80, extraction_ms=50)
# Cleanup check
if alloc.should_unload_models():
for model in alloc.idle_models():
unload(model)
ewma_overhead_pct property ¶
Return the ewma overhead pct.
window_count property ¶
Return the current window count.
consecutive_over_cap property ¶
Return the consecutive over cap.
disabled_stages property ¶
ML stages are never disabled - always returns empty set.
throughput_level property ¶
Current throughput level: normal, throttled, or constrained.
hardware property ¶
Return the hardware.
record_window(total_ms, llm_ms, *, envelope_ms=0.0, extraction_ms=0.0) ¶
Record overhead from a completed window and adapt.
extraction_profile() ¶
Get recommended extraction profile based on current state.
ML stages (3=GLiNER, 4=UIE, 5=Discourse) are ALWAYS enabled. Under pressure, throughput is throttled via max_facts_per_stage.
envelope_profile() ¶
Get recommended envelope profile based on current state.
CKF (Contextual Knowledge Fusion) is ALWAYS enabled - it is core to CRP's knowledge retrieval. Under pressure, batch sizes increase and packing limits decrease to reduce overhead.
should_unload_models() ¶
Check if idle models should be unloaded.
idle_models() ¶
Get list of models eligible for unloading.
should_run_gc() ¶
Check if garbage collection should run.
overhead_trend(last_n=5) ¶
Return recent overhead percentages.
prompt_efficiency() ¶
Get LLM prompt efficiency recommendations.
After 2+ windows the system prompt is stable and should be cached. Under constrained throughput, envelope compression is recommended.
summary() ¶
Full allocator state for diagnostics.
EnvelopeProfile dataclass ¶
Recommended envelope configuration based on resource state.
ExtractionProfile dataclass ¶
Recommended extraction configuration based on resource state.
WindowOverheadRecord dataclass ¶
Per-window overhead measurement.
BudgetWarning dataclass ¶
A budget warning emitted when approaching limits.
BudgetWarningLevel ¶
Bases: str, Enum
Budget warning levels.
CostModel ¶
Real-time cost tracking with budget enforcement (§6.8).
Tracks per-window, per-session, and cumulative costs. Warns at 80%, hard-stops at 100% of user-set budgets.
pricing property ¶
Return the pricing.
total_windows property ¶
Return the total windows.
total_input_tokens property ¶
Return the total input tokens.
total_output_tokens property ¶
Return the total output tokens.
total_cost_usd property ¶
Return the total cost usd.
warnings property ¶
Return the warnings.
record_window(window_id, input_tokens, output_tokens, is_overhead=False, feature_name='') ¶
Record a completed window's cost. Returns the WindowCost record.
check_budget(input_tokens=0) ¶
Check all budget caps. Raises BudgetExhaustedError if exceeded.
Returns any warnings generated (80%+ of cap).
estimate(planned_dispatches=1, avg_input_tokens=0, avg_output_tokens=0) ¶
Pre-flight cost estimation.
reset() ¶
Reset all counters (on session start).
OverheadBudget dataclass ¶
Caps total protocol overhead across all features (§6.9).
Default: 15% of productive windows. Feature shedding: lowest priority shed first. Never shed: extraction (always runs) and Tier 1 validation (zero LLM cost).
OverheadDecision ¶
Bases: str, Enum
Result of overhead budget check.
ProviderPricing dataclass ¶
Token pricing for an LLM provider (USD per 1M tokens).
WindowCost dataclass ¶
Cost record for a single window.
OverheadBudgetManager ¶
Orchestrates feature shedding and lazy embedding batching.
Usage::
mgr = OverheadBudgetManager(max_overhead_pct=15, batch_size=32)
# After each window, update overhead ratio:
mgr.update_overhead(current_pct=12.5)
assert mgr.is_feature_enabled("gliner")
# When overhead spikes:
mgr.update_overhead(current_pct=18.0)
assert not mgr.is_feature_enabled("community_detection") # shed first
# Lazy embedding:
mgr.defer_embedding("f1", "some fact text")
mgr.defer_embedding("f2", "another fact text")
batch = mgr.flush_embeddings() # returns pending, clears queue
shed_log property ¶
Return the shed log.
enabled_features property ¶
Return the enabled features.
pending_embedding_count property ¶
Return the current pending embedding count.
current_overhead_pct property ¶
Return the current overhead pct.
max_overhead_pct property ¶
Return the max overhead pct.
update_overhead(current_pct) ¶
Update the current overhead percentage and shed/restore features.
Returns a list of features that changed state (shed or restored).
is_feature_enabled(name) ¶
Check if a feature is currently enabled.
defer_embedding(fact_id, text) ¶
Queue a fact for deferred embedding.
Returns True if the batch is now full and should be flushed.
flush_embeddings() ¶
Return all pending embeddings as (fact_id, text) and clear queue.
summary() ¶
Return a JSON-serializable summary of the manager state.
DeviceProfile dataclass ¶
The detected (or supplied) device characteristics.
DeviceTier ¶
Bases: str, Enum
Coarse device class used to pick a resource plan.
ResourceGovernor ¶
Picks a resource plan that holds utilisation under the target.
detect_device(target_utilisation=0.4) staticmethod ¶
Detect the device tier from CPU/RAM (psutil optional; safe default constrained).
plan(requested_profile=None) ¶
Return a plan for the current device, capping any requested profile.
The Governor never upsizes beyond the device tier's ceiling. If the caller requests a smaller (more frugal) profile, that is honoured - frugality is always allowed; exceeding the tier ceiling is not.
ResourcePlan dataclass ¶
The Governor's plan for a run - the protocol levers, not engine settings.
ResourceManager ¶
Centralized resource tracking and pressure management.
Usage::
rm = ResourceManager(budget_mb=512)
rm.register_model("sentence-transformers", 150)
rm.mark_model_loaded("sentence-transformers")
snap = rm.snapshot()
print(snap.pressure_level)
register_model(name, estimated_mb) ¶
Register an ML model (call at init, before loading).
mark_model_loaded(name) ¶
Mark a model as loaded into memory.
mark_model_used(name) ¶
Record model usage (resets idle timer).
mark_model_unloaded(name) ¶
Mark a model as unloaded from memory.
update_fact_count(count) ¶
Update the tracked fact-store fact count.
update_session_count(count) ¶
Update the tracked active session count.
snapshot() ¶
Take a point-in-time resource snapshot.
should_cleanup() ¶
Return True if pressure warrants cleanup actions.
run_gc() ¶
Run Python garbage collection and return the number of objects collected.
get_idle_models(idle_seconds=300.0) ¶
Return names of loaded models idle for longer than idle_seconds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
idle_seconds | float | Idle threshold in seconds. | 300.0 |
Returns:
| Type | Description |
|---|---|
list[str] | List of model names eligible for unload. |
mark_unloaded(model_name) ¶
Mark a model as unloaded (freed from memory).
trigger_gc() ¶
Run GC if pressure warrants it (alias for should_cleanup + run_gc).
summary() ¶
Return a human-readable resource utilization summary dict.
ResourceSnapshot dataclass ¶
Point-in-time resource utilization snapshot.
utilization_ratio property ¶
Return the utilization ratio.
detect_hardware() ¶
Detect available hardware resources (best-effort).
resources.adaptive_allocator¶
crp.resources.adaptive_allocator ¶
Adaptive resource allocator - efficient CRP pipeline without cutting intelligence.
Integrates ResourceManager (memory pressure) with OverheadBudgetManager (feature shedding) to dynamically adapt CRP's processing pipeline based on real-time performance measurements.
Design philosophy
CRP is an EFFICIENCY protocol, not a speed protocol. ML extraction stages (GLiNER, UIE, Discourse) are core intelligence and are NEVER disabled. Under pressure, the system throttles throughput (fewer facts per stage, larger batches, prompt caching) rather than cutting analytical capability.
Design goals
- Keep CRP overhead below a configurable cap (default 15% of wall time)
- Shed optional optimization features under pressure (community detection, cross-encoder reranking) while protecting ML intelligence
- Throttle throughput gracefully: reduce fact counts, increase batches
- Provide prompt-level efficiency hints (caching, dedup, connection reuse)
- Track per-window overhead history and adapt thresholds
- Provide hardware-aware defaults (CPU count, available RAM)
Efficiency strategies (applied in order of priority): Phase 1 - Prompt efficiency (caching, deduplication, connection reuse) Phase 2 - Feature shedding (community_detection, cross_encoder only) Phase 3 - Throughput throttling (reduce facts per stage, larger batches) Phase 4 - Model lifecycle (idle models freed after timeout) Phase 5 - GC sweep (Python garbage collection under memory pressure)
WindowOverheadRecord dataclass ¶
Per-window overhead measurement.
ExtractionProfile dataclass ¶
Recommended extraction configuration based on resource state.
EnvelopeProfile dataclass ¶
Recommended envelope configuration based on resource state.
PromptEfficiency dataclass ¶
LLM-side efficiency recommendations based on pipeline state.
These hints allow providers to optimize token usage, connection management, and prompt caching - reducing cost and latency without cutting any intelligence features.
AdaptiveAllocator ¶
Dynamically tunes CRP pipeline based on real-time overhead and pressure.
The allocator observes per-window overhead ratios and resource pressure, then adjusts throughput parameters and prompt efficiency hints to keep overhead below the configured cap.
Core principle: ML extraction stages (GLiNER, UIE, Discourse) are NEVER disabled - they are the protocol's core intelligence. Under pressure, the system reduces fact counts and increases batch sizes (graceful throttling) rather than cutting analytical capability.
Usage::
alloc = AdaptiveAllocator(
resource_manager=rm,
overhead_manager=om,
overhead_cap_pct=15.0,
)
# Before each dispatch - get recommended profiles
ext_profile = alloc.extraction_profile()
env_profile = alloc.envelope_profile()
efficiency = alloc.prompt_efficiency()
# After each dispatch - record overhead
alloc.record_window(total_ms=1200, llm_ms=1050,
envelope_ms=80, extraction_ms=50)
# Cleanup check
if alloc.should_unload_models():
for model in alloc.idle_models():
unload(model)
ewma_overhead_pct property ¶
Return the ewma overhead pct.
window_count property ¶
Return the current window count.
consecutive_over_cap property ¶
Return the consecutive over cap.
disabled_stages property ¶
ML stages are never disabled - always returns empty set.
throughput_level property ¶
Current throughput level: normal, throttled, or constrained.
hardware property ¶
Return the hardware.
record_window(total_ms, llm_ms, *, envelope_ms=0.0, extraction_ms=0.0) ¶
Record overhead from a completed window and adapt.
extraction_profile() ¶
Get recommended extraction profile based on current state.
ML stages (3=GLiNER, 4=UIE, 5=Discourse) are ALWAYS enabled. Under pressure, throughput is throttled via max_facts_per_stage.
envelope_profile() ¶
Get recommended envelope profile based on current state.
CKF (Contextual Knowledge Fusion) is ALWAYS enabled - it is core to CRP's knowledge retrieval. Under pressure, batch sizes increase and packing limits decrease to reduce overhead.
should_unload_models() ¶
Check if idle models should be unloaded.
idle_models() ¶
Get list of models eligible for unloading.
should_run_gc() ¶
Check if garbage collection should run.
overhead_trend(last_n=5) ¶
Return recent overhead percentages.
prompt_efficiency() ¶
Get LLM prompt efficiency recommendations.
After 2+ windows the system prompt is stable and should be cached. Under constrained throughput, envelope compression is recommended.
summary() ¶
Full allocator state for diagnostics.
detect_hardware() ¶
Detect available hardware resources (best-effort).
resources.cost_model¶
crp.resources.cost_model ¶
Provider pricing, real-time cost tracking, and budget enforcement (§6.8).
Budget invariants
- Warn at 80% of any cap.
- Hard stop at 100% → BudgetExhaustedError.
- OverheadBudget caps total overhead at 15%.
- Feature shedding cascade: review_tier3 → orc_steps → curation → re_grounding → review_tier2 (lowest priority shed first).
ProviderPricing dataclass ¶
Token pricing for an LLM provider (USD per 1M tokens).
WindowCost dataclass ¶
Cost record for a single window.
BudgetWarningLevel ¶
Bases: str, Enum
Budget warning levels.
BudgetWarning dataclass ¶
A budget warning emitted when approaching limits.
CostModel ¶
Real-time cost tracking with budget enforcement (§6.8).
Tracks per-window, per-session, and cumulative costs. Warns at 80%, hard-stops at 100% of user-set budgets.
pricing property ¶
Return the pricing.
total_windows property ¶
Return the total windows.
total_input_tokens property ¶
Return the total input tokens.
total_output_tokens property ¶
Return the total output tokens.
total_cost_usd property ¶
Return the total cost usd.
warnings property ¶
Return the warnings.
record_window(window_id, input_tokens, output_tokens, is_overhead=False, feature_name='') ¶
Record a completed window's cost. Returns the WindowCost record.
check_budget(input_tokens=0) ¶
Check all budget caps. Raises BudgetExhaustedError if exceeded.
Returns any warnings generated (80%+ of cap).
estimate(planned_dispatches=1, avg_input_tokens=0, avg_output_tokens=0) ¶
Pre-flight cost estimation.
reset() ¶
Reset all counters (on session start).
FeaturePriority dataclass ¶
Priority entry for overhead feature shedding.
OverheadDecision ¶
Bases: str, Enum
Result of overhead budget check.
OverheadBudget dataclass ¶
Caps total protocol overhead across all features (§6.9).
Default: 15% of productive windows. Feature shedding: lowest priority shed first. Never shed: extraction (always runs) and Tier 1 validation (zero LLM cost).
resources.governor¶
crp.resources.governor ¶
Resource Governor - protocol-layer "slow and steady over fast and failing" (CRP v5).
The Governor is CRP's orchestration-layer answer to local-LLM resource pressure. It does NOT do quantization / KV compression / model offloading - those belong to the inference engine (llama.cpp / vLLM / llama-swap). It governs the protocol levers that keep utilisation under a target on a constrained device:
- capability profile (frame size: small-local 1–2 tools … frontier 5–7),
- operation cap (loop guard),
- tool concurrency (serialised on constrained devices - the steadiness lever).
Prime directive: on a constrained device, run slow and steady - never exceed the target utilisation, accepting slower wall-clock time rather than risking OOM/overheat. When the device is unknown, the Governor defaults to the safe (constrained) plan.
DeviceTier ¶
Bases: str, Enum
Coarse device class used to pick a resource plan.
DeviceProfile dataclass ¶
The detected (or supplied) device characteristics.
ResourcePlan dataclass ¶
The Governor's plan for a run - the protocol levers, not engine settings.
ResourceGovernor ¶
Picks a resource plan that holds utilisation under the target.
detect_device(target_utilisation=0.4) staticmethod ¶
Detect the device tier from CPU/RAM (psutil optional; safe default constrained).
plan(requested_profile=None) ¶
Return a plan for the current device, capping any requested profile.
The Governor never upsizes beyond the device tier's ceiling. If the caller requests a smaller (more frugal) profile, that is honoured - frugality is always allowed; exceeding the tier ceiling is not.
resources.overhead_manager¶
crp.resources.overhead_manager ¶
Overhead budget manager - shedding cascade + lazy embedding (§6.12, §3.2).
OverheadBudgetManager sits on top of the existing OverheadBudget in cost_model.py and adds two features from the spec:
-
Feature shedding cascade (§6.12) When overhead exceeds the cap (default 15%), features are shed in a fixed cost order:
community_detection → cross_encoder → gliner → uie → discourse
Each feature can be individually disabled and later re-enabled when overhead drops.
- Lazy embedding batch (§3.2) Instead of computing embeddings on every
fact.createdevent, the manager defers them and batch-processes once N facts have accumulated or a flush is requested.
Design goals
- Pure logic - no external deps, no I/O.
- Works with or without an EventEmitter.
- Thread-safe (uses a threading.Lock around mutable state).
SheddingState dataclass ¶
Tracks which features are currently enabled or shed.
OverheadBudgetManager ¶
Orchestrates feature shedding and lazy embedding batching.
Usage::
mgr = OverheadBudgetManager(max_overhead_pct=15, batch_size=32)
# After each window, update overhead ratio:
mgr.update_overhead(current_pct=12.5)
assert mgr.is_feature_enabled("gliner")
# When overhead spikes:
mgr.update_overhead(current_pct=18.0)
assert not mgr.is_feature_enabled("community_detection") # shed first
# Lazy embedding:
mgr.defer_embedding("f1", "some fact text")
mgr.defer_embedding("f2", "another fact text")
batch = mgr.flush_embeddings() # returns pending, clears queue
shed_log property ¶
Return the shed log.
enabled_features property ¶
Return the enabled features.
pending_embedding_count property ¶
Return the current pending embedding count.
current_overhead_pct property ¶
Return the current overhead pct.
max_overhead_pct property ¶
Return the max overhead pct.
update_overhead(current_pct) ¶
Update the current overhead percentage and shed/restore features.
Returns a list of features that changed state (shed or restored).
is_feature_enabled(name) ¶
Check if a feature is currently enabled.
defer_embedding(fact_id, text) ¶
Queue a fact for deferred embedding.
Returns True if the batch is now full and should be flushed.
flush_embeddings() ¶
Return all pending embeddings as (fact_id, text) and clear queue.
summary() ¶
Return a JSON-serializable summary of the manager state.
resources.resource_manager¶
crp.resources.resource_manager ¶
Centralized resource manager - memory tracking, model registry, pressure (§audit R1).
Tracks memory consumption, ML model lifecycle, session budgets, and provides pressure signals for adaptive dispatch behavior.
Pressure levels
"none" : usage < 50% of budget → normal operation "low" : 50-70% → compaction eligible "medium" : 70-85% → aggressive compaction, idle model unload "high" : 85-95% → forced compaction, model unload, GC "critical" : >95% → emergency cleanup, session pruning
ResourceSnapshot dataclass ¶
Point-in-time resource utilization snapshot.
utilization_ratio property ¶
Return the utilization ratio.
ModelEntry dataclass ¶
Registered ML model with lifecycle tracking.
ResourceManager ¶
Centralized resource tracking and pressure management.
Usage::
rm = ResourceManager(budget_mb=512)
rm.register_model("sentence-transformers", 150)
rm.mark_model_loaded("sentence-transformers")
snap = rm.snapshot()
print(snap.pressure_level)
register_model(name, estimated_mb) ¶
Register an ML model (call at init, before loading).
mark_model_loaded(name) ¶
Mark a model as loaded into memory.
mark_model_used(name) ¶
Record model usage (resets idle timer).
mark_model_unloaded(name) ¶
Mark a model as unloaded from memory.
update_fact_count(count) ¶
Update the tracked fact-store fact count.
update_session_count(count) ¶
Update the tracked active session count.
snapshot() ¶
Take a point-in-time resource snapshot.
should_cleanup() ¶
Return True if pressure warrants cleanup actions.
run_gc() ¶
Run Python garbage collection and return the number of objects collected.
get_idle_models(idle_seconds=300.0) ¶
Return names of loaded models idle for longer than idle_seconds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
idle_seconds | float | Idle threshold in seconds. | 300.0 |
Returns:
| Type | Description |
|---|---|
list[str] | List of model names eligible for unload. |
mark_unloaded(model_name) ¶
Mark a model as unloaded (freed from memory).
trigger_gc() ¶
Run GC if pressure warrants it (alias for should_cleanup + run_gc).
summary() ¶
Return a human-readable resource utilization summary dict.