Testing & Benchmarks¶
CRP has a comprehensive test suite with 1,537+ tests across 34 files, covering every module from smoke tests to live LLM verification.
Overview¶
| Category | Files | Tests | What It Covers |
|---|---|---|---|
| Smoke | 1 | 6 | Package imports, version, basic types |
| Unit (core) | 9 | 636 | All 9 SDK phases |
| Unit (specialized) | 14 | 634 | Deep module-level coverage |
| Integration | 1 | 57 | Cross-module E2E |
| Production hardening | 1 | 40 | Circuit breakers, retries, cleanup |
| Benchmarks | 1 | 12 | Performance regression |
| Live LLM | 4 | 52 | Real model verification |
| Total | 34 | 1,537+ |
Quick Start¶
# Install dev dependencies
pip install -e ".[dev]"
# Run smoke tests (fast, ~2 seconds)
python -m pytest tests/test_smoke.py -v
# Run a specific phase
python -m pytest tests/test_phase1.py -v --tb=short
# Run with coverage
python -m pytest tests/test_smoke.py --cov=crp --cov-report=term
Do NOT run all tests at once
The full suite is resource-intensive. Run one file at a time to avoid maxing out system resources.
Sections¶
- Running Tests — How to run each test category, what each file covers, fixtures, and tips
- Benchmarks — Performance results: continuation engine, extraction pipeline, protocol overhead
- Reproduce Benchmarks — Step-by-step guide to reproduce our benchmark results on your own hardware