CRPv6 Public Demo & Test Runbook¶

Last updated: 2026-08-12
Applies to: crprotocol >= 6.0.0
Repositories:
- https://github.com/AutoCyber-AI/context-relay-protocol
- https://github.com/AutoCyber-AI/crp-gateway
- https://github.com/AutoCyber-AI/crp-comply
- https://github.com/AutoCyber-AI/crp-scan

1. What this runbook covers¶

This guide maps every public-facing demo, script, and test that proves CRPv6 works. It tells you:

What each demo proves - the exact protocol behaviour on display.
How to run it - copy-paste commands for PowerShell, CMD, and Git Bash.
What dependencies you need - and how to verify they are up to date.
What is real vs. what is hardcoded - so a video or live stream is trustworthy.

If you are preparing a launch video, LinkedIn demo, or public benchmark stream, start here.

2. What CRPv6 emits during operation¶

CRPv6 is a governance middleware layer around an LLM call. When you run a CRP agent, the protocol emits:

Field	Meaning	Where it appears
`risk`	Per-operation risk tier (`LOW`, `MEDIUM`, `HIGH`, `CRITICAL`)	`result.crp.risk`
`grounded`	Whether the answer is backed by a tool observation or source	`result.crp.grounded`
`chain_valid`	Whether the HMAC audit chain for this window is intact	`result.crp.chain_valid`
`audit_url`	Pointer to the tamper-evident audit record (empty in local-only mode)	`result.crp.audit_url`
`operations`	STL operations used (e.g. `RETRIEVE`, `TRANSFORM`, `ANALYSE`)	`result.operations`
`sources`	Tool observations, CKF facts, or retrieved documents with provenance	`result.sources`

These values are computed at runtime by the protocol, not written into the demo script.

3. Quick sanity check: is the latest protocol installed?¶

# PowerShell / CMD
python -m pip show crprotocol | findstr Version

# Git Bash / Linux / macOS
python -m pip show crprotocol | grep Version

Expected: Version: 6.0.0 or newer.
Latest release: https://pypi.org/project/crprotocol/

To upgrade:

python -m pip install -U crprotocol

To install from source:

# Clone the public repo
git clone https://github.com/AutoCyber-AI/context-relay-protocol.git
cd context-relay-protocol
python -m pip install -e ".[full]"

4. Live LLM demos (recommended for video)¶

These demos connect to a real local LLM. The outputs you see are generated by the model and CRP at runtime.

4.1 Demo: raw LLM vs. CRPv6 Agent¶

Files: - examples/crp_demos/live_crp_slm_proof.py - full side-by-side proof including single tool, tool chain, RAG, and long-form report. - examples/crp_demos/live_llm_vs_crp.py - compact one-question demo.

What it proves:
The same small local model, the same tools, the same question - once with a raw prompt and once with CRPv6 positioning, tool orchestration, and governance.

Prerequisites:

LM Studio (or any OpenAI-compatible endpoint) running on http://localhost:1234/v1.
A model loaded, e.g. meta-llama-3.1-8b-instruct.
crprotocol installed.

Run (PowerShell / CMD):

set CRP_LMSTUDIO_URL=http://localhost:1234/v1
set CRP_LMSTUDIO_MODEL=meta-llama-3.1-8b-instruct
python examples/crp_demos/live_crp_slm_proof.py

Run (Git Bash / Linux / macOS):

CRP_LMSTUDIO_URL=http://localhost:1234/v1 \
CRP_LMSTUDIO_MODEL=meta-llama-3.1-8b-instruct \
python examples/crp_demos/live_crp_slm_proof.py

What is real:

The model's raw JSON tool request.
The CRP agent's final natural-language answer.
The risk, grounded, chain_valid, operations, and sources fields.

What is hardcoded:

The weather tool itself returns a fixed string (22°C and sunny). This is intentional so the demo works offline. The protocol's decision to call the tool is not hardcoded.

4.2 Demo: real-world public API proof¶

Files: - examples/crp_demos/live_real_world_api_proof.py - side-by-side raw LLM vs. CRPv6 using live public APIs. - examples/agents/real_world_research_agent.py - Agent SDK version of the same agent.

What it proves:
CRPv6 executes tools against real, free public APIs and turns the structured responses into natural-language answers with governance metadata. The raw LLM only emits a JSON tool request.

APIs used (no API key required): - Open-Meteo - live weather - CoinGecko - live cryptocurrency prices - Nominatim - geocoding (used by weather) - Wikipedia REST - article summaries

Run (PowerShell / CMD):

set CRP_LMSTUDIO_URL=http://localhost:1234/v1
set CRP_LMSTUDIO_MODEL=meta-llama-3.1-8b-instruct
python examples/crp_demos/live_real_world_api_proof.py

Run (Git Bash / Linux / macOS):

CRP_LMSTUDIO_URL=http://localhost:1234/v1 \
CRP_LMSTUDIO_MODEL=meta-llama-3.1-8b-instruct \
python examples/crp_demos/live_real_world_api_proof.py

What is real:

Live API calls to Open-Meteo, CoinGecko, and Wikipedia.
The model's raw JSON tool request (it does not execute the tool).
CRP's execution of the tool, natural-language synthesis, and governance metadata.

What is hardcoded:

Nothing in the data path. The only static values are the tool implementations and the public API URLs.

4.3 Demo: four-case agent harness¶

File: examples/crp_demos/live_agent_test_harness.py

What it proves:
CRPv6 can handle single-tool, tool-chain, customer-support, and RAG retrieval scenarios against a live local model.

Run:

set CRP_LMSTUDIO_URL=http://localhost:1234/v1
set CRP_LMSTUDIO_MODEL=meta-llama-3.1-8b-instruct
python examples/crp_demos/live_agent_test_harness.py

CRP_LMSTUDIO_URL=http://localhost:1234/v1 \
CRP_LMSTUDIO_MODEL=meta-llama-3.1-8b-instruct \
python examples/crp_demos/live_agent_test_harness.py

What is real:

All four [PASS] / [FAIL] verdicts are based on whether CRP emitted the expected governance metadata, which is computed at runtime.
The operations and sources arrays come from the protocol.

What is hardcoded:

The weather tool return value.
The pass/fail assertions check for non-empty result.answer and valid metadata, not semantic perfection.

5. Agent SDK examples¶

These examples run against a mock or live provider and show the CRP Agent SDK surface.

5.1 Real-world API research agent¶

File: examples/agents/real_world_research_agent.py

What it proves:
crp.Agent with real public APIs: weather, crypto prices, and Wikipedia summaries. Works against a live SLM or the built-in mock provider.

# Live SLM
export CRP_LMSTUDIO_URL=http://localhost:1234/v1
export CRP_LMSTUDIO_MODEL=meta-llama-3.1-8b-instruct
python examples/agents/real_world_research_agent.py

5.2 Weather agent¶

File: examples/agents/weather_agent.py

What it proves:
@client.tool decorator + client.ask(...) gives you a governed one-tool agent.

python examples/agents/weather_agent.py

5.3 RAG agent¶

File: examples/agents/rag_agent.py

What it proves:
Ingest a directory, ask a question, get a sourced answer with CSO carry-forward.

python examples/agents/rag_agent.py --docs_dir ./sample_docs --query "What is CRP?"

File: examples/agents/gdpr_dsr_agent.py

What it proves:
A sensitive workflow (data subject request) is gated by a checkpoint and audit trail.

python examples/agents/gdpr_dsr_agent.py --action delete --subject_id user-123

5.5 Report agent¶

File: examples/agents/report_agent.py

What it proves:
A multi-step agent that scans a codebase, summarises, and generates a report.

python examples/agents/report_agent.py --path ./src --output report.md

6. Automated test suite¶

CRPv6 ships with a non-live test suite that does not require API keys. It uses CustomProvider with fixed generate_fn mocks.

6.1 Smoke + integration¶

pytest tests/test_smoke.py tests/test_integration.py -v --tb=short

What it proves:
Core orchestrator, envelope builder, and session lifecycle work without a live LLM.

6.2 Security / compliance wiring¶

pytest tests/test_compliance_security.py tests/test_compliance_wiring.py -v --tb=short

What it proves:
Safety checks, header filtering (Axiom 4), RBAC, and provenance chains work.

6.3 Agentic dispatch¶

pytest tests/test_agentic.py -v --tb=short

What it proves:
Multi-step tool execution, continuation, and decision provenance.

6.4 Full suite¶

pytest tests/ -q --tb=short

Expected: 3232+ tests pass, 3 skipped (Windows GLiNER disabled).

7. ML model evaluation (Phase A)¶

These scripts prove the ML-first foundation.

7.1 Intent classifier¶

python scripts/eval_crp_intent_setfit.py \
  --model AutoCyberAI/crp-intent-setfit \
  --dataset banking77 \
  --split test

What it proves:
The SetFit intent model achieves ~0.934 held-out accuracy.

7.2 Safety classifier¶

python scripts/eval_crp_safety_classifier.py \
  --model AutoCyberAI/crp-safety-deberta-v1 \
  --dataset yiting/MD-Judge

What it proves:
The safety model passes 12/12 adversarial test classes.

7.3 PRM scorer¶

python scripts/eval_crp_prm.py \
  --model AutoCyberAI/crp-prm-deberta-v1 \
  --dataset prm800k_test

What it proves:
The process-reward model reaches AUC ~0.793 on held-out reasoning steps.

8. CRP Scan demo¶

Repo: https://github.com/AutoCyber-AI/crp-scan

Run locally against any Python repo:

pip install crprotocol[cli]
crp scan --paths ./src --format markdown --sarif ./crp-scan.sarif

What it proves:
Static governance scanning (CRP001–CRP006 rules) works without sending code to a third party.

9. Gateway + Comply (managed cloud)¶

These products are public repos but require your own Stripe + Clerk credentials to run in production.

Gateway: git clone https://github.com/AutoCyber-AI/crp-gateway.git
Comply: git clone https://github.com/AutoCyber-AI/crp-comply.git

Both use placeholder Stripe/Clerk values. Set real values in .env (never committed). See each repo's README.md and .env.example.

10. What is hardcoded in tests (and why that is OK)¶

Hardcoded item	Why it is hardcoded	What is NOT hardcoded
Mock LLM responses in `tests/`	Tests must run without API keys	CRP's handling of the response
Weather tool return value	Demo must work offline	The decision to call the tool
Sample documents in `examples/sample_docs`	Controlled RAG input	Retrieval, ranking, and provenance
Eval label sets	Standard benchmarks	Model predictions on those benchmarks

The protocol's value is in the governance wrapper, not the underlying tool output. Hardcoded tool outputs are acceptable; hardcoded governance metadata would defeat the purpose.

11. Keeping demos up to date¶

Before any public demo:

Check the installed version:

python -m pip show crprotocol | findstr Version

Check the latest release:

gh release list --repo AutoCyber-AI/context-relay-protocol --limit 3

Upgrade if needed:
```
python -m pip install -U crprotocol
```
Verify LM Studio is running and the model is loaded.
Run live_llm_vs_crp.py as a one-command smoke test.

12. Video checklist¶

For a launch video, show:

Terminal: python -m pip show crprotocol version is 6.0.0+.
LM Studio loaded with an 8B model.
live_crp_slm_proof.py running end-to-end (single tool, tool chain, RAG, long-form report).
live_llm_vs_crp.py running end-to-end.
The CRP governance block (risk, grounded, chain_valid, operations, sources) is visible.
live_agent_test_harness.py passing 4/4 cases.
One automated test command passing.

This combination proves CRPv6 is operational, observable, and not a hardcoded script.

CRPv6 Public Demo & Test Runbook¶

1. What this runbook covers¶

2. What CRPv6 emits during operation¶

3. Quick sanity check: is the latest protocol installed?¶

4. Live LLM demos (recommended for video)¶

4.1 Demo: raw LLM vs. CRPv6 Agent¶

4.2 Demo: real-world public API proof¶

4.3 Demo: four-case agent harness¶

5. Agent SDK examples¶

5.1 Real-world API research agent¶

5.2 Weather agent¶

5.3 RAG agent¶

5.4 GDPR DSR agent¶

5.5 Report agent¶

6. Automated test suite¶

6.1 Smoke + integration¶

6.2 Security / compliance wiring¶

6.3 Agentic dispatch¶

6.4 Full suite¶

7. ML model evaluation (Phase A)¶

7.1 Intent classifier¶

7.2 Safety classifier¶

7.3 PRM scorer¶

8. CRP Scan demo¶

9. Gateway + Comply (managed cloud)¶

10. What is hardcoded in tests (and why that is OK)¶

11. Keeping demos up to date¶

12. Video checklist¶