# Headroom C2 rerun with large hosted models: plan (written BEFORE any hosted request)
Written: 2026-10-20 12:01:55 UTC+8. Reason: founder asked to retest C2 "Same answers" with large models before publishing the F.

## What stays the same (copied from 00-test-plan.md and 09-c2b-local-llm.py, unchanged)
- Headroom 0.41.0 (same venv as the first run), plain `compress(messages, model="gpt-4o")` with the same 4 messages; HEADROOM_TELEMETRY=off, HEADROOM_BEACON=off, DO_NOT_TRACK=1.
- Same 2 slices: synthetic_logs[1250:1430] (180 rows) and wiki_search[:100] (100 rows); same 10 code-checked questions; same prompt ("Tool output:\n...\n\nQuestion: ...\nAnswer with only the value, nothing else.").
- Same scoring: exact match after norm() (strip whitespace/quotes/dots, lowercase), or truth contained with <= 12 extra chars.
- github_issues and pypi_info still skipped: Headroom returns them byte-identical, so the model input is identical.

## What changes
- Answering model only: OpenRouter free models ($0 account; no credits bought), temperature 0, seed 1, one user message = the same prompt string. Target: 2 large models from different families: nvidia/nemotron-3-ultra-550b-a55b:free (NVIDIA, 550B MoE / 55B active) and thinkingmachines/inkling:free (Thinking Machines, 975B MoE / 41B active). No Llama-70B free model was listed on /api/v1/models today.
- Budget: 50 free requests/day. Slices = 2 models x 20 = 40 requests. Leftover requests (if any) go to the full payloads, reported separately and NOT used for the verdict unless all questions are run the same way.

## Pass mark (the C2 rule from 00-test-plan.md, unchanged)
V2 if compressed accuracy >= original accuracy - 1 question AND >= 95% of answers retained or retrievable;
V1 if retention 70-95% OR accuracy drop 2-4 questions; F otherwise (drop >= 5 or retention < 70%).
Retention (C2a) is already measured: 20/20 = 100% retained or retrievable via CCR, so the verdict is decided by the accuracy drop.
Applied per model on the 10 slice questions. Verdict for C2 = the worst model among the large hosted models (same "worst" logic the plan uses for the overall).
The earlier small local Qwen results (drops of 5) stay in the record as a separate, labelled finding.

## Overall (unchanged rule)
Overall = worst of C1b, C2, C3, C4. C1b is already F (0/4 payloads reached 60%), so the overall stays F whatever C2 shows.

## Retrieval markers
For every compressed-input miss we check whether the truth value / the asked row is hidden behind a <<ccr:...>> marker (plain compress() has no headroom_retrieve tool for the model).