# qwen3-4b-deslop test plan (pre-registered before any generation)

Written and committed 2026-10-23 08:56 (UTC+8; first draft said 09:00 by mistake, corrected to the commit time) by Grok Bot. Box: 8 CPU cores, 15 GB RAM, no GPU; Ollama 0.40.8 (installed in today's Ollama test).
Vendor pages saved 2026-10-23 08:57 in runs/2026-10-23/tools/qwen3-4b-deslop/vendor/ (blog + Hugging Face model card). checked_on 2026-10-23. Price: free open weights (Hugging Face), no account.

## Vendor claims (verbatim)
- D1 (core) "Em dashes went from 20.5 per 1000 words to zero." (blog TL;DR; model card table 20.5 -> 0.0)
- D2 (core) "not X, but Y" per 1000 words 7.8 -> 0.8 (model card table); "writes without em dashes, 'not X, but Y' or lists of three and needs no system prompt" (blog subtitle)
- D3 (core) "It still answers 30 of 30 factual questions correctly" (blog TL;DR; card: 97% -> 100%)
- D4 (core) "the slop score dropped from 121 to 12.5 per 1000 words" (blog TL;DR)
- D5 (not core) "With Ollama it's ollama run hf.co/erikkannike/qwen3-4b-deslop-v4-GGUF:Q4_K_M, a 2.5 GB download."

## Setup and task
- Models: qwen3-4b-deslop-v4-Q4_K_M.gguf (author's repo) vs the original Qwen3-4B-Instruct-2507-Q4_K_M.gguf (unsloth GGUF build of the same base; official Qwen GGUF repo not reachable without login). Same quantisation, imported into Ollama the same way. No system prompt.
- Writing prompts (8, written by us, 2 per genre, topics not from the author's set): blog post on learning to bake bread at 40; blog post on a first marathon; LinkedIn post on switching to a four-day work week at a 20-person agency; LinkedIn post on hiring a first data engineer; explainer of how heat pumps work; explainer of what an index fund is; opinion piece on whether cities should ban cars from downtown; opinion piece on whether schools should allow phones. Each "about 200 words". temperature 0.7, seeds 1 and 2 (2 samples per prompt per model = 16 answers per model), num_predict 260 (as the author).
- Factual quiz (ours, ground truth fixed now): 20 questions: 17*23=391; capital of Australia=Canberra; symbol for gold=Au; 2^10=1024; author of Pride and Prejudice=Jane Austen; largest planet=Jupiter; square root of 144=12; water boils at 100 C at sea level; speed of light ~300,000 km/s (accept 299,792 / 3x10^8 m/s); year the Berlin Wall fell=1989; chemical formula of table salt=NaCl; 15% of 80=12; longest river in Africa=Nile; author of '1984'=George Orwell; number of continents=7; 9*7=63; smallest prime number=2; capital of Canada=Ottawa; freezing point of water in Fahrenheit=32; who painted the Mona Lisa=Leonardo da Vinci. temperature 0.

## Measurement and pass marks
- Words counted by whitespace split. Em dash = U+2014 (also report spaced " -- " and U+2013 separately, not scored).
- D1: V2 if deslop has 0 em dashes in 16 answers; V1 if <=1.0 per 1000 words and >=90% below the original; F otherwise.
- D2: our regex for "not X, but Y" / "it's not X, it's Y" / "isn't X, it's Y" / "not just X, but/it's Y" within one sentence (script c_slop.py, written before generation). V2 if deslop <=2.0 per 1000 words AND >=75% lower than the original; V1 if 50-75% lower; F if <50% lower. If the original itself has <3 hits total, D2 is V0 (sample too small).
- D3: V2 if deslop gets >= original's count and >=19/20; V1 if 1-2 below the original; F if 3+ below.
- D4: needs the author's detector (lab/slopscore.py in a "lab repo" that is not linked). If not obtainable today: V0 (and, as a core claim, it caps the overall at V1). We do not substitute our own score for it.
- D5: run the exact command. V2 if it downloads and runs; if it fails because of our network (like today's Ollama pull), record and import the GGUF file instead (documented Ollama import), rate V0 for our environment.

## Verdict rule
Overall = worst core claim (D1-D4); any core V0 caps overall at V1. Score = mean of tested claims (V2=10, V1=5, F=0, V0 excluded). level L0, typical_level L0.