The refutation ledger
Claims we made, tested, and published the measured outcome of. Four of them killed our own bets. Publishing what doesn't work is the wall a competitor cannot copy without admitting limits.
Published — the corpus, kept
Every record below is a measured, signed event. The chain is sha256-linked today; the label on the verify button changes when Ed25519 / ML-DSA ship, in the same commit.
| Record | Claim tested | Measured result | n | Tag | Chain |
|---|---|---|---|---|---|
| DR-0001 | Per-dimension expert routing improves score | +0.90 [-1.99, +3.79] — not separable from noise | 195 | [REFUTED] | ledger/DR-0001 |
| DR-0002 | Statute retrieval (ungated) improves compliance | −9.16 [-17.64, -0.69] — HARM, retrieval adds noise the model follows | 31 | [REFUTED] | ledger/DR-0002 |
| DR-0003 | Relevance-gated statute retrieval improves compliance | −5.26 [-12.66, +2.13] — still negative; gate insufficient | 31 | [REFUTED] | ledger/DR-0003 |
| DR-0004 | Adding all 13 annexes recovers statute retrieval | −5.70 — corpus not the cause, the 0.5B retriever is | 14lower bound | [REFUTED] | ledger/DR-0004 |
| DR-0005 | 3-leg Byzantine quorum increases reliability | n_eff 1.21 of 3 · φ +0.743 — quorum retracted; legs enumerate, do not vote | 36 | [REFUTED] | ledger/DR-0005 |
| DR-0006 | CAD α-sweep improves composition | NULL — no effect across α range | 60 | [REFUTED] | ledger/DR-0006 |
| DR-0007 | Diet diversity decorrelates subjects (Qwen vs OLMo-2) | ρ=0.756 · oracle = best_single · gain 0.0 — diet diversity dead | 240 | [REFUTED] | ledger/DR-0007 |
| DR-0015 | COSE ML-DSA serialisation works | pycose 1.1.0 predates RFC 9964 — COSE layer NOT; crypto layer YES (OpenSSL 3.6.3) | 1lower bound | [REFUTED] | ledger/DR-0015 |
| DR-0032 | Handoff delivered (9 backend modules + 3 JSON files) | HANDOFF GAP — NONE on this disk. New LAW DR-0033: handoff ≠ delivery. | 1lower bound | [REFUTED] | ledger/DR-0032 |
Superseded — kept visible with the correction attached
We do not delete superseded records. The wrong number still ships, with the correction next to it. Hiding a correction is its own form of drift.
| Record | Original claim | Correction | Chain |
|---|---|---|---|
| DR-0012 | ProvBench CI: 3.43% (cell-level) | Asset is the unit, n=12. One-sided 22.1%. Transforms cluster. | ledger/DR-0012 |
Why this is top-level nav, not a footer link
Audit criterion C7: the refutation ledger is not buried. If a competitor can scroll past it, the discipline isn't visible. The ledger is a first-class surface — the same way the C2PA conformance register names what doesn't conform.
Three corrections were caught this pass and pinned: the ProvBench CI (asset vs cell unit), the 0/108 measured-vs-modelled drift, and the cron's overclaim. Every one was caught by the estate's own lanes before it shipped — the product working on itself.