SIGN IN SIGN UP

dflash2 CTX=huge FULL: 0 broken of 128 residues, and ship the sweep

I gave dflash2 FULL capture on five sampled residues, which is the same evidence
that made the mtp claim wrong. Swept all 128 with a self-hit measured at each:
0 broken. So FULL stays for dflash2, now on evidence rather than on having sampled
lucky, and the comment says which.

The contrast is the useful part and both halves are now measured the same way:

  SPEC=dflash2 CTX=huge FULL   128/128 clean            -> FULL (also 96.5% vs 95.0%
                                                          GSM8K, and 2-3x on a VM)
  SPEC=mtp     CTX=huge FULL   1/128 broken, residue 4  -> PIECEWISE, for correctness

bench/residue_sweep.py is the harness, generalised from the throwaway. Stepping the
pad by one token covers every residue exactly once, which is the coverage gap
mjungnickel18 hit from the other direction (stepping ctx by 1 skipped 35 of 128).
Its docstring leads with the arithmetic that should have stopped me: one broken
length in 128 survives five samples 82% of the time.
M
mhenrichsen committed
b356e31526886b4bd614a79cd8600e7cc9383cf9
Parent: a72e902