SIGN IN SIGN UP

ci(convergence): fix gemma4 gate tolerance (#3696)

The gate's `stderr` was lm-eval's within-eval bootstrap figure (0.0215), used
as if it were the between-run sigma of the whole train->eval pipeline. The
measured between-run sigma is ~2x larger, so the 3*0.0215 floor (0.4642) sat
~1.2 sigma below the mean and tripped roughly 1 healthy run in 9.

That is what the Aug-24 red was: it scored 0.4492, and re-running the same
commit (r0.6.0@94387e431) scored 0.5046, so nothing had regressed. Set stderr
to the measured 0.0455 with k=2, giving a floor of 0.4378 that clears all nine
known runs while a real training failure (~0.25, base level) still fails.

Signed-off-by: Abhishree <abhishreetm@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
A
Abhishree Thittenamane committed
c1730fe605dd08080e3af11469a5213f59a9b5bb
Parent: 6a61cc4
Committed by GitHub <noreply@github.com> on 8/26/2026, 10:24:27 PM