feat(mtp): skip the replay forward on rejection via rollback_state (gated, +6%)
Every rejected MTP cycle paid a full extra backbone forward. The verify was rolled back to BEFORE it ran and the confirmed tokens re-run to rebuild state: verify 42.3ms + draft 2.5ms + replay ~40ms = 84.8ms for a reject against 44.8ms for an accept, with replay_main=28..38 per 64 cycles in the logs. The model already had the machinery and the engine never used it. Passing n_confirmed to the verify makes GatedDeltaNet process the confirmed prefix separately and stash the resulting (conv, ssm) pair as cache.rollback_state (patches/mlx_vlm_mtp/qwen35_vl.py:501). Restoring that lands exactly where the replay would have, and the corrected token's hidden state is already in the verify output. At depth 1 n_confirmed=1 is exact: accept keeps the cache untouched, reject restores the rollback point. Proven live on Qwen3.8-27B-JANG_4D-CRACK: replay_main drops 28 -> 0 and the generation is correct (747 tokens of coherent Python, no interruption). But the win is only about 6 percent - 22.1 -> 23.4 t/s - NOT the ~50 percent the naive arithmetic predicted, and the reason is worth recording: n_confirmed splits every GDN layer into TWO _process_chunk calls instead of one, so across 48 GDN layers the verify itself gets materially more expensive and eats most of the saved replay. Anyone re-deriving "remove the replay, get 34 t/s" should stop here. Gated OFF (VMLX_MTP_SKIP_REPLAY=1 enables) because 6 percent does not justify the extra rollback path by default. Bug found and fixed while building it: ArraysCache.state's setter does self.cache = v, so assigning a TUPLE made the model's next indexed write die with "'tuple' object does not support item assignment" and every generation aborted after 3 tokens at a fake-looking 56 t/s. Writes now go through __setitem__ exactly as the model does. That is also a reminder that a high t/s number on a generation that did not COMPLETE is not a measurement.
J
Jinho Jang committed
d5b667383c598ae3576a8462b62d324131c08d37
Parent: b05377a