SIGN IN SIGN UP

Drafter-free n-gram chains for DFlash2 (VLLM_DFLASH2_CHAIN=1, off by default) (#38)

Ported from Dmtrii-tesla/dflash2-ngram-vllm with the author's explicit
cherry-pick invitation in #38. While a request reproduces its own context,
whole verify blocks are proposed from history alone and the drafter forward
plus its graph replay are skipped until the first rejected token; the
point-mass draft distribution keeps rejection sampling exact. Their three
load-bearing design calls preserved (host state machine in propose, entry
evidence as match_len not the clamped valid, one-normal-step re-entry).
Added here: a greedy-only entry gate (default on) -- a point-mass q accepts
with probability p(token), so at temperature the chain displaces strictly
better drafter proposals, measured -8% C1 at T=default; with the gate that
cell returns to the sampling band. Measured on the shipped artifact: copy
256.9 -> 276.1 tok/s (+7%), six-task labd flat, C1 greedy within spread,
off-state pool byte-identical at 68,605. README paragraph + patch header
carry the numbers.

Co-authored-by: Dmtrii-tesla <Dmtrii-tesla@users.noreply.github.com>
M
mhenrichsen committed
c954724104a7856a07abb7031cc4af780ae7f5bf
Parent: 1e56173