Drafter-free n-gram chains for DFlash2 (VLLM_DFLASH2_CHAIN=1, off by default) (#38)
Ported from Dmtrii-tesla/dflash2-ngram-vllm with the author's explicit cherry-pick invitation in #38. While a request reproduces its own context, whole verify blocks are proposed from history alone and the drafter forward plus its graph replay are skipped until the first rejected token; the point-mass draft distribution keeps rejection sampling exact. Their three load-bearing design calls preserved (host state machine in propose, entry evidence as match_len not the clamped valid, one-normal-step re-entry). Added here: a greedy-only entry gate (default on) -- a point-mass q accepts with probability p(token), so at temperature the chain displaces strictly better drafter proposals, measured -8% C1 at T=default; with the gate that cell returns to the sampling band. Measured on the shipped artifact: copy 256.9 -> 276.1 tok/s (+7%), six-task labd flat, C1 greedy within spread, off-state pool byte-identical at 68,605. README paragraph + patch header carry the numbers. Co-authored-by: Dmtrii-tesla <Dmtrii-tesla@users.noreply.github.com>
M
mhenrichsen committed
c954724104a7856a07abb7031cc4af780ae7f5bf
Parent: 1e56173