Refresh the single-stream numbers, measured with the harness that produced them
Issue #3 compared against 83/87 for CTX=long and got ~49. Re-measured all three
single-stream configurations on current main with bench/run_benchmarks.sh, which
is what produced the originals -- vllm bench serve, the 8 real prompts, 1024
output tokens, C1, decode = C / mean TPOT:
README now (default / greedy) tok/step
MTP CTX=fast 111 / 120 120.9 / 119.9 2.96 / 2.90
MTP CTX=long 85 / 89 96.2 / 102.2 2.61 / 2.66
DFlash2 CTX=fast 122 / 131 127.2 / 130.4 3.29 / 3.34
So the table understated by 0-15% rather than being wrong, and greedy is no longer
faster than default sampling at CTX=fast -- 119.9 against 120.9, where the old row
claimed 120 against 111. Updated, with a note naming the harness and the output
length, because that is the part that makes the numbers mean anything.
I first measured this with a throwaway client at 256 output tokens and got
109.6/114.1 for CTX=long, then compared it against a README number produced at
1024 by a different harness and called the docs 30% stale. They were 13% stale.
Same class of error as the rest of this week: the number was fine, the comparison
was not. Correcting the #3 comment too -- the conclusion there does survive, since
the reporter's box is ~1.8x off our same-config figure either way, but the
specific numbers I quoted him were not comparable to his. M
mhenrichsen committed
2ae239fc0250cd29d37f35c6a31e9eae749ef1c8
Parent: b356e31