SIGN IN SIGN UP

Refresh the single-stream numbers, measured with the harness that produced them

Issue #3 compared against 83/87 for CTX=long and got ~49. Re-measured all three
single-stream configurations on current main with bench/run_benchmarks.sh, which
is what produced the originals -- vllm bench serve, the 8 real prompts, 1024
output tokens, C1, decode = C / mean TPOT:

                        README      now (default / greedy)   tok/step
  MTP     CTX=fast      111 / 120   120.9 / 119.9            2.96 / 2.90
  MTP     CTX=long       85 /  89    96.2 / 102.2            2.61 / 2.66
  DFlash2 CTX=fast      122 / 131   127.2 / 130.4            3.29 / 3.34

So the table understated by 0-15% rather than being wrong, and greedy is no longer
faster than default sampling at CTX=fast -- 119.9 against 120.9, where the old row
claimed 120 against 111. Updated, with a note naming the harness and the output
length, because that is the part that makes the numbers mean anything.

I first measured this with a throwaway client at 256 output tokens and got
109.6/114.1 for CTX=long, then compared it against a README number produced at
1024 by a different harness and called the docs 30% stale. They were 13% stale.
Same class of error as the rest of this week: the number was fine, the comparison
was not. Correcting the #3 comment too -- the conclusion there does survive, since
the reporter's box is ~1.8x off our same-config figure either way, but the
specific numbers I quoted him were not comparable to his.
M
mhenrichsen committed
2ae239fc0250cd29d37f35c6a31e9eae749ef1c8
Parent: b356e31