SIGN IN SIGN UP

Bump vLLM to 0.28.0 (#645)

0.28.0 refactored the KV cache page sizing (vllm#51704) and our
TurboQuant spec stopped working: real_page_size_bytes is just an alias
now, so the scheduler went back to budgeting blocks with the dense int8
formula. I moved the packed page size into the state_content_bytes
field, which is how upstream's own TurboQuant backend does it, and made
the hybrid realign update mamba_block_size as well, otherwise the
coordinator asserts at startup.

The other two changes are new defaults. 0.28.0 turns prefix caching on
by default for hybrid models (vllm#50991) and async scheduling on for
draft-model spec decode (vllm#48341). Both are resolved before our
platform hook sees the config, so if we kept raising like before, a
plain vllm serve that worked on 0.27.1 would now die. The hook disables
them with a warning instead. An explicit --mamba-block-size still fails
fast.

We also get to delete a shim this time. The UniProc VLLM_HOST_IP
workaround can go, since 0.28.0 does rendezvous through a file:// store
now (vllm#50999). That is the only one: vllm#48724 is still not in,
CoreEngineActorManager still calls bare ray.init(), and vLLM still caps
fastapi below 0.137, so those all stay.

I also added huggingface_hub>=1.27.0 to pyproject because we import it
directly and vLLM added the floor this release.

For evidence, MetalPlatform resolves after a clean ./install.sh, 1874
non-slow tests pass, both serve smokes return their goldens (the Qwen3.5
one runs with the new APC-on default), a TurboQuant hybrid serve boots,
and the n-gram e2e passes 7/7. The slow suite has three failures on my
machine (the hybrid APC parity check, the draft-model e2e, and a
test-ordering memory issue). I reinstalled the 0.27.1 wheel and they
fail the same way there, so they were already broken before this bump.

`vllm bench serve`, Qwen3-0.6B, sonnet, 100 prompts, `--request-rate inf
--temperature 0 --ignore-eos --seed 0`, each arm on its own install. The
first 0.28.0 session came out slow because it was the first run after a
fresh install, so I dropped the first session of both arms and kept two
warm ones each. All sessions completed 100 requests with 54256 input and
15000 generated tokens.

| Metric | vLLM 0.27.1 | vLLM 0.28.0 | Delta |
|---|---:|---:|---:|
| Output tok/s | 1068.84 | 1070.41 | +0.1% |
| Total tok/s | 4934.90 | 4942.15 | +0.1% |
| Mean TTFT (ms) | 3752.78 | 3727.07 | -0.7% |
| P99 TTFT (ms) | 6848.13 | 6832.76 | -0.2% |
| Mean TPOT (ms) | 67.58 | 67.62 | +0.1% |
| P99 TPOT (ms) | 85.28 | 85.45 | +0.2% |
| Duration (s) | 14.03 | 14.01 | -0.1% |

---------

Signed-off-by: RickyChen / 陳昭儒 <ricky.chen@infinirc.com>
C
Chao-Ju Chen committed
db4d7c57c7d72f731f7010fd3b60fc25fc489229
Parent: 24c9c5b
Committed by GitHub <noreply@github.com> on 8/26/2026, 1:39:22 PM