Document Qwen3.8 support and update status markers (#609)
## Summary - Add Qwen3.8 to the existing Qwen3.5/3.6 family row and update the example checkpoint to `mlx-community/Qwen3.8-27B-8bit`. - Mark automatic prefix caching as experimental for the hybrid GDN rows (`Qwen3.5 / 3.6 / 3.8` and `Qwen3-Next`) following #584. Prefix caching remains opt-in through `--enable-prefix-caching`. - Mark Gemma 4 as supported. - Add Qwen3.8 to the README news section. ## Verification Tested `mlx-community/Qwen3.8-27B-8bit` end to end on an M5 Pro with 68.7 GB unified memory, vLLM 0.27.0, and default plugin settings: | Metric | Result | | --- | --- | | Engine initialization | 52.5 s | | GPU KV cache | 187,830 tokens (22.9x concurrency at 8k context) | | Single-stream decode | 9.01 tok/s | | Batch size 3 | 18.3 tok/s aggregate | Reasoning, coding, and word-problem prompts completed successfully. No code changes were required because vLLM 0.27.0 already registers `Qwen3_5ForConditionalGeneration`. The 55.6 GB upstream BF16 checkpoint does not fit within the 55.66 GB Metal working set. The 28.6 GB MLX 8-bit conversion fits, but contains neither vision nor MTP tensors and therefore serves text only. The prefix-cache markers reflect support added in #584; this PR does not add model-specific prefix-cache benchmarks. Signed-off-by: Ranran <ranranhaoranzhang@gmail.com>
R
Ranran committed
494fb3e96b53c289ad9e05f4a6ce0652aeb889dd
Parent: dce0517
Committed by GitHub <noreply@github.com>
on 8/15/2026, 12:51:03 AM