SIGN IN SIGN UP

Support remote GGUF references (#635)

This PR is:

- To add remote GGUF serving for `repo_id:quant` references.
- To extend the existing local-only GGUF path without changing local
GGUF behavior.
- To support local or Hugging Face tokenizer/config sources for GGUF
weights.
- To fail clearly for unsupported qtypes, sharded GGUF, ambiguous
matches, and multimodal GGUF.

Before this PR, vllm-metal only supported local `.gguf` weight files.
After this PR, users can serve either a local GGUF file or a remote GGUF
reference:

```bash
vllm serve Qwen/Qwen3-0.6B-GGUF:Q8_0 \
  --tokenizer Qwen/Qwen3-0.6B
```

```bash
vllm serve /local/model.gguf \
  --tokenizer Qwen/Qwen3-0.6B
```

Remote GGUF support is intentionally limited to single-file `Q8_0`,
`Q4_0`, and `Q4_1` checkpoints for now.

---------

Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
L
Lik Xun Yuan (Lx) committed
3c9bbdc8839d6bbe99ad4e5b9ad652b15d25da8e
Parent: acce614
Committed by GitHub <noreply@github.com> on 8/22/2026, 1:07:07 PM