SIGN IN SIGN UP

Support Hugging Face config sources for local GGUF weights (#624)

This PR is:

- To support serving local `.gguf` weights with a Hugging Face
config/tokenizer source.
- To resolve non-local GGUF config sources before building the MLX model
skeleton.
- To keep existing local config-directory GGUF loading unchanged.

This enables the normal GGUF serve shape:

```bash
vllm serve /path/to/Qwen3-0.6B-Q8_0.gguf \
  --tokenizer Qwen/Qwen3-0.6B
```

Before this change, the GGUF loader treated `Qwen/Qwen3-0.6B` as a local
directory and failed while looking for:

```text
Qwen/Qwen3-0.6B/config.json
```

After this change, the loader downloads only the config files needed to
build the MLX model skeleton. Tokenizer loading continues through the
existing MLX-LM tokenizer path.

Validation:

- `tests/test_gguf_loader.py`: 42 passed.
- Ruff, compileall, and `git diff --check` passed.
- Real smoke: local `Qwen3-0.6B-Q8_0.gguf` weights with
`Qwen/Qwen3-0.6B` config/tokenizer loaded successfully.

---------

Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
L
Lik Xun Yuan (Lx) committed
489da60b5955a1cb3cc438d6cd0809c84ba42d65
Parent: 4edfe6a
Committed by GitHub <noreply@github.com> on 8/19/2026, 6:56:00 AM