Support Hugging Face config sources for local GGUF weights (#624)
This PR is: - To support serving local `.gguf` weights with a Hugging Face config/tokenizer source. - To resolve non-local GGUF config sources before building the MLX model skeleton. - To keep existing local config-directory GGUF loading unchanged. This enables the normal GGUF serve shape: ```bash vllm serve /path/to/Qwen3-0.6B-Q8_0.gguf \ --tokenizer Qwen/Qwen3-0.6B ``` Before this change, the GGUF loader treated `Qwen/Qwen3-0.6B` as a local directory and failed while looking for: ```text Qwen/Qwen3-0.6B/config.json ``` After this change, the loader downloads only the config files needed to build the MLX model skeleton. Tokenizer loading continues through the existing MLX-LM tokenizer path. Validation: - `tests/test_gguf_loader.py`: 42 passed. - Ruff, compileall, and `git diff --check` passed. - Real smoke: local `Qwen3-0.6B-Q8_0.gguf` weights with `Qwen/Qwen3-0.6B` config/tokenizer loaded successfully. --------- Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
L
Lik Xun Yuan (Lx) committed
489da60b5955a1cb3cc438d6cd0809c84ba42d65
Parent: 4edfe6a
Committed by GitHub <noreply@github.com>
on 8/19/2026, 6:56:00 AM