fix: four reported issues — garbled qwen3_5, embeddings cap, VLM context cap, guard diagnostics
vmlx#259 — garbled output on already-converted Qwen3.8 bundles Qwen3.5-family checkpoints store zero-centered RMSNorm weights, so the loader adds 1.0. Bundles a conversion pipeline saved AFTER running sanitize are already shifted; the base family loaders shifted them again, roughly doubling the norm scale — fluent-looking multilingual noise with empty visible content. Measured on real checkpoints without downloading them: mlx-community/Qwen3.8-27B-4bit layer0 input_layernorm mean +0.96 vs JANG bundles -0.04, and both ship MLX-side key names, so naming cannot tell them apart. The native-MTP adapter already had exactly this detector; it is now canonical in utils/zero_centered_norms.py and used by all six sanitize sites. Post-fix both bundle families converge on the same healthy ~0.96 effective norm scale. Note for future edits: the vendored qwen3_5 files are installed under the mlx_vlm package name, so imports back into vmlx_engine must be ABSOLUTE. A relative import there passes in isolation and fails once registration has run — caught by the full suite, not by the targeted tests. vmlx#255 — embeddings endpoint truncated every input to 512 tokens max_length was hardcoded, so a long-context embedding model silently indexed the first 512 tokens of each chunk and returned vectors that ignored the rest. The limit now comes from the tokenizer and model config (transformers' no-limit sentinel ignored, smallest stated limit wins, 512 only when nothing is stated), and truncation logs a warning naming the limit and the longest input instead of failing silently. vmlx#254 — VLM prompt cap far tighter than the hardware requires The auto prompt cap divides a KV budget by estimated bytes-per-token, and that estimator charged every layer as full attention. Qwen3.5/3.6 stacks are interval-hybrid (full_attention_interval 4): three of four layers hold a fixed-size recurrent state that does not grow with context, so the charge was ~4x too high. Same shape of fix as the existing dots3_note and deepseek_v4 special cases, keyed on config the family declares. Measured: 21,626 -> 86,507 tokens at the same free memory (the pre-fix number lands within 8% of the 20,119 the reporter hit). Dense models and both existing special cases are unchanged, with tests pinning that. vmlx#256 — image-prefill guard rejected without actionable diagnostics The guard stays: a Metal OOM in the media forward raises inside a completion handler and kills the engine uncatchably, so it must predict. But every number needed to act was already at the raise site and none of it reached the user. The message now states the cost model (heads x tokens^2 x 2B), the resident/limit split for working-set rejections, the media-expanded prompt's image/text composition, and the inverse budget — how many tokens actually fit, and therefore how much text can remain alongside the attached images.
J
Jinho Jang committed
c69267a2366a388b310c7c762893e3ebd9f01c48
Parent: e40ee6e