SIGN IN SIGN UP

fix: four reported issues — garbled qwen3_5, embeddings cap, VLM context cap, guard diagnostics

vmlx#259 — garbled output on already-converted Qwen3.8 bundles
  Qwen3.5-family checkpoints store zero-centered RMSNorm weights, so the
  loader adds 1.0. Bundles a conversion pipeline saved AFTER running
  sanitize are already shifted; the base family loaders shifted them again,
  roughly doubling the norm scale — fluent-looking multilingual noise with
  empty visible content. Measured on real checkpoints without downloading
  them: mlx-community/Qwen3.8-27B-4bit layer0 input_layernorm mean +0.96
  vs JANG bundles -0.04, and both ship MLX-side key names, so naming cannot
  tell them apart. The native-MTP adapter already had exactly this detector;
  it is now canonical in utils/zero_centered_norms.py and used by all six
  sanitize sites. Post-fix both bundle families converge on the same healthy
  ~0.96 effective norm scale.

  Note for future edits: the vendored qwen3_5 files are installed under the
  mlx_vlm package name, so imports back into vmlx_engine must be ABSOLUTE.
  A relative import there passes in isolation and fails once registration
  has run — caught by the full suite, not by the targeted tests.

vmlx#255 — embeddings endpoint truncated every input to 512 tokens
  max_length was hardcoded, so a long-context embedding model silently
  indexed the first 512 tokens of each chunk and returned vectors that
  ignored the rest. The limit now comes from the tokenizer and model config
  (transformers' no-limit sentinel ignored, smallest stated limit wins,
  512 only when nothing is stated), and truncation logs a warning naming the
  limit and the longest input instead of failing silently.

vmlx#254 — VLM prompt cap far tighter than the hardware requires
  The auto prompt cap divides a KV budget by estimated bytes-per-token, and
  that estimator charged every layer as full attention. Qwen3.5/3.6 stacks
  are interval-hybrid (full_attention_interval 4): three of four layers hold
  a fixed-size recurrent state that does not grow with context, so the
  charge was ~4x too high. Same shape of fix as the existing dots3_note and
  deepseek_v4 special cases, keyed on config the family declares. Measured:
  21,626 -> 86,507 tokens at the same free memory (the pre-fix number lands
  within 8% of the 20,119 the reporter hit). Dense models and both existing
  special cases are unchanged, with tests pinning that.

vmlx#256 — image-prefill guard rejected without actionable diagnostics
  The guard stays: a Metal OOM in the media forward raises inside a
  completion handler and kills the engine uncatchably, so it must predict.
  But every number needed to act was already at the raise site and none of
  it reached the user. The message now states the cost model
  (heads x tokens^2 x 2B), the resident/limit split for working-set
  rejections, the media-expanded prompt's image/text composition, and the
  inverse budget — how many tokens actually fit, and therefore how much text
  can remain alongside the attached images.
J
Jinho Jang committed
c69267a2366a388b310c7c762893e3ebd9f01c48
Parent: e40ee6e