fix(vlm): no_chunked_prefill protects vision spans, it does not forbid chunking
gemma4 sets no_chunked_prefill from use_bidirectional_attention == "vision", and that config DEFAULTS to "vision". Honouring it as an absolute kill switch meant gemma4 media prompts NEVER chunked, so an 80,611-token media conversation reached [metal::malloc] Attempting to allocate 207,940,266,272 bytes which is greater than the maximum allowed buffer size of 86,586,540,032 bytes and every turn after it failed. Measured live on a 14-turn variation matrix: turns 11-14 all returned nothing once the prompt crossed ~55k. What the flag protects is vision spans -- a bidirectional mask over an image would break if the span were split across forwards. (The MLX language model implements no such mask today, _make_masks is causal-only, so a split is currently output-identical; that is a fact about today, not a licence.) _media_chunk_boundaries already guarantees no boundary lands inside a media run: a run starting after the cut moves whole to the next chunk, and a run already open at the cut extends the chunk to cover it -- even when the run is longer than the nominal chunk size. So the intent is satisfiable without refusing to chunk. The invariant is now VERIFIED rather than assumed: before forwarding, every boundary is checked against every run, and if any boundary would split a span the request falls back to the one-shot path with a log line naming the run. Tests: the old test asserted the kill-switch behaviour and now asserts the opposite, with the metal::malloc failure in its docstring so the reason survives. Two new tests pin the invariant directly, including a media run LONGER than the chunk size.
J
Jinho Jang committed
0abdd34cd3caef76c629c557ca36cb52eef2b9ae
Parent: d1c7880