SIGN IN SIGN UP

fix(vlm): truncate only the token axis in gemma4 collate (#3531)

`gemma4_prefix_collate_fn`'s `max_length` truncation sliced every 2-D batch
tensor on axis 1 except `pixel_values`. That denylist missed
`image_position_ids`, which is indexed by patch, not by token: the positions
were clipped to `max_length` while the patch tensor kept its full length, and
the vision tower then failed on `hidden_states + position_embeddings` with a
shape mismatch.

Truncate by an allowlist of token-aligned keys instead, so a newly added media
key is safe by default. Also refuse to truncate into multimodal placeholder
tokens: dropping them would leave image features with nowhere to scatter, and a
clear error beats a corrupted batch.

No shipped recipe passes `max_length` to this collate function, so this is a
latent bug rather than a live regression.

Signed-off-by: Abhishree <abhishreetm@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
A
Abhishree Thittenamane committed
da5eba2374743e11238b145e40df0f4ed5444ba9
Parent: c4c0e4b
Committed by GitHub <noreply@github.com> on 8/20/2026, 5:38:03 AM