fix(vlm): truncate only the token axis in gemma4 collate (#3531)
`gemma4_prefix_collate_fn`'s `max_length` truncation sliced every 2-D batch tensor on axis 1 except `pixel_values`. That denylist missed `image_position_ids`, which is indexed by patch, not by token: the positions were clipped to `max_length` while the patch tensor kept its full length, and the vision tower then failed on `hidden_states + position_embeddings` with a shape mismatch. Truncate by an allowlist of token-aligned keys instead, so a newly added media key is safe by default. Also refuse to truncate into multimodal placeholder tokens: dropping them would leave image features with nowhere to scatter, and a clear error beats a corrupted batch. No shipped recipe passes `max_length` to this collate function, so this is a latent bug rather than a live regression. Signed-off-by: Abhishree <abhishreetm@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
A
Abhishree Thittenamane committed
da5eba2374743e11238b145e40df0f4ed5444ba9
Parent: c4c0e4b
Committed by GitHub <noreply@github.com>
on 8/20/2026, 5:38:03 AM