SIGN IN SIGN UP

Add LLaVA-OneVision (#2115)

SigLIP vision tower with a Qwen2 language model, covering single images,
multiple images, and video in one checkpoint.

Images are tiled with AnyRes and packed as base features plus the unpadded
grid with newline separators, scaled down when the grid exceeds
vision_aspect_ratio. Video frames are pooled 2x bilinearly to 196 tokens per
frame with one closing newline per video.

The image processor that AutoImageProcessor resolves for this model type is
torchvision-backed, so the PIL backend is used directly and video frames are
handled with numpy, keeping mlx-vlm free of a torch dependency.
A
Alazer Manakelew committed
0f185b16b593eb973621908bad159a47e406f9aa
Parent: 6c30153
Committed by GitHub <noreply@github.com> on 8/31/2026, 4:31:50 PM