SIGN IN SIGN UP

sam3: fix pred_boxes inflation and offset (RoPE scale, RPB grid, ref_point_head) (#2128)

* sam3: scale global-attention RoPE grid to match the reference

Global-attention blocks span the full feature map but keep the
window-sized coordinate stride. HF applies this as
rotary_scale = window_size / rotary_input_size[0], which is 1.0 for
windowed blocks and window_size / feat_size for global ones.

compute_axial_cis had no scale, so global-block positions advanced
3x too fast at 1008px (24/72), pushing the vision features away from
the reference and causing some objects to be missed entirely.

* sam3: drop half-cell offset from the box RPB coordinate grid

The reference builds the RPB grid on cell corners (arange(n) / n). Adding
half a cell shifted every box-boundary delta, biasing the decoder's spatial
cross-attention down and right, so predicted boxes came out offset in the
same direction.

* sam3: drop spurious output activation from ref_point_head

The reference 2-layer decoder MLP applies ReLU only between its layers.
ref_point_head also applied one to its output, clamping query_pos to
non-negative values and discarding the negative half of the per-query
positional conditioning, which inflated every predicted box.
A
Alazer Manakelew committed
4b2d8f9a9ab759e0f88da21195a9743e44616e4e
Parent: 73fe1fb
Committed by GitHub <noreply@github.com> on 9/1/2026, 3:55:05 PM