SIGN IN SIGN UP

fix(model): correct Mistral4 attention and distributed MoE routing (#3348)

* fix(model): align Mistral4 attention scaling with HF

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(moe): synchronize expert load bias updates

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* test(vlm): calibrate Mistral4 parity thresholds

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* test(vlm): bound Mistral4 fresh-runtime parity

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* ci(vlm): install media dependencies directly

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(moe): replicate routing bias as DTensor

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(mistral4): preserve routing bias in fp32

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

* fix(moe): load routing bias into replicated DTensor

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>

---------

Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>
Y
Yuhe Zhang committed
ea16f15e458b8dabd7617327ccd1ae5ea1ab2925
Parent: 82c8493
Committed by GitHub <noreply@github.com> on 8/4/2026, 10:49:53 AM