fix(model): correct Mistral4 attention and distributed MoE routing (#3348)
* fix(model): align Mistral4 attention scaling with HF Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * fix(moe): synchronize expert load bias updates Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * test(vlm): calibrate Mistral4 parity thresholds Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * test(vlm): bound Mistral4 fresh-runtime parity Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * ci(vlm): install media dependencies directly Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * fix(moe): replicate routing bias as DTensor Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * fix(mistral4): preserve routing bias in fp32 Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * fix(moe): load routing bias into replicated DTensor Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> --------- Signed-off-by: Yuhe Zhang <yuhez@nvidia.com>
Y
Yuhe Zhang committed
ea16f15e458b8dabd7617327ccd1ae5ea1ab2925
Parent: 82c8493
Committed by GitHub <noreply@github.com>
on 8/4/2026, 10:49:53 AM