Compute MoE load-balancing loss per layer to avoid giant one-hot materialization −99.7% @ 128k (#48131)
Compute MoE load-balancing loss per layer to avoid giant one-hot materialization
Q
Quentin Gallouédec committed
a353632607c59463e6ced86a44c2de3c2cd62d5e
Parent: a530903
Committed by GitHub <noreply@github.com>
on 8/24/2026, 6:46:24 PM