fix(glm): align and diagnose cross-framework router parity (#3635)
* fix(glm): diagnose cross-framework router drift Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * test(glm): calibrate routed parity profiles Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * test(ci): add parity shape and router diagnostics Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * fix(ci): auto-report shortened parity gate shape Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * docs(ci): explain GLM parity gate rationale Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * docs(ci): clarify parity diagnostic purposes Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * test(ci): enable GLM-4.5-Air robustness coverage Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * docs(ci): clarify router diagnostic support Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * fix(ci): size GLM-4.5 robustness batches for EP32 Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * test: compute parity cosine in float64 Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * test: keep GLM-4.5 robustness experiment-only Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * fix(glm): own router precision and parity diagnostics Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * test(ci): enable GLM-5.2 checkpoint robustness, AutoModel-side phases Enable the checkpoint-robustness suite for GLM-5.2 with the cross-framework phases (vanilla-HF source parity and HF reload) skipped: they load the 1.5TB checkpoint in a single rank-0 process (~50 min just to load), idling the 32-node allocation. Cross-framework parity was verified out-of-band via a single-node CPU-offload vanilla-HF reference against the EP64 AutoModel stack on the bundled parity document: mean KL 0.0049 at 512 tokens (passes the standard profile); 0.0071/0.0065 at 2K/4K with p95 within standard and no divergent-token tail (median ~0.002, max 0.57) - a 78-layer bf16 noise floor, not routing instability. Also set model_save_format/save_consolidated in the checkpoint block: the train_and_save phase runs the recipe's checkpoint config verbatim, and without save_consolidated: final the save produces sharded safetensors plus an offline consolidate.sh only, leaving the reload phase an empty consolidated directory (verified empirically; inline distributed consolidation of the 1.4TB export measured 4m34s end-to-end on 8 nodes, well within the CI time budget). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * test(glm): update backend assertions for fp32 routing Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * fix(glm): harden parity diagnostics Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * fix(checkpoint): harden GLM resume diagnostics Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * fix(checkpoint): stabilize GLM5.2 resume validation Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> * refactor(deepseek): require explicit MLA latent norm epsilon The shared MLA is reused by DeepSeek-V3, Mistral 4, and GLM MoE Lite, so a hidden epsilon default would let a future family silently inherit numerics it never chose. Each Block now states its checkpoint contract explicitly; all current upstream implementations construct the two latent RMSNorms with the Hugging Face reference default of 1e-6 rather than config.rms_norm_eps, and the HF config carries no field for it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> --------- Signed-off-by: Yuhe Zhang <yuhez@nvidia.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Y
Yuhe Zhang committed
37ff7feacbfdc14a1c69d178eb3f9a9119f1ab0d
Parent: 792a6cd
Committed by GitHub <noreply@github.com>
on 8/28/2026, 6:15:15 PM