🚀🚀 「大模型」2小时完全从0训练64M的小参数GPT!🌏 Train a 64M-parameter GPT from scratch in just 2h!
[fix] reduce aux_loss_alpha
J
jingyaogong committed
ecd1ae1563df48d3e295f275a7014bd3cbfc0daf
Parent: 5e1447b
🚀🚀 「大模型」2小时完全从0训练64M的小参数GPT!🌏 Train a 64M-parameter GPT from scratch in just 2h!