🚀🚀 「大模型」2小时完全从0训练64M的小参数GPT！🌏 Train a 64M-parameter GPT from scratch in just 2h!
fix: attn_forwad when is_causal=True assert attn_mask is None
Y
yuyu5333 committed
7d02ce673cdbfe54a47c1a241a74c4b9620ca3e0
Parent: 9c98cab
🚀🚀 「大模型」2小时完全从0训练64M的小参数GPT！🌏 Train a 64M-parameter GPT from scratch in just 2h!