feat(examples): add Qwen3-32B Tulu-3 finetune recipes (#3104)
* feat(examples): add Qwen3-32B Tulu-3 CP finetune recipes Add cp1 and cp2 (cp_size=2) HellaSwag-style Tulu-3 SFT recipes for Qwen3-32B using ChatDataset with a fixed sequence length and max_length padding, enabling long-sequence cp1-vs-cp2 validation without packing. * feat(examples): add Qwen3-32B Tulu-3 chat template for assistant loss masking The Qwen3 chat template shipped in the checkpoint rewrites earlier turns (drops <think> from non-final assistant turns), so ChatDataset cannot build the answer-only loss mask by prefix diffing and raises. Add a generation-block chat template so the tokenizer emits the assistant mask directly, wire it into the Tulu-3 CP1/CP2 recipes, and set max_steps=100. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add comment Signed-off-by: Abhishree <abhishreetm@gmail.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
A
Abhishree Thittenamane committed
e539952e8834991c08a4093bb10504c6588585a1
Parent: 8dae92f
Committed by GitHub <noreply@github.com>
on 8/12/2026, 4:55:22 PM