SIGN IN SIGN UP

feat(examples): add Qwen3-32B Tulu-3 finetune recipes (#3104)

* feat(examples): add Qwen3-32B Tulu-3 CP finetune recipes

Add cp1 and cp2 (cp_size=2) HellaSwag-style Tulu-3 SFT recipes for
Qwen3-32B using ChatDataset with a fixed sequence length and
max_length padding, enabling long-sequence cp1-vs-cp2 validation
without packing.

* feat(examples): add Qwen3-32B Tulu-3 chat template for assistant loss masking

The Qwen3 chat template shipped in the checkpoint rewrites earlier turns (drops <think> from non-final assistant turns), so ChatDataset cannot build the answer-only loss mask by prefix diffing and raises. Add a generation-block chat template so the tokenizer emits the assistant mask directly, wire it into the Tulu-3 CP1/CP2 recipes, and set max_steps=100.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Abhishree <abhishreetm@gmail.com>

* Add comment

Signed-off-by: Abhishree <abhishreetm@gmail.com>

---------

Signed-off-by: Abhishree <abhishreetm@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
A
Abhishree Thittenamane committed
e539952e8834991c08a4093bb10504c6588585a1
Parent: 8dae92f
Committed by GitHub <noreply@github.com> on 8/12/2026, 4:55:22 PM