feat: Add {% generation %} chat template for DiffusionGemma SFT/LoRA examples (#3353)
The released google/diffusiongemma-26B-A4B-it chat template has no
{% generation %} tags, so format_chat_template falls back to
_build_multiturn_assistant_mask. That path derives the supervised region from
prefix lengths rendered without add_generation_prompt=True, so it starts three
tokens early -- at '<|turn>model\n' rather than after it. The model is trained
to re-emit the turn header, and because 'model' is an ordinary non-special
token it survives detokenization and shows up as a stray 'model\n' at the
start of generations.
Adds examples/dllm_sft/diffusion_gemma_chat_template.jinja, a copy of the
released template that captures the assistant content and its turn close into
a single generation block, and points dataset.chat_template in
diffusion_gemma_sft.yaml and diffusion_gemma_lora.yaml at it. Rendered text is
byte-identical to the released template, so inference behavior is unchanged.
This applies the pattern llada_sft.yaml already uses for LLaDA-8B-Base.
Fixes the loss-mask boundary reported in #3352.
Signed-off-by: Tianqianjin Lin <lintqj@zju.edu.cn> T
Tianqianjin Lin committed
4a48864bb36c23ea59cd4014d21564139a092009
Parent: 38f4464
Committed by GitHub <noreply@github.com>
on 8/5/2026, 6:30:33 AM