fix: stop generation on the tokenizer's EOS when the config omits it (#2112)
Chandra OCR 2 configures <|endoftext|> in generation_config.json but ends turns with <|im_end|>, so the configured EOS alone never stopped generation. Union both sources wherever the stop set is built.
A
Alazer Manakelew committed
6c301536963e51b31c0d20c3d581e6700ef8c76b
Parent: 09a987a
Committed by GitHub <noreply@github.com>
on 8/31/2026, 3:09:18 PM