SIGN IN SIGN UP

fix: stop generation on the tokenizer's EOS when the config omits it (#2112)

Chandra OCR 2 configures <|endoftext|> in generation_config.json but ends
turns with <|im_end|>, so the configured EOS alone never stopped generation.
Union both sources wherever the stop set is built.
A
Alazer Manakelew committed
6c301536963e51b31c0d20c3d581e6700ef8c76b
Parent: 09a987a
Committed by GitHub <noreply@github.com> on 8/31/2026, 3:09:18 PM