Fix gemma2 tokenizer convert (#8244)
* fix gemma2 tokenizer convert * remove scores * improve code, fix new line issue
X
Xuan Son Nguyen committed
5fac350b9cc49d0446fc291b9c4ad53666c77591
Parent: cb5fad4
Committed by GitHub <noreply@github.com>
on 7/1/2024, 11:07:23 PM