llama : more checks before assuming FIM tokens (#7644)
* More checks before assuming FIM tokens for Llama arch * extensive token check
S
Sigbjørn Skjæret committed
6fcd1331efbfbb89c8c96eba2321bb7b4d0c40e4
Parent: 41b9260
Committed by GitHub <noreply@github.com>
on 6/14/2024, 10:20:04 AM