grammars: 1.5x faster inference w/ complex grammars (vector reserves / reuses) (#6609)
* grammars: reserve rejects & next candidates * grammars: reuse new_stacks * grammars: fix missing sig change in llama.h * grammars: fix test (api changed) * grammars: update gbnf-validator.cpp * grammars: simpler syntax (no swap)
O
Olivier Chafik committed
cbaadc92942c50aab599a9e4c163afc1f44f7c26
Parent: 1bbdaf6
Committed by GitHub <noreply@github.com>
on 4/11/2024, 6:47:34 PM