grammar : pre-computed pieces + reserve mem + less string copies (#4330)
* reserve space for codepoints * improvement for the appended 0 * used precomputed token text for grammar sample * reserve canidates_decoded * reserve canidates_grammar * remove candidates_decoded * Revert "remove candidates_decoded" This reverts commit 3773328080e6a139ee83198329a13cf4ff61d707. * changed decode_utf8 to take src by ref
M
Marcus Dunn committed
5f6e0c0dff1e7a89331e6b25eca9a9fd71324069
Parent: 5aa365d
Committed by GitHub <noreply@github.com>
on 12/5/2023, 8:55:12 PM