SIGN IN SIGN UP

grammar : pre-computed pieces + reserve mem + less string copies (#4330)

* reserve space for codepoints

* improvement for the appended 0

* used precomputed token text for grammar sample

* reserve canidates_decoded

* reserve canidates_grammar

* remove candidates_decoded

* Revert "remove candidates_decoded"

This reverts commit 3773328080e6a139ee83198329a13cf4ff61d707.

* changed decode_utf8 to take src by ref
M
Marcus Dunn committed
5f6e0c0dff1e7a89331e6b25eca9a9fd71324069
Parent: 5aa365d
Committed by GitHub <noreply@github.com> on 12/5/2023, 8:55:12 PM