SIGN IN SIGN UP

speculative : add heuristic algorithm (#3006)

* Add heuristic algo for speculative

* Constrain minimum n_draft to 2

* speculative : improve heuristic impl

* speculative : be more rewarding upon guessing max drafted tokens

* speculative : fix typos

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
L
Leng Yue committed
35f73049af6c676a106a5a990a819ae0bc3fcd7d
Parent: 71ca2fa
Committed by GitHub <noreply@github.com> on 9/14/2023, 4:14:44 PM