speculative : fix handling of some input params (#9963)
* speculative : fix batch sizes at initialization ggml-ci * speculative : handle params.n_predict == -1 * speculative : limit batch size to llama_n_batch
G
Georgi Gerganov committed
bc219750845a59166d79f0d4ee3da1993b369b8a
Parent: 1db8c84
Committed by GitHub <noreply@github.com>
on 10/21/2024, 6:37:12 AM