perplexity : faster HellaSwag via batching (#5017)
* perplexity : faster HellaSwag ggml-ci * perplexity : clean-up ggml-ci * perplexity : no need for decode_helper ggml-ci * perplexity : add comments * perplexity : option to specify max batched tasks via `n_parallel` * perplexity : remove HellaSwag restruction for n_batch
G
Georgi Gerganov committed
ad19812cda4062c9f154ef16315df41fbe6a770a
Parent: 682986a
Committed by GitHub <noreply@github.com>
on 1/18/2024, 1:33:01 PM