server : add speculative decoding support (#10455)
* server : add speculative decoding support ggml-ci * server : add helper function slot.can_speculate() ggml-ci
G
Georgi Gerganov committed
9ca2e677626fce759d5d95c407c03677b9c87a26
Parent: 5931c1f
Committed by GitHub <noreply@github.com>
on 11/25/2024, 2:31:38 PM