internal/server: parse vLLM speculative decoding metrics (#1039)
vLLM started with --per-request-spec-decode-metrics reports draft token counts in the response's metrics.speculative_decoding object. Map those onto the existing draft token fields so the activity table's drafted acceptance rate works for vLLM the same way it already does for llama-server's timings. - read num_draft_tokens/num_accepted_draft_tokens from metrics.speculative_decoding - require both counters so the rate is never derived from a partial object - cover non-streaming, streaming and partial-object responses in tests fix: #1032
B
Benson Wong committed
0bdb3720199f530b622a1934c0db89f32c94819c
Parent: cb53616
Committed by GitHub <noreply@github.com>
on 8/22/2026, 4:29:57 PM