SIGN IN SIGN UP

internal/server: parse vLLM speculative decoding metrics (#1039)

vLLM started with --per-request-spec-decode-metrics reports draft token
counts in the response's metrics.speculative_decoding object. Map those
onto the existing draft token fields so the activity table's drafted
acceptance rate works for vLLM the same way it already does for
llama-server's timings.

- read num_draft_tokens/num_accepted_draft_tokens from
  metrics.speculative_decoding
- require both counters so the rate is never derived from a partial
  object
- cover non-streaming, streaming and partial-object responses in tests

fix: #1032
B
Benson Wong committed
0bdb3720199f530b622a1934c0db89f32c94819c
Parent: cb53616
Committed by GitHub <noreply@github.com> on 8/22/2026, 4:29:57 PM