SIGN IN SIGN UP

A high-throughput and memory-efficient inference and serving engine for LLMs

0 0 122 Python

[Model] Add num_cached_tokens for PoolingRequestOutput (#27378)

Signed-off-by: wang.yuqi <noooop@126.com>
W
wang.yuqi committed
3729ed00ba4c5a9f351b1241b15a2ca3ca12a5e1
Parent: 6644796
Committed by GitHub <noreply@github.com> on 10/23/2025, 6:03:42 AM