Fix multi-turn crash on hybrid models with untrimmable caches (#2152)
_prefix_cache_trim_amount only returned None (forcing a cold prefill) when a cache had already evicted part of the prefix. Caches that cannot be trimmed at all -- ArraysCache, used by hybrid/linear-attention layers -- slipped through: they are fully retained but have no trim(), so the per-turn reuse loop's c.trim(n_drop) raised AttributeError on the second turn of any conversation. Also require is_trimmable() for every cache, so any non-trimmable cache forces a cold prefill instead of crashing. Dense and rotating caches are unaffected (all report trimmable); hybrid models still reuse the cache when nothing needs dropping (n_drop == 0).
A
Alazer Manakelew committed
89baff42bdece512ad76d5a8154188cbb9e23070
Parent: 85ae5dc
Committed by GitHub <noreply@github.com>
on 9/3/2026, 7:01:26 PM