harden(cache): clamp every truncation through the LOGICAL extent, not the buffer
A live KVCache allocates keys/values in step (256) token chunks, so keys.shape[seq] > offset is the normal steady state and the trailing rows are allocation slack, not tokens. Two code shapes kept deriving a length from the buffer anyway: safe_target = min(target_len, keys.shape[-2]) # offset missing from the clamp layer_cache.offset = prefix_len # assigned unconditionally Both can turn slack into tokens. In a truncation path that feeds store_cache that would persist zero rows into the prefix cache as though a user had sent them, and _vmlx_trim_prompt_cache could RAISE an offset (100 -> 199) when prefix_len landed between the offset and the buffer size. utils/cache_extent.py adds logical_truncate_target(), applied at the 12 safe_target sites in scheduler.py, memory_cache.py and mllm_scheduler.py and in _vmlx_trim_prompt_cache. offset == 0 is treated as UNKNOWN rather than empty: some cache types never populate it and clamping those to zero would delete a valid cache. This is hardening, not a fix for an observed failure. Every caller today passes target_len == offset, so no user-visible defect is known here; the mllm_scheduler path was proven reachable with a padded cache and is one changed caller away from being live. It is the same class that shipped in v1.6.34 as the dots3 empty-answer corruption. Audit also cleared, with probes: every store/serialize path funnels through .state (which slices to :offset), so no disk poisoning is possible; the tq_disk_store offset-from-shape sites read deserialized tensors whose on-disk extent IS the logical length; and the mllm_scheduler quantize sites carry offset correctly, so padding there is wasted work rather than corruption, and was deliberately left alone because trimming trades quantize cost for a realloc with no measurement behind it.
J
Jinho Jang committed
e6776b2943859d6f0aef450cd653f41be82ca11b
Parent: 9936e83