SIGN IN SIGN UP

harden(cache): clamp every truncation through the LOGICAL extent, not the buffer

A live KVCache allocates keys/values in step (256) token chunks, so
keys.shape[seq] > offset is the normal steady state and the trailing rows are
allocation slack, not tokens. Two code shapes kept deriving a length from the
buffer anyway:

  safe_target = min(target_len, keys.shape[-2])   # offset missing from the clamp
  layer_cache.offset = prefix_len                 # assigned unconditionally

Both can turn slack into tokens. In a truncation path that feeds store_cache
that would persist zero rows into the prefix cache as though a user had sent
them, and _vmlx_trim_prompt_cache could RAISE an offset (100 -> 199) when
prefix_len landed between the offset and the buffer size.

utils/cache_extent.py adds logical_truncate_target(), applied at the 12
safe_target sites in scheduler.py, memory_cache.py and mllm_scheduler.py and
in _vmlx_trim_prompt_cache. offset == 0 is treated as UNKNOWN rather than
empty: some cache types never populate it and clamping those to zero would
delete a valid cache.

This is hardening, not a fix for an observed failure. Every caller today
passes target_len == offset, so no user-visible defect is known here; the
mllm_scheduler path was proven reachable with a padded cache and is one
changed caller away from being live. It is the same class that shipped in
v1.6.34 as the dots3 empty-answer corruption.

Audit also cleared, with probes: every store/serialize path funnels through
.state (which slices to :offset), so no disk poisoning is possible; the
tq_disk_store offset-from-shape sites read deserialized tensors whose on-disk
extent IS the logical length; and the mllm_scheduler quantize sites carry
offset correctly, so padding there is wasted work rather than corruption, and
was deliberately left alone because trimming trades quantize cost for a
realloc with no measurement behind it.
J
Jinho Jang committed
e6776b2943859d6f0aef450cd653f41be82ca11b
Parent: 9936e83