perf(cache): stop re-running the whole prefix to attach an SSM companion
On the hybrid lane, a partial KV hit whose exact companion missed fell back to deriving that companion over the ENTIRE matched prefix. For a 15k prompt with a 12k paged hit that is ~12k tokens of hidden forward compute plus the 3k visible tail: the same total work as a cold prefill, while usage still reports cached_tokens=11968. The hit saved nothing it claimed to save, and additionally paid block reconstruction, a companion deepcopy, and the transient memory of a second full-prefix cache. The graceful resume ladder above it already looks for an earlier companion, but it can only accept a BLOCK-ALIGNED one, because it trims the KV back to the checkpoint and pairing SSM at an arbitrary length with KV floored to a block boundary re-feeds the gap through layers whose state already absorbed it -- a one-token version of that caused the v1.3.77 think-loop. Companion stores land on gpl-stripped prompt boundaries, which are essentially never multiples of 64, so the ladder misses them almost every time and drops through here. That alignment requirement does not apply to a derive. This path leaves the KV hit untouched and only advances the recurrent state up to the hit boundary, so ANY complete checkpoint below it is a valid seed. Same 15k/12k case with a checkpoint at 8,037: 6,963 tokens of forward work instead of 15,000. Resuming is not a new numeric regime -- the recurrent state advances exactly as it does across an ordinary chunk boundary, which is what the resume path and the default-on chunked re-derive already do. Guards kept deliberate: - an incomplete checkpoint is refused, because it was stored after the generation-prompt suffix was processed and seeding from it would bake that contamination into every future exact hit at the boundary - a DSV4 delta-capture derive still runs from token zero, since its anchors have to be stamped from a full pass to stay composable - the seed slices attention KV using OFFSET, never keys.shape: restored buffers are zero-padded up to the cache step, and reading the shape there is the mistake that silently emptied dots3 answers - caches needing constructor arguments are declined rather than sliced, since a typed or windowed cache is not position-sliceable - with no complete checkpoint at any length, the previous whole-prefix derive is still the last resort Text scheduler only. The MLLM twin still abandons the hit outright on a misaligned checkpoint and needs the same treatment; it is deeper in a nested path and a wrong recurrent splice there produces plausible-but-wrong output rather than a failure, so it is not being changed unverified.
J
Jinho Jang committed
2327f06ebc3ddabe19497c3d0afd66ee3d199864
Parent: 6c7f50c