SIGN IN SIGN UP

fix(dots3): adopt restored sliding slots into the latent lane

Root cause of the ~1.7MB/token span retention that walled dots3 at
every Metal ceiling (96.8->118.7GB of active over one 12.7k-token
prefill, released only at request end): the SSD-only hit lane hands
sliding slots back as EMPTY generic KVCache shells (the sliding
cumulative tuple only restores on exact boundaries), and
_adopt_restored_caches audited full layers only. Un-adopted sliding
slots run the materialized lane from the restore point on - 64-head
K/V at ~48KB/token/layer with the window enforced only by the mask, so
across 33 sliding layers nothing ever trims. Confirmed live: "dots3
adopted 13 restored full-layer cache(s)" with zero sliding adoptions
on the exact span that retained 22GB.

Adopt sliding slots too: empty shells re-enter the window-bounded
latent lane (both lanes mask from PHYSICAL length, so an empty window
start forgets the same <window of pre-restore context the KVCache lane
already forgot), and a reconstructor that carried (latent, k_pe)
content is adopted with state intact. Latent-scale memory is ~2.2KB
per token per sliding layer with hysteresis trims - a 22x reduction on
restored spans.
J
Jinho Jang committed
5ee0ba6b3bae2b93c2fa732fb7a10740904815eb
Parent: c5d3fea