SIGN IN SIGN UP

feat: KVFlash pool paging for hybrid full-attention KV

Port Lucebox-style decode-time KV residency: allocate attention cache at
a fixed pool (LLAMA_KVFLASH), map logical positions via KvFlashPager,
page cold 64-token chunks to host, and reselect every LLAMA_KVFLASH_TAU
tokens (default LRU). GDN/SSM state is never paged.

Fixes same-pos re-apply purge that wiped sequences, and guards
token_to_piece for NULL/OOB tokens. Unit tests cover map, eviction, and
reselect; short-gen quality matches full cache; gen past pool works.
A
andi committed
80f1de0fe25d8835b724b05e35bf5e3d2b627a4b
Parent: 6bd32c8