feat: KVFlash pool paging for hybrid full-attention KV
Port Lucebox-style decode-time KV residency: allocate attention cache at a fixed pool (LLAMA_KVFLASH), map logical positions via KvFlashPager, page cold 64-token chunks to host, and reselect every LLAMA_KVFLASH_TAU tokens (default LRU). GDN/SSM state is never paged. Fixes same-pos re-apply purge that wiped sequences, and guards token_to_piece for NULL/OOB tokens. Unit tests cover map, eviction, and reselect; short-gen quality matches full cache; gen past pool works.
A
andi committed
80f1de0fe25d8835b724b05e35bf5e3d2b627a4b
Parent: 6bd32c8