fix(cache): say what the stored cache actually does, and bench it that way
Three places still described the cache as quantized after the SSD-first overhaul made it full precision for every family. 1. The serve log announced "KV cache auto mode: TurboQuant enabled for compatible models; stored prefix cache quantization=none". Two different levers in one sentence, the misleading half first. Reading that line cost a whole investigation before the on-disk dtypes settled it: F16 keys and values, zero scale or bias tensors, nothing quantized anywhere in an 11GB live cache. It now leads with the cache and names the live working KV as the only thing TurboQuant touches. 2. bench defaulted stored quantization to q4 while serve used none, so any benchmark measured a codec the product no longer writes. The whole point of benching is to predict what users will see. Now identical to serve, with the same environment override for diagnostics. 3. The stored-cache selector still offered "Auto (engine-selected: native/TurboQuant + stored fallback)". Auto has meant full precision for every family since the overhaul, so the label advertised behaviour that no longer exists. Relabelled across all five locales. Auto stays the default rather than an explicit none, deliberately: passing --kv-cache-quantization with any value also sets VMLX_DISABLE_TQ_KV, which would switch off the calibrated live in-memory TurboQuant and raise resident memory. That is not what turning off stored quantization was for. q8 and q4 remain selectable wherever the architecture tolerates a lossy stored codec; families that need exact stored KV keep offering only auto and none.
J
Jinho Jang committed
6c7f50c17456c9176e8d9b3fbc6278bed105212c
Parent: 2c931e3