SIGN IN SIGN UP

fix(prefix-cache): cumulative state keeps its dtype across the numpy bridge

The numpy store bridge casts bf16 cumulative/SSM state through fp32
(np.array(bf16) raises under MLX >= 0.32) and, unlike every KV branch,
recorded no orig_dtype - so restart/L2-path hybrid restores handed the
model FLOAT32 conv/SSM states. The GDN forward concatenates the conv
state with the incoming chunk, which promotes the block output and
then the residual stream to f32 for the rest of the request: every QMM
silently loses the bf16 fast path, AR and MTP alike. The L1 companion
lane preserves dtype, which is why in-process runs never showed it.

Wrap the cumulative meta as {"m": meta, "dt": [per-array dtypes]} at
store (JSON-safe through the block-disk meta), unwrap at both restore
sites (main cumulative and CacheList subs) and cast each array back.
Records stored before the wrapper unwrap to a no-op. fp32-native state
stays fp32 - only arrays that were bf16 at store time are cast back.
J
Jinho Jang committed
15ab55c09109fdfb16f9a57a5f74443f84282c6b
Parent: 08768f9