feat(cli): suggest SSD-only caching on low-memory Macs at session startup
The in-RAM paged KV mirror (L1) shares unified memory with the model's weights. On a 16-32GB Mac that trade is usually a loss, and the SSD block cache (L2) gives nearly all of the benefit without taking memory the model needs. Measured on Qwen3.8-27B at default capacities, same workload, only the RAM mirror differing: paged RAM + L2 cold 20.18s full hit 2.68s (7.5x) partial 3.48s (5.8x) SSD-only cold 19.81s full hit 2.70s (7.3x) partial 3.57s (5.5x) Within about 2 percent. The suggestion therefore costs the user almost nothing in speed while freeing gigabytes for weights. It is a suggestion and nothing else. It does not disable the paged cache, does not mutate any argument, and does not refuse to start. An invented RAM guard once refused to load big models across six releases on exactly the hardware this product exists for, so a guess about memory must never acquire veto power over a user's configuration. When RAM cannot be detected the advisory stays silent rather than becoming a warning of its own, because a failed detection must not turn into a user-facing restriction either. It only appears when the paged cache is actually on, calls out when the L2 cache is off so the advice is actionable, and the threshold is tunable through VMLX_LOW_RAM_ADVISORY_GB for support cases. Tests cover the ladder from 16GB to 128GB, silence on failed detection, and an assertion that the advisory leaves the parsed arguments byte-identical.
J
Jinho Jang committed
200e6ba7ccd14872d510fe74f55df4c0a82dcc38
Parent: 8208b36