SIGN IN SIGN UP

Turn expandable_segments off on WSL instead of documenting it

Three separate people lost a day to this (#2, #26, and @willy92wins in #2), and
all three found the note in docs/docker.md only afterwards. The scripts now
detect WSL from /proc/sys/kernel/osrelease (or WSL_DISTRO_NAME) and default
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False there, printing a line saying
so. Native Linux is untouched, where gotcha 3 applies. Explicit values still win
in both directions.

The reason a note was never going to be enough is that the failure does not
present as an allocator problem. One CUDA VMM rejection inside
process_weights_after_loading / gptq_marlin_repack, four messages:

  RuntimeError: CUDA driver error: device not ready        610.74, 610.62, 610.57
  RuntimeError: CUDA driver error: out of memory           591.86
  torch_call_dispatcher("aten::empty", ...) ops.h:631      610.62
  dxgkio_make_resident: Ioctl failed: -12                  dmesg, alongside all

The second is not an OOM: same failure at GPU_UTIL 0.75 and 0.90 with 23 GiB
free for a 16 GiB model, and :False fixes it at 0.93. Anyone searching the OOM
string rather than the ENOMEM one was landing nowhere.

Also corrects this note's claim that driver 591.86 does not hit it -- it does --
so the default is WSL-gated rather than driver-gated. And records the two things
the venv path needs on WSL2 that the container does not: nvcc missing from PATH
(reported as EACCES, because WSL inherits a Windows PATH with ACL-restricted
directories and execvp prefers EACCES to ENOENT), and the fp8 KV default sending
FlashInfer into a ninja build that can fail there.
M
mhenrichsen committed
ac7a9f462daac6fa77d843be7ba2e660dcde5e98
Parent: 1d36852