SIGN IN SIGN UP

sm80 Marlin repack: grow-only staging for the per-layer transients (#27 candidate)

The GPU-resident version of ahnguyen17 CPU-fallback workaround: pad and
contiguous land in one persistent staging buffer per device, source dropped
before the output allocates, so per-layer allocation traffic collapses to
the output tensor alone. Bit-exact on the 3090 across the checkpoint real
shape ladder (w4+w8, padded/strided, regrow+reuse: kernel inputs and
outputs torch.equal). Default on only for compute capability 8.0; costs up
to ~1.2 GiB resident staging there (the quantized embedding whale sizes
it). Whether it prevents the Xid-31 needs the 170HX test.
M
mhenrichsen committed
3cc6afc453cb0e40bb17d6608d3295d7f0033ea2
Parent: 60921ed