libfabric: PT-owns-endpoint with MPSC lock-free ring (#1949)
libfabric: Move fi_writemsg/fi_readmsg posting into the Progress Thread (when progress thread is enabled) via a lock-free MPSC ring buffer, eliminating ep_mutex_ contention on the data path, resulting in performance improvement with larger batch sizes. Main thread enqueues to per-rail MpscRing; PT dequeues, posts, and polls completions in a single thread. - MpscRing<T,2048>: CAS multi-producer, single-consumer ring - drainPostQueue: PT posts with FI_MORE batching, interleaves CQ - cudaSetDevice in PT for multi-GPU VRAM buffer validation - Graceful error: completion callback on failure, continue draining <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Performance** - Added a deferred, progress-thread-aware per-rail post queue that batches requests and interleaves completion draining with retry on temporary backpressure. - Introduced configurable post-queue sizing (default: 2048) to improve transfer throughput under progress-thread mode. - **Reliability** - Strengthened CUDA context validation and selection for CUDA VRAM transfers, with more defensive handling of mismatches and query failures. - Improved handling of `-FI_EAGAIN` so progress continues via CQ drain/retry instead of stopping early. - Enabled per-transfer device targeting for multi-GPU CUDA operations. - **Chores** - Added clearer error reporting when per-rail post queue initialization fails in progress-thread mode. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Adit Ranadive <aranadive@nvidia.com>
O
Oren Amor committed
3d764d0e62ea78999f2c55106d974dd5a28100d1
Parent: a9a66cd
Committed by GitHub <noreply@github.com>
on 7/29/2026, 2:53:27 AM