SIGN IN SIGN UP

libfabric: PT-owns-endpoint with MPSC lock-free ring (#1949)

libfabric: Move fi_writemsg/fi_readmsg posting into the Progress Thread
(when progress thread is enabled) via a lock-free MPSC ring buffer,
eliminating ep_mutex_ contention on the data path, resulting in
performance improvement with larger batch sizes. Main thread enqueues to
per-rail MpscRing; PT dequeues, posts, and polls completions in a single
thread.

- MpscRing<T,2048>: CAS multi-producer, single-consumer ring
- drainPostQueue: PT posts with FI_MORE batching, interleaves CQ
- cudaSetDevice in PT for multi-GPU VRAM buffer validation
- Graceful error: completion callback on failure, continue draining

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Performance**
- Added a deferred, progress-thread-aware per-rail post queue that
batches requests and interleaves completion draining with retry on
temporary backpressure.
- Introduced configurable post-queue sizing (default: 2048) to improve
transfer throughput under progress-thread mode.

- **Reliability**
- Strengthened CUDA context validation and selection for CUDA VRAM
transfers, with more defensive handling of mismatches and query
failures.
- Improved handling of `-FI_EAGAIN` so progress continues via CQ
drain/retry instead of stopping early.
  - Enabled per-transfer device targeting for multi-GPU CUDA operations.

- **Chores**
- Added clearer error reporting when per-rail post queue initialization
fails in progress-thread mode.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Adit Ranadive <aranadive@nvidia.com>
O
Oren Amor committed
3d764d0e62ea78999f2c55106d974dd5a28100d1
Parent: a9a66cd
Committed by GitHub <noreply@github.com> on 7/29/2026, 2:53:27 AM