[nixlbench] add worker part for device api (#2015)
## What? Integrate Device API into nixlbench so transfers can be initiated from a CUDA kernel instead of the host path. Adds --use_device_api to enable GPU-kernel-initiated PUT transfers. When enabled, --num_threads is repurposed as the CUDA kernel block size, and the internal CPU thread count is forced to 1. This builds on the kernel launch support landed in #1761. ## Why? nixlbench could not exercise Device API end-to-end now. Wire the existing device kernels into the worker so we can benchmark and validate GPU-initiated WRITE transfers under a supported configuration. ## How? - Add --use_device_api and validate required options at config parse time (UCX, VRAM↔VRAM, --enable_pt, pairwise, SG, WRITE, pipeline_depth=1, and a build with HAVE_UCX_GPU_DEVICE_API). - In the NIXL worker, prepare local/remote nixlMemViews, allocate/exchange a VRAM completion counter, and launch nixlbenchLaunchDevicePut for the transfer path. - On the target, wait on the device completion/error counters instead of host-side request completion. - Keep the existing host transfer path unchanged when --use_device_api is not set. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an optional NIXL Device API mode for GPU-based transfer benchmarking. * Added GPU-resident completion tracking and automatic monitoring for warmup and timed transfers. * Added GPU memory preparation and cleanup for local and remote transfer views. * Added validation and status reporting for device API settings, including CUDA block size and supported transfer configurations. * **Bug Fixes** * Device transfers no longer require remote completion signaling to be explicitly enabled. * Completion and error signaling now consistently use the remote counter region. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
F
fteng-NV committed
63e2bce61d867f643ebd17f2c4af3ec98fc3c5af
Parent: 3e26bfd
Committed by GitHub <noreply@github.com>
on 8/12/2026, 1:07:26 PM