PLUGINS/UCX: Improve the performance by specifying memory type in sendAm(). (#2057)
## What? This PR improves the performance of NIXL by specifying memory type in sendAm(). ## Why? Each nixlbench batch triggers a notification (in postXfer function). The notification calls sendAm without providing mh or memory type, in which case UCX must detect the memory type of the notification buffer in ucp_memory_detect_internal. If memtype cache is NULL, ucp_memory_detect_internal enters the slow path and then queries all possible memory domians, including CUDA memory, which introduces significant overhead. Below is the experiment result with block size of 1024B: | Batch | no-opt BW (GB/s) | opt BW (GB/s) | Gain | |:-------:|:--------------------------:|:-----------------------:|:-----:| | 32 | 0.823 | 0.844 | 2.55% | | 64 | 0.944 | 0.961 | 1.80% | | 128 | 1.037 | 1.059 | 2.12% | | 256 | 1.103 | 1.121 | 1.63% | <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved compatibility and reliability for host-memory data transfers over UCX. * Preserved existing transfer flags and callback behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: ZhenlongMa <zhenlongm@nvidia.com>
Z
Zhenlong Ma committed
4d030b94e4705e660f2ebe9f39ef7b636bdd6204
Parent: 54ba75f
Committed by GitHub <noreply@github.com>
on 8/7/2026, 4:08:37 PM