CI: run UCX master nightly instead of on every PR (#1908)
## What?
UCX `master` moves off the per-PR path to a nightly run. The 3 GPU test
jobs (`nixl-ci-gpu`, `nixl-ci-dl-gpu`, `nixl-ci-dl-gpu-ep`) test one UCX
version per run via a new `UCX_VERSION` param (default `v1.22.x`); a
nightly cron reruns each against `master`.
- Drop the `master` leg from the `ucx_version` axis in the 3 test
matrices (version now from `${UCX_VERSION}`).
- Add `UCX_VERSION` param + nightly cron `H 2 * * * %UCX_VERSION=master`
to the 3 job templates in `proj-jjb.yaml`.
- Update `.ci/docs/ci-overview.md`.
## Why?
A `master` regression often failed on unrelated PRs, and doubled the
pressure on scarce DL cluster resources. Nightly keeps the signal
without blocking PRs. Per-PR still covers `v1.22.x`.
## How?
`UCX_VERSION` drives the version everywhere the matrices build, tag, and
name. The dispatcher passes only `sha1`/`githubData` to the leaf jobs,
so PR runs use the `v1.22.x` default. Nightly cron overrides to
`master`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a nightly UCX `master` validation orchestrator that runs GPU
tests in parallel, aggregates results, and supports consolidated email
reporting.
* **CI Improvements**
* Updated GPU/distributed GPU CI matrices to use a single `UCX_VER`
value (default `v1.22.x`) and simplified Docker/Slurm identifiers to be
`BUILD_NUMBER`-based rather than including per-run UCX version segments.
* **Documentation**
* Refreshed CI scheduling/job-template docs and adjusted
core-dump/crash-debug instructions to use the updated Enroot `.sqsh`
naming based on `BUILD_NUMBER` only.
<!-- end of auto-generated comment: release notes by coderabbit.ai --> A
Alexey Rivkin committed
7b8129f972e20a6141a86aa97d1d8b0252158ba2
Parent: 21cfcd4
Committed by GitHub <noreply@github.com>
on 8/6/2026, 6:42:51 AM