[Feature][EPLB] Add asynchronous EPLB support for Model Runner V2 (#14099)
### What this PR does / why we need it? This PR completes the asynchronous Model Runner V2 EPLB follow-up to RFC #13410. It keeps vLLM's EPLB controller, load-window lifecycle, default placement policy, placement state, and asynchronous worker as the source of truth, while adding the Ascend adapters required for graph-safe load collection, staged expert movement, and routing refresh. This PR deliberately does not add or override an EPLB placement policy. Policy selection and execution use the upstream `default` implementation. Refs #13410 Implementation and review hotspots: - Organize the shared Ascend EPLB state and communicator under `vllm_ascend.distributed.eplb`, preserving the intended directory refactor. - Maintain a graph-stable replica routing table and map logical expert IDs to local physical experts in a graph-safe device operation. Physical-expert load recording excludes padding and follows the configured `load_collection_phase`. - Preserve quantization-owned expert weight views so asynchronous movement updates the weights used by the active fused-MoE implementation. - Adapt upstream asynchronous transfers to Gloo CPU staging on Ascend. After a layer is committed, refresh its Ascend routing table before acknowledging the upstream worker result. - Require asynchronous mode and the Gloo communicator for MRv2 EPLB, and reject elastic EP because that combination is not supported. - Keep upstream release-contract compatibility isolated in the downstream state/controller adapters. ### Does this PR introduce _any_ user-facing change? Yes. Model Runner V2 EPLB is available on Ascend with asynchronous expert movement and the upstream default placement policy. Users enable it with `--enable-eplb`. Ascend selects `torch_gloo` when the communicator is unset, normalizes MRv2 EPLB to asynchronous mode, and supports `additional_config.eplb_config.load_collection_phase` with `all`, `prefill`, or `decode`. Elastic EP and non-Gloo communicators are rejected for this path. Legacy Model Runner V1 EPLB and EPLB-disabled execution are unchanged. ### How was this patch tested? - `bash format.sh ci`: all repository hooks passed. - Full mypy passed for Python 3.10, 3.11, and 3.12 across `vllm_ascend`, `examples`, and `tests` in a remote Linux lint environment. - Focused unit tests: 52 passed, covering EPLB state, communicator staging, routing/load operations, configuration validation, ordered async acknowledgement, fused-MoE integration, and load-collection phase handling. - Runtime-stack preflight passed with the pinned vLLM source and required Ascend custom-op symbol available. - Real Ascend NPU routing/load operator test: 1 passed. - Real four-card Ascend E2E: 1 passed with DP2 × TP2, EP world size four, W8A8, asynchronous scheduling, `FULL_AND_PIECEWISE`, prefill-only load collection, and four redundant experts. The test verified answer prefixes and observed a complete asynchronous EPLB cycle. ### Documentation - Updated the Model Runner V2 EPLB architecture to describe upstream ownership, graph-safe load recording, Gloo staging, and commit ordering. - Updated the EPLB user guide, additional configuration reference, support matrix, localization files, and Model Runner V1 architecture cross-links. - vLLM version: v0.27.1 - vLLM main: https://github.com/vllm-project/vllm/commit/ba07e4a48fc951300d97eb506217dd530583dea3 --------- Signed-off-by: freyfwt <freytian1996@gmail.com>
F
freyfwt committed
d37529f4df49e9da7a50123f4f810e4657e57013
Parent: 07f0eac
Committed by GitHub <noreply@github.com>
on 8/28/2026, 11:47:48 AM