SIGN IN SIGN UP

[Feature][EPLB] Add asynchronous EPLB support for Model Runner V2 (#14099)

### What this PR does / why we need it?

This PR completes the asynchronous Model Runner V2 EPLB follow-up to RFC
#13410. It keeps vLLM's EPLB controller, load-window lifecycle, default
placement policy, placement state, and asynchronous worker as the source
of truth, while adding the Ascend adapters required for graph-safe load
collection, staged expert movement, and routing refresh.

This PR deliberately does not add or override an EPLB placement policy.
Policy selection and execution use the upstream `default`
implementation.

Refs #13410

Implementation and review hotspots:

- Organize the shared Ascend EPLB state and communicator under
`vllm_ascend.distributed.eplb`, preserving the intended directory
refactor.
- Maintain a graph-stable replica routing table and map logical expert
IDs to local physical experts in a graph-safe device operation.
Physical-expert load recording excludes padding and follows the
configured `load_collection_phase`.
- Preserve quantization-owned expert weight views so asynchronous
movement updates the weights used by the active fused-MoE
implementation.
- Adapt upstream asynchronous transfers to Gloo CPU staging on Ascend.
After a layer is committed, refresh its Ascend routing table before
acknowledging the upstream worker result.
- Require asynchronous mode and the Gloo communicator for MRv2 EPLB, and
reject elastic EP because that combination is not supported.
- Keep upstream release-contract compatibility isolated in the
downstream state/controller adapters.

### Does this PR introduce _any_ user-facing change?

Yes. Model Runner V2 EPLB is available on Ascend with asynchronous
expert movement and the upstream default placement policy. Users enable
it with `--enable-eplb`. Ascend selects `torch_gloo` when the
communicator is unset, normalizes MRv2 EPLB to asynchronous mode, and
supports `additional_config.eplb_config.load_collection_phase` with
`all`, `prefill`, or `decode`.

Elastic EP and non-Gloo communicators are rejected for this path. Legacy
Model Runner V1 EPLB and EPLB-disabled execution are unchanged.

### How was this patch tested?

- `bash format.sh ci`: all repository hooks passed.
- Full mypy passed for Python 3.10, 3.11, and 3.12 across `vllm_ascend`,
`examples`, and `tests` in a remote Linux lint environment.
- Focused unit tests: 52 passed, covering EPLB state, communicator
staging, routing/load operations, configuration validation, ordered
async acknowledgement, fused-MoE integration, and load-collection phase
handling.
- Runtime-stack preflight passed with the pinned vLLM source and
required Ascend custom-op symbol available.
- Real Ascend NPU routing/load operator test: 1 passed.
- Real four-card Ascend E2E: 1 passed with DP2 × TP2, EP world size
four, W8A8, asynchronous scheduling, `FULL_AND_PIECEWISE`, prefill-only
load collection, and four redundant experts. The test verified answer
prefixes and observed a complete asynchronous EPLB cycle.

### Documentation

- Updated the Model Runner V2 EPLB architecture to describe upstream
ownership, graph-safe load recording, Gloo staging, and commit ordering.
- Updated the EPLB user guide, additional configuration reference,
support matrix, localization files, and Model Runner V1 architecture
cross-links.

- vLLM version: v0.27.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/ba07e4a48fc951300d97eb506217dd530583dea3

---------

Signed-off-by: freyfwt <freytian1996@gmail.com>
F
freyfwt committed
d37529f4df49e9da7a50123f4f810e4657e57013
Parent: 07f0eac
Committed by GitHub <noreply@github.com> on 8/28/2026, 11:47:48 AM