SIGN IN SIGN UP
milvus-io / milvus UNCLAIMED

Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search

0 0 164 Go

enhance: optimize QueryCoord ChannelDistManager collection filtering (#49587)

## Summary

This PR optimizes QueryCoord `ChannelDistManager` under the 100K
collections recovery scenario.

The main change is to optimize the ChannelDist write/update path.
`ChannelDistManager.Update` no longer rebuilds a global collection index
while holding the manager write lock. This shortens the write-side
critical section and reduces read-lock holding time during QueryNode
distribution updates, so Scheduler can observe distribution changes and
dispatch recovery tasks sooner.

The expected tradeoff is that part of the cost moves to the read path:
`GetByCollectionAndFilter` is removed, while `GetByFilter` becomes the
unified filtering path and increases in the profile. The increase is
bounded, and the overall 30-minute CPU profile still decreases.

## Profile Result

100K collections, 30-minute MixCoord CPU profile:

| Metric | master | this PR | change |
|---|---:|---:|---:|
| Total CPU samples | `1,672,474` | `1,598,355` | `-74,119 / -4.43%` |
| `ChannelDistManager.Update` | `39,981 / 2.39%` | `20,588 / 1.29%` |
`-48.5%` |
| `ChannelDistManager.GetByCollectionAndFilter` | `22,416 / 1.34%` | not
sampled | cleared |
| `ChannelDistManager.GetByFilter` | `22,771 / 1.36%` | `55,853 / 3.49%`
| `+145.3%` |

## Code Changes

- Remove the global `collectionIndex` from `ChannelDistManager`.
- Remove `GetByCollectionAndFilter` and migrate callers to `GetByFilter`
with `WithCollectionID2Channel`.
- Reuse per-node `nodeChannels.collChannels` for collection-scoped
lookup instead of rebuilding a global index on every update.
- Reduce temporary allocations in ChannelDist and SegmentDist filtering
paths.
- Update QueryCoord callers and tests for the unified ChannelDist filter
path.

Fixes #49511

## Test Plan

- `git diff --check upstream/master...HEAD`
- `gofumpt -l $(git diff --name-only upstream/master...HEAD -- '*.go')`
- Extended `TestGetChannelDistJSON` to cover `GetChannelDist(100)` and
`GetLeaderView(0|100|300)`.
- Updated `TestBalanceChecker_ConstructNormalBalanceQueue_*` mocks to
use the unified `GetByFilter` API.

---------

Signed-off-by: sijie-ni-0214 <sijie.ni@zilliz.com>
S
sijie-ni-0214 committed
ad95e8b86e388f750d8d20e62ab8c80fb0b7c07b
Parent: ef1f2a9
Committed by GitHub <noreply@github.com> on 5/18/2026, 5:44:30 PM