fix: clear import segments NumOfRows when import task retries (#49986)
issue: #49985 ## Summary When a DataNode running an import task fails or returns `Retry`, DataCoord resets the task to `Pending` and reschedules it on another node. The preallocated segment IDs are reused on retry, but `PickSegment` on the DataNode picks **randomly** whenever more than one segment exists for a `(vchannel, partition)` (https://github.com/milvus-io/milvus/blob/master/internal/datanode/importv2/util.go#L150-L162), so the retried attempt can write to a different subset of the preallocated segments than the failed attempt did. `NumOfRows` updates from progress reports were monotonic-only and were never cleared across attempts. As a result, segments touched by the failed attempt but skipped by the retried attempt kept stale `NumOfRows` without any insert binlogs being persisted (DataCoord only persists binlogs on `Completed`, https://github.com/milvus-io/milvus/blob/master/internal/datacoord/import_task_import.go#L209-L234). Sort compaction later read this stale `NumOfRows` via `originSegment.GetNumOfRows()` (https://github.com/milvus-io/milvus/blob/master/internal/datacoord/import_util.go#L899), found zero binlog rows for the orphan segment, and failed with `unexpected row count` or EOF (https://github.com/milvus-io/milvus/blob/master/internal/datanode/compactor/sort_compaction.go#L335-L343). ## Fix - New operator `ResetImportingSegmentRows(segmentIDs ...int64)` in `internal/datacoord/meta.go` — clears `NumOfRows` and `MaxRowNum` only on segments still in `Importing` state. - In `(*importTask).QueryTaskOnWorker`, call it on every preallocated segment **before** transitioning the task back to `Pending`. If the reset fails, the state transition is skipped so the next polling tick retries the whole sequence. - Orphan segments touched by the failed attempt but skipped by the retried attempt now have `NumOfRows = 0` after the retry completes, so `createSortCompactionTask`'s zero-rows guard drops them instead of feeding stale counts into sort compaction. ## Test plan - [x] `TestImportTask_QueryTaskOnWorker/QueryImport_rpc_failed_resets_NumOfRows` — new subtest verifies that `NumOfRows` on every preallocated segment is reset to 0 and the task transitions to `Pending` when QueryImport returns an error. - [x] Existing `TestImportTask_QueryTaskOnWorker` subtests still pass (the existing `QueryImport rpc failed` subtest's meta was extended with `segments: NewSegmentsInfo()` so the reset code path runs without nil-deref). - [x] `golangci-lint run ./internal/datacoord/` is clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: bigsheeper <yihao.dai@zilliz.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Y
yihao.dai committed
650f81d89bbd19f93de8593453e225b5ea37df43
Parent: 1f123a8
Committed by GitHub <noreply@github.com>
on 5/25/2026, 5:54:36 PM