SIGN IN SIGN UP

fix(run-engine): stop task retries consuming the queue nack budget (#4810)

## Summary

A run whose task retries were delayed long enough to go back through the
queue could end up failed with `TASK_RUN_DEQUEUED_MAX_RETRIES` and
status `SYSTEM_FAILURE` even though every attempt had actually executed.
The real failure from the final attempt was replaced by that placeholder
error, and tasks configured for more retries than the queue redelivery
limit never got them.

## Root cause

Retries with a delay at or above the warm-start threshold are requeued
via `tryNackAndRequeue`, which nacks the queue message. `nackMessage`
increments the message attempt counter by default and dead-letters the
message once it reaches the queue retry limit. That counter is meant to
bound dequeues that never reach execution; a task retry after a
completed attempt was being charged against it anyway, so a long-backoff
retry schedule exhausted it.

## Fix

`nackMessage` gains a `resetAttemptCount` option that zeroes the counter
instead of incrementing it. `tryNackAndRequeue` exposes it as
`resetQueueAttempts`, and the attempt-retry path passes it, since a
completed attempt proves the run can start. The dequeue-failure and
stalled `PENDING_EXECUTING` paths keep incrementing, as those are the
genuine "could not start" cases the budget exists for.

Tests cover the queue-level reset (no dead-letter at the limit) and an
engine-level run that retries past the queue limit and finishes with its
own error rather than a system failure.
M
Matt Aitken committed
1f8f23027d1ace8c8e32ce022217524415df0464
Parent: 82ea723
Committed by GitHub <noreply@github.com> on 8/28/2026, 5:12:20 PM