[ENH]: Propagate cancellations to NAC queue (#5129)
## Description of changes _Summarize the changes made by this PR._ - Improvements & Bug fixes - When a future running a NAC request gets dropped, it "removes" its priority from the pending request if it hasn't been sent out to object storage yet. - The priority data structure for in-flight requests has been updated from a binary value of priorities to a set of counters, denoting how many tasks requested each priority level. - New functionality - Ideally, the above changes should make it such that high priority NAC requests get lowered in priority when the calling requests are cancelled/timed out. ## Test plan _How are these changes tested?_ - [x] Tests pass locally with `pytest` for python, `yarn test` for js, `cargo test` for rust ## Migration plan N/A ## Observability plan It would be a good idea to add a metric that tracks the number of total waiters on the NAC layer. ## Documentation Changes _Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the [docs section](https://github.com/chroma-core/chroma/tree/main/docs/docs.trychroma.com)?_
T
tanujnay112 committed
fd4d8b43785909e69e5187c803c167a5debb4556
Parent: b6b059d
Committed by GitHub <noreply@github.com>
on 8/18/2025, 9:00:46 PM