assistant2: Improve tracking of pending completions (#21186)

This PR improves the tracking of pending completions in `assistant2`
such that we actually remove ones that have been completed.

Release Notes:

- N/A