Revert PR #6924 - go to reference when there's only one (#10094)

This PR reverts #6924, for the reasons stated in
https://github.com/zed-industries/zed/pull/6924#issuecomment-2033076300.

It also fixes an issue were the `find_all_references_task_sources`
wasn't cleaned up in all cases.

Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>