Unify non-Unicode file detection code (#62581)

Closes https://github.com/zed-industries/zed/issues/62464
Closes https://github.com/zed-industries/zed/issues/62212

As a bonus, fixes the project search not working in BOM'd UTF-16 files.

Release Notes:

- Fixed project search not working in some non-Unicode files