SIGN IN SIGN UP

perf(search): SWAR popcount + max-heap for the semantic Hamming scan (RFC 0001 §10)

§10(B) flagged the binary-code Hamming scan (~14% of search CPU) as the
next lever. Measure-first redo: cpu-prof's self-time attribution misled
(it pinned 93.8% on insertSorted), but direct wall-clock isolation showed
the ~40ms scan over 831k 64-byte codes is ~99% popcount, not selection.

Fix (output byte-identical): (a) a SWAR popcount over Uint32 views
(hammingU32 — 16 word-ops vs 64 byte-LUT lookups), scan 40 -> 15.6ms;
(b) replace the O(K)-splice insertion sort with a fixed-size max-heap keyed
(dist, idx) — admit iff d < root, evict the max (dist,idx), final sort
(dist asc, idx asc). Neutral at the default K=200 but O(K)->O(log K)
protects the configurable shortlist (<=5000).

End-to-end semantic search 55.8 -> 18.9ms (3.0x); eval:search byte-flat
(recall/ndcg/mrr identical, 168 judgments x 4 configs); full-search
semantic p50 1.4-1.6x. A property test locks the output-identity (heap ==
old insertSorted across tie-heavy / boundary / both code paths). Lesson:
trust direct timing over sampler self-time.

(cherry picked from commit 11b1c3453183708f7ef6457688d83f45cc22b0db)
G
Gigi committed
8bd6160d2796cc50882804e92148a4446baf2cbe
Parent: a323586