SIGN IN SIGN UP

metal : add Q2_K implementation (#1762)

* metal : add Q2_K implementation

27.1 ms / token on M2 Max 30-core GPU, so about the
same speed as Q4_0. Memory throughput is ~156 GB/s.

The access pattern used in the Q2_K
CUDA implementation resulted in significantly lower
performance (~31 ms/token).

* Fixing merge conflicts

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
K
Kawrakow committed
72ff5282bf0388c60821f504c4c8cc2b1f491aa6
Parent: 0bf7cf1
Committed by GitHub <noreply@github.com> on 6/8/2023, 7:28:21 PM