SIGN IN SIGN UP

clip : support more quantization types (#4846)

Uses ggml functions instead of hardcoded names and adds support to quantize into the modern Q-K variants.
This is just the bare minimum to get k-types working - a more refined choice of types would be needed to get best quality on low quantizations.

I ran a few tests, it doesn't break anything I could notice and a Q6_K ViT works almost as well as Q8_0 but 3 times the inference speed.
J
John committed
d34633d8db6c2e400355de4862cd699154ecc73f
Parent: 4f56458
Committed by GitHub <noreply@github.com> on 1/10/2024, 1:37:09 PM