SIGN IN SIGN UP

CUDA: quantized KV support for FA vec (#7527)

* CUDA: quantized KV support for FA vec

* try CI fix

* fix commented-out kernel variants

* add q8_0 q4_0 tests

* fix nwarps > batch size

* split fattn compile via extern templates

* fix flake8

* fix metal tests

* fix cmake

* make generate_cu_files.py executable

* add autogenerated .cu files

* fix AMD

* error if type_v != FP16 and not flash_attn

* remove obsolete code
J
Johannes Gäßler committed
9b596417af11c9ac44fcae0fcfbc6f3665089083
Parent: a323ec6
Committed by GitHub <noreply@github.com> on 6/1/2024, 6:44:14 AM