SIGN IN SIGN UP

llama: Add attention and final logit soft-capping, update scaling factor to Gemma2 (#8197)

* Add attention and final logit softcapping.

* fix

* Add custom add_ functions

* Disable flash attention for Gemma2

* Update src/llama.cpp

Co-authored-by: slaren <slarengh@gmail.com>

* Add default value for attention and final logit softcap value

* Add custom kq scaling from Gemma2Attention

* Remove custom pre attention scaling and use computed value instead.

---------

Co-authored-by: slaren <slarengh@gmail.com>
A
Andrei committed
1c5eba6f8e628fb0a98afb27d8aaeb3b0e136451
Parent: 72272b8
Committed by GitHub <noreply@github.com> on 6/30/2024, 3:44:08 AM