llama: Add attention and final logit soft-capping, update scaling factor to Gemma2 (#8197)
* Add attention and final logit softcapping. * fix * Add custom add_ functions * Disable flash attention for Gemma2 * Update src/llama.cpp Co-authored-by: slaren <slarengh@gmail.com> * Add default value for attention and final logit softcap value * Add custom kq scaling from Gemma2Attention * Remove custom pre attention scaling and use computed value instead. --------- Co-authored-by: slaren <slarengh@gmail.com>
A
Andrei committed
1c5eba6f8e628fb0a98afb27d8aaeb3b0e136451
Parent: 72272b8
Committed by GitHub <noreply@github.com>
on 6/30/2024, 3:44:08 AM