SIGN IN SIGN UP

llama : allow pooled embeddings on any model (#7477)

* create append_pooling operation; allow to specify attention_type; add last token pooling; update examples

* find result_norm/result_embd tensors properly; update output allocation logic

* only use embd output for pooling_type NONE

* get rid of old causal_attn accessor

* take out attention_type; add in llama_set_embeddings

* bypass logits when doing non-NONE pooling
D
Douglas Hanley committed
80ea089d771f0c2d97afa8bead80ded412f600d7
Parent: 0e64591
Committed by GitHub <noreply@github.com> on 6/21/2024, 5:38:22 AM