llama : allow pooled embeddings on any model (#7477)
* create append_pooling operation; allow to specify attention_type; add last token pooling; update examples * find result_norm/result_embd tensors properly; update output allocation logic * only use embd output for pooling_type NONE * get rid of old causal_attn accessor * take out attention_type; add in llama_set_embeddings * bypass logits when doing non-NONE pooling
D
Douglas Hanley committed
80ea089d771f0c2d97afa8bead80ded412f600d7
Parent: 0e64591
Committed by GitHub <noreply@github.com>
on 6/21/2024, 5:38:22 AM