0 0 1 C++

py : add Phi-1.5/Phi-2 tokenizer (#9361)

* add phi2 tokenizer

* add phi name to convert_hf_to_gguf_update.py

* make tokenizer_pre consistent; llama.cpp work

daminho committed 2y ago

c837981bba7cf6839b69d32b25552ce685936b14

Parent: 3c26a16

Committed by GitHub <noreply@github.com> on 9/12/2024, 11:28:20 AM