0 0 1 C++

llama : add support for SmolLm pre-tokenizer (#8609)

* Adding SmolLM Pre Tokenizer

* Update convert_hf_to_gguf_update.py

Co-authored-by: compilade <git@compilade.net>

* Update src/llama.cpp

Co-authored-by: compilade <git@compilade.net>

* handle regex

* removed .inp and out .out ggufs

---------

Co-authored-by: compilade <git@compilade.net>

Jason Stillerman committed 2y ago

d94c6e0ccbd29ee1ba4f44e9caa8682ad94df9fa

Parent: 566daa5

Committed by GitHub <noreply@github.com> on 7/22/2024, 2:43:01 PM