llama.cpp/convert-new.py at afc4ca288989afea3903c962e1006695d50db8b2

mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-10-31 08:51:55 +00:00

Files

goerch afc4ca2889 convert : update convert-new.py with tokenizer fixes (#2614 )

* Merge tokenizer fixes into the gguf branch.

* Add test vocabularies

* Adapt convert-new.py (and fix a clang-cl compiler error on windows)

2023-08-14 20:20:04 +03:00

View Raw