Michael Podvitskiy 69ff61397d llama : support models without vocabulary (#5798) ...

* additional methods to read model and ctx parameters

* vocab size as a part of a model metadata

* models without vocabulary, convert.py part

* models without vocabulary, llama.cpp part

* PR clean up

* converter scrypt fixes

* llama_vocab_type update (renamed the new key)

* pr review fixes

* revert function renaming

* one more NoVocab assert

2024-03-14 18:21:56 +02:00

44 KiB

Raw Blame History

View Raw