CS348Project/llama.cpp
5
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-11-08 10:07:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
3ca2b100a958a75dae03655e06b2fe2b335393b6
llama.cpp/convert.py
teleprint-me 3ca2b100a9 Restore BpeVocab and SentencePieceVocab classes 
- Restored the BpeVocab class for handling BPE tokenization.
- Restored the SentencePieceVocab class for SentencePiece tokenization.

These classes are essential for maintaining the original behavior of the codebase.
2024-01-07 19:39:20 -05:00

57 KiB
Executable File
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink