Kyle Mistele 37246b1031 common : revert showing control tokens by default for server (#6860) ...

* fix: revert showing control tokens by default

* feat: revert changes to default behavior of llama_token_to_piece; provide overridden declaration to receive "bool special" param to toggle showing control tokens

* feat: use the overridden declaration of llama_token_to_piece from common/common.cpp to specify "false" so that control tokens are not shown in chat completion responses"

* common : simplify

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

2024-04-24 13:15:29 +03:00

152 KiB

Raw Blame History

View Raw