maor-ps 03fb8a002d If first token generated from the server is the stop word the server will crash (#7038) ...

This will reproduce the issue in llama13b
{
'prompt': 'Q: hello world \nA: ',
 'stop': ['\n'],
 'temperature': 0.0,
 'n_predict': 10,
 'cache_prompt': True,
 'n_probs': 10
}

2024-05-04 11:06:40 +02:00

153 KiB

Raw Blame History

View Raw