CS348Project/llama.cpp
5
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-10-31 08:51:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
725f23f1f3f0d3adf49f95d8dfa6e7c74adff149
llama.cpp/ggml/include/ggml-rpc.h
Radoslav Gerganov 553a5c3a9f rpc : do not wait for response when sending RPC_CMD_SET_TENSOR (#12943) 
RPC_CMD_SET_TENSOR always returns an empty response and we send this 4
times per token. We can improve TG speed if we don't wait for this empty
response.

The performance impact of this change depends on the network latency.
2025-04-25 10:08:08 +03:00

1.0 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink