CS348Project/llama.cpp
5
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-10-31 08:51:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,999 Commits 494 Branches 4,666 Tags
b9adcbbf92fc7096bee23fe61496d25652ebf765
Commit Graph

4 Commits

Author SHA1 Message Date
Radoslav Gerganov
db10f01310 rpc : track allocated buffers (#7411) 
* rpc : track allocated buffers

ref: #7407

* rpc : pack rpc_tensor tightly
 2024-05-20 16:36:55 +03:00
Radoslav Gerganov
f4bd8b3d26 rpc : set SO_REUSEADDR for the server socket (#7320) 
ref: #7293
 2024-05-17 17:25:44 +03:00
Radoslav Gerganov
3b3963c55c rpc : add command line arg for specifying backend memory 
ref: #7293
 2024-05-16 09:58:29 +03:00
Radoslav Gerganov
5e31828d3e ggml : add RPC backend (#6829) 
* ggml : add RPC backend

The RPC backend proxies all operations to a remote server which runs a
regular backend (CPU, CUDA, Metal, etc).

* set TCP_NODELAY

* add CI workflows

* Address review comments

* fix warning

* implement llama_max_devices() for RPC

* Address review comments

* Address review comments

* wrap sockfd into a struct

* implement get_alignment and get_max_size

* add get_device_memory

* fix warning

* win32 support

* add README

* readme : trim trailing whitespace

* Address review comments

* win32 fix

* Address review comments

* fix compile warnings on macos
 2024-05-14 14:27:19 +03:00