5 Commits

Author	SHA1	Message	Date
Georgi Gerganov	d00cbea63c	server : host-memory prompt caching (#16391) ... * minor : code style * server : fix prompt similarity calculation * server : initial host-memory prompt caching * cont * server : refactor * cont * cont : make the server task of the slot const * cont : minor [no ci] * server : cache prompts and checkpoints only for completion tasks * server : improve prompt caching logic * cont : fix check for number of cached prompts [no ci] * server : improve caching logic, add -cram CLI arg * server : print prompt mismatch info * cont : better naming [no ci] * server : improve prompt cache loading logic * server : add option to debug the slot contents (#16482) * server : add option to debug the slot contents * Update tools/server/server.cpp --------- Co-authored-by: Xuan-Son Nguyen <son@huggingface.co> * server : add option to disable prompt cache --------- Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>	2025-10-09 18:54:51 +03:00
Aleksander Grygier	a7a98e0fff	SvelteKit-based WebUI (#14839)	2025-09-17 19:29:13 +02:00
Xuan-Son Nguyen	3c3635d2f2	server : speed up tests (#15836) ... * server : speed up tests * clean up * restore timeout_seconds in some places * flake8 * explicit offline	2025-09-06 14:45:24 +02:00
Georgi Gerganov	d2fcd91cf9	server : disable context shift by default (#15416) ... * server : disable context shift by default ggml-ci * server : make scopr of test parameters local	2025-08-19 16:46:37 +03:00
Diego Devesa	1d36b3670b	llama : move end-user examples to tools directory (#13249) ... * llama : move end-user examples to tools directory --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>	2025-05-02 20:27:13 +02:00