llama.cpp/examples/parallel/parallel.cpp at 806d397c1a13416c10e01908d95b04ce0e9fa8a4 - llama.cpp - Gitea - Peisong Xiao

CS348Project/llama.cpp

mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-11-03 09:22:01 +00:00

Files

Georgi Gerganov 806d397c1a parallel : try smaller batches when the KV cache is fragmented

2023-09-19 13:21:36 +03:00

11 KiB

Raw Blame History

View Raw