CS348Project/llama.cpp
5
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-11-01 09:01:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

kv-cache : rework kv_cell (#13706)

Browse Source 
* kv-cache : rework kv_cell

ggml-ci

* kv-cells : use "shift" instead of "delta" consistently

ggml-ci

* llama : add llama_max_parallel_sequences()

ggml-ci

* kv-cells : update comments [no ci]

* context : fail upon construction if sequences exceed max value

ggml-ci

* kv-cells : get_pos() -> pos_get() + comments

ggml-ci

* kv-cells : fix tracking of "used" cells

ggml-ci
This commit is contained in:
Georgi Gerganov
2025-05-25 16:34:36 +03:00
committed by GitHub
parent c508256db2
commit de2ef53a4b
8 changed files with 470 additions and 253 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/llama-cparams.h
View File

@@ -4,6 +4,8 @@
#include <cstdint>
#define LLAMA_MAX_PARALLEL_SEQUENCES 64
struct llama_cparams {
uint32_t n_ctx; // context size used during inference
uint32_t n_batch;