CS348Project/llama.cpp
5
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-10-27 08:21:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

kv-cells : track min/max used cells and per-sequence positions (#13808)

Browse Source 
* kv-cells : track min/max used cells and per-sequence positions

ggml-ci

* kv-cells : fix pos-modification updates for seq_pos

ggml-ci

* kv-cells : add comments

ggml-ci
This commit is contained in:
Georgi Gerganov
2025-05-27 13:49:41 +03:00
committed by GitHub
parent f9cd68398b
commit 81713121ee
3 changed files with 124 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
src/llama-kv-cache.h
View File

@@ -246,10 +246,6 @@ private:
// return true if cells have been moved
bool defrag_prepare(int32_t n_max_nodes);
// find how many cells are currently in use
// TODO: optimize
uint32_t cell_max() const;
size_t total_size() const;
size_t size_k_bytes() const;