CS348Project/llama.cpp
5
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2025-10-30 08:42:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
5ef588ba58c449d1fa22cddcdecb7fe7da2600b5
llama.cpp/tests/test-backend-ops.cpp
Jeff Bolz f01bd02376 vulkan: Implement split_k for coopmat2 flash attention. (#12627) 
When using group query attention, we have one workgroup per KV batch and this
can be very few workgroups (e.g. just 8 in some models). Enable split_k to
spread the work across SMs. This helps a lot when the KV cache is large.
2025-04-02 14:25:08 -05:00

173 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink