8 Commits

Author	SHA1	Message	Date
Jeff Bolz	94e82c7ead	vulkan: clamp matmul and FA results to the max finite value (#15652) ... * vulkan: clamp matmul and FA results to the max finite value * only clamp for fp16	2025-08-31 08:27:57 +02:00
Jeff Bolz	c4f53563df	vulkan: support fattn sinks (#15126)	2025-08-07 22:44:20 +02:00
Jeff Bolz	a0374a67e2	vulkan: Handle updated FA dim2/3 definition (#14518) ... * vulkan: Handle updated FA dim2/3 definition Pack mask boolean and n_head_log2 into a single dword to keep the push constant block under the 128B limit. * handle null mask for gqa * allow gqa with dim3>1	2025-07-05 09:26:04 +02:00
Jeff Bolz	2b72bedec1	vulkan: support mixed/deepseekR1 FA head sizes (#14509) ... * vulkan: better parameterize FA by head sizes * vulkan: support mixed/deepseekR1 FA head sizes	2025-07-03 20:21:14 +02:00
Jeff Bolz	8875523eb3	vulkan: support softmax/FA batch and broadcast (#14449)	2025-07-02 15:48:33 +03:00
Jeff Bolz	2f5a4e1e09	vulkan: move common FA code to flash_attn_base.comp (#13556) ... * vulkan: move common FA code to flash_attn_base.comp * vulkan: move common FA index/stride setup code to flash_attn_base.comp * build fix	2025-05-17 09:14:55 +02:00
Jeff Bolz	ab3971f2a0	vulkan: workaround FA compile failures on macos (#13517)	2025-05-14 06:15:50 +02:00
Jeff Bolz	dc1d2adfc0	vulkan: scalar flash attention implementation (#13324) ... * vulkan: scalar flash attention implementation * vulkan: always use fp32 for scalar flash attention * vulkan: use vector loads in scalar flash attention shader * vulkan: remove PV matrix, helps with register usage * vulkan: reduce register usage in scalar FA, but perf may be slightly worse * vulkan: load each Q value once. optimize O reduction. more tuning * vulkan: support q4_0/q8_0 KV in scalar FA * CI: increase timeout to accommodate newly-supported tests * vulkan: for scalar FA, select between 1 and 8 rows * vulkan: avoid using Float16 capability in scalar FA	2025-05-10 08:07:07 +02:00