Matteo Boschini 1873ff586b metal : add gqa8 kernel to allow llama-2-70B on metal (#2459) ...

* Added gqa8 kernel to allow llama-2-70B on metal

* Update ggml-metal.m

Co-authored-by: Cebtenzzre <cebtenzzre@gmail.com>

* Extend kernel_mul_mat_f16_f32 to handle gqa broadcast

* Added ne03==ne13 assertion

---------

Co-authored-by: Cebtenzzre <cebtenzzre@gmail.com>

2023-08-01 10:43:12 +03:00

57 KiB

Raw Blame History

View Raw