Kawrakow 147b17ac94 2-bit quantizations (#4897) ...

* imatrix: load

* imatrix: WIP

* imatrix: Add Q2_K quantization

* imatrix: also guard against Q2_K_S quantization without importance matrix

* imatrix: guard even more against low-bit quantization misuse

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2024-01-14 09:45:56 +02:00

9.7 KiB

Raw Blame History

View Raw