Kawrakow 469e75d0a3 llama : restore intended k-quants mixes for MoE models (#4872) ...

* Restore intended k-quants quantization mixes for MoE models

* Update Q2_K_S values in the quantize tool

Still using LLaMA-v1 PPL values in the quant description
today does not make much sense. But let's leave this update
for another PR.

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

2024-01-11 21:43:15 +02:00

39 KiB

Raw Blame History

View Raw