An IQ4_XS GGUF of Qwen/Qwen2.5-1.5B-Instruct, quantized with an importance matrix built from the yes/no referee prompts of the Hushword word game. The game runs it on-device (llama.cpp) to rule on whether a word fits a clue question, scoring P(Yes) / (P(Yes) + P(No)) and accepting at 0.5.
| File | Bytes | SHA-256 |
|---|---|---|
qwen2.5-1.5b-instruct-iq4_xs-imatrix.gguf |
895731808 | 8c824c0ccfcd3f9946bc32e9b42cc3c007bd3e1d8a0d8e0e44866efd0b1f8399 |
qwen2.5-1.5b-instruct-hushword.imatrix.gguf |
4137728 | 1a092c220392c586de988f4420c7c6d2eede9caa95fbcce9a848491be632a6d8 |
aa588a3463357ee990b314f41dda242869130704).convert_hf_to_gguf.py from the original HF safetensors to f16, then
llama-quantize --imatrix ... IQ4_XS. Not requantized from an existing quant.llama-imatrix over 1200 rendered referee prompts (about 620 KB of text)
from the training split of the game's judge evaluation set only. The matrix is included
(llama.cpp's GGUF imatrix format).The game's own judge, same prompt, threshold 0.5 (not tuned), on its 259-case held-out set: share of fitting answers, typo'd fitting answers and non-fitting answers accepted, plus the mean score gap between fitting and non-fitting answers.
| Quant | MB | Fits | Typos | Misses | Separation |
|---|---|---|---|---|---|
| Q4_K_M (official) | 1117 | 97% | 78% | 13% | 0.955 |
| IQ4_XS, task imatrix (this) | 896 | 95% | 78% | 13% | 0.958 |
It also passes the game's startup sanity check, and a probe of misspelled wrong answers found no added tendency to accept them (-1pp, against -2pp for Q4_K_M).
Equivalent quality at 20% smaller size. It is slower per judgment: about 1.29x Q4_K_M on 4 CPU threads and 1.21x on Vulkan. It has not been tested on WebGPU.
Apache 2.0, as for the base model; see LICENSE.
Qwen/Qwen2.5-1.5B-Instruct