Hushword referee: Qwen2.5-1.5B-Instruct, IQ4_XS (task imatrix)
An IQ4_XS GGUF of Qwen/Qwen2.5-1.5B-Instruct, quantized with an importance matrix built from the yes/no referee prompts of the Hushword word game. The game runs it on-device (llama.cpp) to rule on whether a word fits a clue question, scoring P(Yes) / (P(Yes) + P(No)) and accepting at 0.5.
Files
| File | Bytes | SHA-256 |
|---|---|---|
qwen2.5-1.5b-instruct-iq4_xs-imatrix.gguf |
895731808 | 8c824c0ccfcd3f9946bc32e9b42cc3c007bd3e1d8a0d8e0e44866efd0b1f8399 |
qwen2.5-1.5b-instruct-hushword.imatrix.gguf |
4137728 | 1a092c220392c586de988f4420c7c6d2eede9caa95fbcce9a848491be632a6d8 |
How it was made
- llama.cpp b12014 (commit
aa588a3463357ee990b314f41dda242869130704). convert_hf_to_gguf.pyfrom the original HF safetensors to f16, thenllama-quantize --imatrix ... IQ4_XS. Not requantized from an existing quant.- Importance matrix:
llama-imatrixover 1200 rendered referee prompts (about 620 KB of text) from the training split of the game's judge evaluation set only. The matrix is included (llama.cpp's GGUF imatrix format).
Evaluation
The game's own judge, same prompt, threshold 0.5 (not tuned), on its 259-case held-out set: share of fitting answers, typo'd fitting answers and non-fitting answers accepted, plus the mean score gap between fitting and non-fitting answers.
| Quant | MB | Fits | Typos | Misses | Separation |
|---|---|---|---|---|---|
| Q4_K_M (official) | 1117 | 97% | 78% | 13% | 0.955 |
| IQ4_XS, task imatrix (this) | 896 | 95% | 78% | 13% | 0.958 |
It also passes the game's startup sanity check, and a probe of misspelled wrong answers found no added tendency to accept them (-1pp, against -2pp for Q4_K_M).
Equivalent quality at 20% smaller size. It is slower per judgment: about 1.29x Q4_K_M on 4 CPU threads and 1.21x on Vulkan. It has not been tested on WebGPU.
License
Apache 2.0, as for the base model; see LICENSE.