MiMo-V2.6-Distill-Qwen-9B — ROCmFP4 imatrix for AMD Strix Halo (gfx1151)

Importance-matrix builds of the two ROCmFP4 tiers from MiMo-V2.6-Distill-Qwen-9B-ROCmFP4-GGUF. Use that repo for the standard builds and the vision mmproj.

⚠ Read this before downloading: imatrix did not help this model

On the only metric measured, these files are worse than the plain builds they came from.

This was measured twice, at two calibration scales, and more calibration made it worse:

The published files are v2 — the build with the more defensible methodology, not the one with the flattering number. Shipping v1 because it scored better would be picking the result.

What this is not

It is not a broken imatrix path. The ROCmFP4 quantizers take the importance data and branch on it:

if (!imatrix) { rocmfp4_quantize_row_q4_0_ref(...); }        // unweighted
else          { rocmfp4_quantize_row_q4_0_weighted(..., imatrix); }

The weighted path runs. It simply does not pay off here on prose perplexity.

What was not measured

Only wikitext-2 prose perplexity. The v2 calibration mixed code and chat with prose, and an importance matrix weighted toward code may well help code and tool-use work while costing a little wiki PPL — this is an agentic/tool-use distill, so that is a live possibility. It was not measured, and no number is published for it. If your workload is code, benchmark both and trust your own data over this card.

Which file should I use?

For general use, take the plain builds in the other repo. These exist so the imatrix variant is reproducible and available to anyone who wants to evaluate it on their own workload.

Quick start

llama-server -m MiMo-V2.6-Distill-Qwen-9B-Q4_0-ROCmFP4-STRIX_LEAN-imatrix.gguf \
  --mmproj mmproj-MiMo-V2.6-Distill-Qwen-9B-F16.gguf \
  -dev ROCm0 -ngl 999 -fa on -dio --jinja -c 32768

The mmproj lives in the other repo. Requires a llama.cpp built with ROCmFP4 support (ggml types 100–106); stock llama.cpp rejects these tensor types.

Speed is unchanged from the plain builds — imatrix alters weights, not layout or bit width. Decode measured 40.75 tok/s (LEAN-imatrix) and 43.45 (FAST-imatrix) against 40.97 and 43.46 plain, i.e. inside run-to-run noise on a 0.19 % drift control.

Reproduction

binary : https://github.com/charlie12345/ROCmFPX @ 85d8f7e83499e434c24993e6e1f3800568b5adfd
         build dir: build-hipvk (quantize/imatrix from build)   ROCm 7.2.4   gfx1151
         cmake -S . -B build-hipvk -DCMAKE_BUILD_TYPE=Release \
           -DGGML_HIP=ON -DGGML_VULKAN=ON -DGPU_TARGETS=gfx1151 \
           -DGGML_HIP_GRAPHS=ON -DGGML_HIP_NO_VMM=ON -DLLAMA_CURL=OFF
model  : XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
         revision 2367e865d009c13ac81713a2878291d33ab28177
box    : Ryzen AI Max+ 395, Radeon 8060S (gfx1151), 128 GiB unified, idle and dedicated
env    : HSA_OVERRIDE_GFX_VERSION=11.5.1
date   : 2026-09-24

# calibration corpus: 11,220,262 bytes = calibration_datav3.txt + wikitext-2 wiki.train.raw
llama-imatrix -m MiMo-V2.6-Distill-Qwen-9B-BF16.gguf -f calib-mixed.txt \
  -o MiMo-V2.6-Distill-Qwen-9B-v2.imatrix --output-format dat \
  -ngl 999 -fa on -c 4096 -b 2048 -ub 512 --chunks 200 -dev ROCm0

llama-quantize --imatrix …-v2.imatrix --output-tensor-type q6_K \
  BF16.gguf …-STRIX_LEAN-imatrix.gguf Q4_0_ROCMFP4_STRIX_LEAN 8
llama-quantize --imatrix …-v2.imatrix \
  BF16.gguf …-FAST-imatrix.gguf        Q4_0_ROCMFP4_FAST        8

Evaluation used a held-out corpus: wiki.test.raw, disjoint from the wiki.train.raw used for calibration — llama-perplexity -f wiki.test.raw -ngl 999 -fa on -c 4096 --chunks 40 -dev ROCm0.

Files

The .imatrix itself is included so the result can be reproduced or re-used without re-running calibration.

Known issues and limits

• imatrix measured worse than no imatrix on wikitext-2 PPL, at both calibration scales.
• Only prose perplexity was measured. No code, tool-use or vision benchmark is published.
• Perplexity used 40 chunks at ctx 4096 — enough to separate these builds, not enough for fine claims about BF16.
• This checkpoint ships no MTP head despite mtp_num_hidden_layers: 1 in its config, so no speculative-decoding mode helps. See the other repo's card for the full backend matrix.

License and attribution

MIT, inherited from XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B. Quantized and measured on AMD Strix Halo (gfx1151). All credit for the model itself to Xiaomi MiMo.