Qwen3-VL-8B-Instruct — ROCmFP4 / ROCmFPX GGUF

AMD-native FP4 / FP8 GGUF builds of Qwen/Qwen3-VL-8B-Instruct for RDNA3.5 / Strix Halo

(gfx1151). A vision-language model — the bundled mmproj-BF16.gguf is the point of the build.

Variants

|---|---|---|---|---|

| Qwen3-VL-8B-Instruct-Q4_0_ROCMFP4_COHERENT.gguf | 102 | 4.60 GiB | 44.86 t/s | 1.0013 |

| Qwen3-VL-8B-Instruct-Q6_0_ROCMFPX_AGENT.gguf | 114 | 7.22 GiB | 28.50 t/s | 1.0004 |

| Qwen3-VL-8B-Instruct-Q8_0_ROCMFPX.gguf | 111 | 7.91 GiB | 26.29 t/s | 1.0015 |

| Qwen3-VL-8B-Instruct-Q8_0_ROCMFPX_AGENT.gguf | 115 | 8.02 GiB | 26.08 t/s | 1.0012 |

Measured on an idle Ryzen AI MAX+ 395 (Strix Halo, gfx1151, ROCm 7.2.4):

-ngl 999 -c 4096 -fa on -fit off -np 1, 300-token generations, 12 samples with

two warm-ups on the same prompt as the measurement. Spread = slowest/fastest.

⚠️ An earlier pass of these same files, taken while other jobs shared the GPU, read

20% low with 20%+ spread. On this hardware a co-resident job is the single largest

source of benchmark error — measure on an idle box or say what else was resident.

Vision verified 4/4 on a four-quadrant colour image (red / blue / yellow / green) with

the bundled mmproj-BF16.gguf. ⛔ Vision needs -fa off.

Every artifact was loaded on real hardware and checked for: exact stat bytes vs the

--dry-run projection (a constant header delta; a varying one means truncation), the

actual token_embd / output.weight types, three correctness answers asserted against

content + reasoning with finish_reason recorded, and a decode median.

FP4/FP8 tensor types from the ROCmFPX fork of llama.cpp. These types do not exist in

mainline llama.cpp — a ROCmFPX-capable build is required to load them.