Qwen3.8-27B pure IQ4_XS: +12% speed, a measured quality step below UD

The speed pick for lucebox serving of Qwen3.8-27B on bandwidth-bound GPUs (measured on one AMD Radeon AI PRO R9700). It is bartowski's imatrix IQ4_XS requantized with llama-quantize --allow-requantize --pure, which drops the mixed-precision quality hedge so every tensor is IQ4_XS: 13.54 GiB instead of 14.50, and decode reads fewer bytes per token through the fastest kernel path on RDNA4.

Know the trade before you download. Measured against a Q8_0 reference (300x512-chunk wikitext-2 paired KL divergence; HumanEval-164 and GSM8K-200 through the serving path, exact-verify speculation so the drafter cannot change outputs):

Pro: about 12% more tokens per second on every request than the UD file, and the fastest Qwen3.8-27B target we have measured on this class of hardware.

Con: the quality cost is small but real and shows up on every instrument: KL divergence rises by half, top-1 agreement gives up two points, and it hands back seven HumanEval and six GSM8K problems against the UD file. If your workload is quality-sensitive, serve UD-IQ4_XS instead; it is what the lucebox quickstart recommends.

Full methodology, exactness verification, and the llama.cpp comparison: the blog post.

Serve it

./dflash_server Qwen3.8-27B-IQ4_XS-pure.gguf \
  --draft qwen38-dflash2-q8_0.gguf \
  --draft-block-size 16 --max-ctx 131072 \
  --cache-type-k q8_0 --cache-type-v q8_0

The drafter is z-lab's DFlash2 converted with server/scripts/convert_dflash_to_gguf.py from the lucebox repo. Reproduce this file yourself from bartowski's IQ4_XS with:

llama-quantize --allow-requantize --pure \
  Qwen3.8-27B-IQ4_XS.gguf Qwen3.8-27B-IQ4_XS-pure.gguf IQ4_XS

Vision: images in, with Qwen3.8-27B-mmproj-Q8_0.gguf

This repo also carries the image encoder (projector) for lucebox image input. It is unsloth's BF16 projector with its weight matrices recast to Q8_0 (rows that are not a multiple of 32 stay F16): 629 MB instead of 931 MB, and it encodes a 975-token image in 443 ms instead of 677 ms on the R9700, with the same scores.

luce_server Qwen3.8-27B-IQ4_XS-pure.gguf --target-device hip:0 \
  --draft qwen38-dflash2-q8_0.gguf --draft-device hip:0 \
  --mmproj Qwen3.8-27B-mmproj-Q8_0.gguf

Images go in OpenAI-style image_url parts of a chat message, up to four per request. Each image costs one token per 32x32 pixels, 64 to 1,024 tokens. Needs a lucebox build with image input (PR #722); the full guide is docs/image-input.md in the repo.

Measured on one R9700 with this file, the Q8_0 projector and the DFlash2 drafter, thinking off:

Text requests are byte-identical with the projector loaded, at the same speed. Image requests decode without the drafter.

sha256 47b8f0575185fcf09a6344fa46288f3506b4134cab77fa09ebb5a88c38d665e5

Credit: unsloth for the projector; bartowski for the imatrix calibration this file inherits; Qwen for the model.