Qwen3.8-27B IQR-V3 — radiance containers, every size in one repository
Qwen3.8-27B quantised with IQR-V3, radiance's trellis format: every trunk linear rotated by a
128-point Hadamard and fitted by GPTQ against calibration Hessians, its codes a tail-biting trellis
down each column whose 2^16 states' values are computed from the state, under one f32 scale a row.
The GEMM decodes the codes in its tile; nothing is widened in memory. Each file is a single .rad
container for the radiance inference engine (AMD RDNA4, ROCm), with the vision tower and the
z-lab/Qwen3.8-27B-DFlash2 block-diffusion drafter
merged in, as in our other production 27B containers.
Needs the radiance release after 1.14.0 (1.15.0 or newer): the IQR-V3 kernels and the qwen35_w8a8
plugin that reads them ship there; 1.14.0 and older refuse these files.
| Weights | IQR-V3 at 2.5 to 8 bits a weight, a mix of 2- to 7-bit trellises and the int8 grid by layer band; int8 activations under the same rotation |
| Speculator | DFlash2, its linears OCP-MXFP4, its vocabulary head as 2-bit codes whose top 32 candidates the engine rescores against the model's own head. Drafts are verified, so they change the speed and never the output |
| Vision | the 27-block vision tower, bf16: images and video in chat requests |
| Context | 262,144 tokens trained; 200K tested |
The files
Scores on the radiance quantisation challenge's private split (1.05M model-written positions, KL against the bf16 model's distribution); the board is at radiance.stilldeadco.de/leaderboard. bpw counts the language model the board counts (the drafter and the vision tower are on top).
| file | bpw | KL mean | top-1 | IQR-V2 at that size | KL |
|---|---|---|---|---|---|
qwen3.8-27b-iq2r-v3.rad |
2.465 | 0.06807 | 90.54% | 0.07303 | -7% |
qwen3.8-27b-iq2r-l-v3.rad |
2.786 | 0.04241 | 92.62% | 0.04698 | -10% |
qwen3.8-27b-iq3r-s-v3.rad |
3.145 | 0.02584 | 94.02% | 0.02798 | -8% |
qwen3.8-27b-iq3r-v3.rad |
3.459 | 0.01698 | 95.35% | 0.01880 | -10% |
qwen3.8-27b-iq3r-l-v3.rad |
3.798 | 0.01129 | 96.16% | 0.01198 | -6% |
qwen3.8-27b-iq4r-s-v3.rad |
4.131 | 0.00762 | 96.90% | 0.00832 | -8% |
qwen3.8-27b-iq4r-v3.rad |
4.458 | 0.00512 | 97.45% | 0.00562 | -9% |
qwen3.8-27b-iq4r-l-v3.rad |
4.783 | 0.00342 | 97.86% | 0.00377 | -9% |
qwen3.8-27b-iq5r-s-v3.rad |
5.064 | 0.00249 | 98.15% | 0.00273 | -9% |
qwen3.8-27b-iq5r-v3.rad |
5.422 | 0.00182 | 98.38% | 0.00204 | -11% |
qwen3.8-27b-iq6r-s-v3.rad |
6.049 | 0.00106 | 98.83% | 0.00115 | -8% |
qwen3.8-27b-iq6r-v3.rad |
6.372 | 0.00088 | 98.93% | 0.00094 | -6% |
qwen3.8-27b-iq6r-l-v3.rad |
6.939 | 0.00072 | 99.04% | 0.00073 | -2% |
qwen3.8-27b-iq7r-v3.rad |
7.420 | 0.00060 | 99.12% | 0.00061 | -2% |
qwen3.8-27b-iq8r-v3.rad |
8.040 | 0.00047 | 99.18% | 0.00047 | 0% |
IQ4R-L-V3 is below the official FP8 checkpoint's KL (0.00389) at 55% of its size. The design, the per-weight and GEMM measurements and the comparison with IQR-V1 and V2 are in the IQR-V3 technical report.
Serve
radiance --model qwen3.8-27b-iq4r-v3.rad --tp 2 --max-model-len 200000 --kv-cache-dtype fp8 \
--max-num-seqs 8 --host 0.0.0.0 --port 8000
The smaller files fit one 32 GB card (--tp 1). The server speaks the OpenAI API
(/v1/chat/completions, /v1/completions), with tool calls and structured output, and image_url /
video_url parts in chat messages. The drafter's depth is chosen automatically
(--num-speculative-tokens N states one, 0 turns speculation off).
How these files were made
Each entry is a mix of per-width bases chosen at the size of its IQR-V2 twin; the recipe that converts
it is in recipes/ (the radiance repository carries the same mixes without the drafter as
data/recipes/q38-27b-*-v3.recipe). The embedding's trellis tables are in data/codebooks/ here and
in the radiance repository; the recipes name them by that relative path, so run rad-convert from
the directory that holds data/. CALIB is the directory a radiance --calib-corpus ... --calib-dir
run wrote. Serving needs neither: each container carries its tables.
CALIB=<calibration Hessians> rad-convert Qwen/Qwen3.8-27B --draft-model z-lab/Qwen3.8-27B-DFlash2 \
--recipe recipes/q38-27b-iq4r-v3-df2.recipe \
--set quantization_config.quant_method=w8a8 --set quantization_config.fmt=fwht128 -o out.rad
Every file's language model scores exactly as the container the board scored: its tier-1 KL rows are checked byte for byte before upload.