Qwen3.8-27B IQR-V4: radiance containers, every size in one repository
Qwen3.8-27B quantised with IQR-V4, radiance's trellis format. Every trunk linear is rotated by a 128-point Hadamard and fitted by GPTQ, in act order, against calibration Hessians. Its codes are a tail-biting trellis down each column whose 2^16 states' values are computed from the state, under one f32 scale a row. The projections fed by the attention heads also get a bf16 scale per 128 weights. The widths come from measured KL prices. Below IQ6R-L the lm_head is vocabulary-weighted: four trellis widths, with the tokens that carry the probability on the widest. The GEMMs decode the codes in their tiles; nothing is widened in memory.
Each file is a single .rad container for the radiance inference engine (AMD RDNA4, ROCm). The
vision tower and the z-lab/Qwen3.8-27B-DFlash2
block-diffusion drafter are merged in.
Needs the radiance release after 1.16.0: the IQR-V4 kernels (the vocabulary-weighted head, the faster GEMV and 16-token tile) ship there. 1.16.0 and older refuse these files.
| Weights | IQR-V4 at 2.5 to 8 bits a weight, a mix of 2- to 7-bit trellises and the int8 grid by layer band and projection; int8 activations under the same rotation |
| Speculator | DFlash2. Its linears are OCP-MXFP4; its vocabulary head is 2-bit codes, and the engine rescores their top 32 candidates against the model's own head. Drafts are verified, so they change the speed, never the output |
| Vision | the 27-block vision tower, bf16: images and video in chat requests |
| Context | 262,144 tokens trained; 200K tested |
The files
Scores are from the radiance quantisation challenge's private split: 1.05M model-written positions, KL against the bf16 model's distribution. The board is at radiance.stilldeadco.de/leaderboard. bpw counts the language model, as the board does; the drafter and the vision tower come on top.
| file | bpw | KL mean | top-1 | IQR-V3 at that size | KL |
|---|---|---|---|---|---|
qwen3.8-27b-iq2r-v4.rad |
2.470 | 0.06113 | 91.23% | 0.06807 | −10% |
qwen3.8-27b-iq2r-l-v4.rad |
2.797 | 0.03602 | 93.29% | 0.04241 | −15% |
qwen3.8-27b-iq3r-s-v4.rad |
3.157 | 0.02241 | 94.75% | 0.02584 | −13% |
qwen3.8-27b-iq3r-v4.rad |
3.472 | 0.01495 | 95.74% | 0.01698 | −12% |
qwen3.8-27b-iq3r-l-v4.rad |
3.804 | 0.00917 | 96.61% | 0.01129 | −19% |
qwen3.8-27b-iq4r-s-v4.rad |
4.142 | 0.00643 | 97.12% | 0.00762 | −16% |
qwen3.8-27b-iq4r-v4.rad |
4.473 | 0.00444 | 97.69% | 0.00512 | −13% |
qwen3.8-27b-iq4r-l-v4.rad |
4.793 | 0.00297 | 98.07% | 0.00342 | −13% |
qwen3.8-27b-iq5r-s-v4.rad |
5.071 | 0.00223 | 98.33% | 0.00249 | −10% |
qwen3.8-27b-iq5r-v4.rad |
5.433 | 0.00154 | 98.60% | 0.00182 | −15% |
qwen3.8-27b-iq6r-s-v4.rad |
6.063 | 0.00100 | 98.86% | 0.00106 | −6% |
qwen3.8-27b-iq6r-v4.rad |
6.439 | 0.00081 | 98.96% | 0.00088 | −8% |
qwen3.8-27b-iq6r-l-v4.rad |
6.939 | 0.00068 | 99.08% | 0.00072 | −6% |
qwen3.8-27b-iq7r-v4.rad |
7.420 | 0.00059 | 99.15% | 0.00060 | −2% |
qwen3.8-27b-iq8r-v4.rad |
8.040 | 0.00046 | 99.22% | 0.00047 | −2% |
On one card, IQ4R-S-V4 decodes a single stream 4% faster than the MXFP4 checkpoint, at five times lower KL, and IQ3R-S-V4 is faster at every concurrency. The design, the kernels and the measurements are in the IQR-V4 technical report.
Serve
radiance --model qwen3.8-27b-iq4r-v4.rad --tp 1 --max-model-len 200000 --kv-cache-dtype fp8 \
--max-num-seqs 8 --host 0.0.0.0 --port 8000
Files up to IQ5R fit one 32 GB card with room for long context; use --tp 2 for the larger ones. The
server speaks the OpenAI API (/v1/chat/completions, /v1/completions), with tool calls and structured
output, and image_url / video_url parts in chat messages. The drafter's depth is chosen
automatically (--num-speculative-tokens N sets it, 0 turns speculation off).
How these files were made
The recipe that converts each file is in recipes/. The radiance repository carries the same mixes
without the drafter as data/recipes/q38-27b-*-v4.recipe.
For the entries with the vocabulary-weighted head, first split the head into its width classes with
tools/vhead.py and the class file the recipe names. The 4.0, 4.5 and 5.0 b class files are in
data/vhead/, here and in the radiance repository.
The embedding's trellis tables are in data/codebooks/. The recipes name them by that relative path,
so run rad-convert from the directory that holds data/. CALIB is the directory a
radiance --calib-corpus ... --calib-dir run wrote. Serving needs none of this: each container carries
its tables.
python3 tools/vhead.py Qwen/Qwen3.8-27B q38-27b-vhead-4.5 data/vhead/q38-27b-vhead-4.5.cls
CALIB=<calibration Hessians> rad-convert q38-27b-vhead-4.5 --draft-model z-lab/Qwen3.8-27B-DFlash2 \
--recipe recipes/q38-27b-iq4r-v4-df2.recipe \
--set quantization_config.quant_method=w8a8 --set quantization_config.fmt=fwht128 -o out.rad
Every file is the container the board scored: its SHA-256 is checked against the scored file before upload.