Qwen3.8-27B IQ8R (GPTQ int8, Hadamard-rotated) — radiance container
Qwen/Qwen3.8-27B with an int8 trunk fitted by GPTQ, each
128 of K rotated by a Hadamard transform, as a single .rad container for the radiance inference
engine (AMD RDNA4, ROCm). It is radiance's IQ8R entry in the Qwen3.8-27B quantization challenge.
| File | qwen3.8-27b-iq8r.rad — 27.95 GiB (text weights 28.59 GB, 8.50 bits per weight) |
| Weights | every trunk linear int8 (codes −127..127) with a bf16 scale per row and 128 of K, the 128 rotated by the unnormalised Hadamard; codes chosen by GPTQ against per-linear Hessians. The lm_head int8 by round to nearest |
| Activations | int8 under the same rotation (W8A8) |
| MTP head | Qwen's one-layer multi-token-prediction head, int8 by round to nearest |
| Vision | the 27-block vision tower, bf16: images and video in chat requests |
| Context | 262,144 tokens trained |
Quality
KL divergence from the bf16 model's next-token distribution, over the positions a model writes (assistant turns including tool calls, and plain text; not system or user turns or tool results), on scoring tier T1 of the challenge's private split (155,666 positions):
| mean KLD | KLD 99% | KLD 99.9% | top-1 agreement |
|---|---|---|---|
| 0.00055 | 0.0033 | 0.0232 | 99.18% |
Serve
radiance --model qwen3.8-27b-iq8r.rad --tp 2 --max-model-len 200000 --max-num-seqs 8 \
--host 0.0.0.0 --port 8000
The server speaks the OpenAI API (/v1/chat/completions, /v1/completions), with tool calls and
structured output, and image_url / video_url parts in chat messages.
How this file was made
The Hessians are the input Gram matrices of every trunk linear, recorded by radiance running the
bf16 model over the calib split of
StillDeadcode/qwen3.8-27b-quant-challenge
(the recording tap is not in a radiance release yet). CALIB names the directory that holds them:
CALIB=DIR rad-convert Qwen/Qwen3.8-27B --recipe q38-27b-iq8r.recipe \
--set quantization_config.quant_method=w8a8 --set quantization_config.fmt=fwht128 \
-o qwen3.8-27b-iq8r.rad
The recipe (q38-27b-iq8r.recipe in this repository); everything it does not name is the
checkpoint's own:
blk.*.attn_qg.weight gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.attn_k.weight gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.attn_v.weight gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.attn_output.weight gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.ssm_inz.weight gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.ssm_out.weight gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.ffn_gate_up.weight gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.ffn_down.weight gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
License
Apache 2.0, as Qwen/Qwen3.8-27B; its LICENSE is in this repository.