A GPTQ-calibrated W4A8 ConvRot quantization of the Qwen3-VL-8B text encoder from Comfy-Org/Qwen-Image-2.1 (qwen3vl_8b_bf16.safetensors), for ComfyUI via comfy-kitchen.
It is a drop-in for Comfy-Org's qwen3vl_8b_w4a8.safetensors: same tensor names, dtypes, shapes and per-layer comfy_quant config, same file size (6.31 GB), same speed. Put it in models/text_encoders.
| Model | File size | KL vs bf16 | Top-1 agree | Hidden cosine | Hidden error |
|---|---|---|---|---|---|
| This repo (GPTQ W4A8) | 6.31 GB | 0.0310 | 95.0% | 0.9957 | 8.0% |
| Comfy-Org INT8 ConvRot | 9.35 GB | 0.0225 | 95.2% | 0.9978 | 5.7% |
| Comfy-Org W4A8 | 6.31 GB | 0.4168 | 78.8% | 0.9482 | 30.2% |
At the same size as Comfy-Org's W4A8, KL is 13.4x lower. It is within about 1.4x of their INT8 file on KL at two thirds of the size.
How it was made
- GPTQ in the ConvRot (group Hadamard, 256) basis on all 252 attention and MLP linears, 4-bit weights with group size 16, Hessian-weighted error 0.15x that of plain round-to-nearest.
- Calibration: 476 prompts (Stable Diffusion style, long prompts, captions, some wiki text and code), wrapped in the Qwen-Image T2I template.
lm_headand the embeddings are int8 ConvRot and the vision tower stays bf16, as in the Comfy-Org file.
Measurement
KL, top-1 agreement and hidden-state error are against the bf16 encoder on 250 held-out prompts (SD-style, long, captions, wiki), using the real comfy_kitchen ops on the quantized linears. Hidden state is the last-layer state before the final norm, which is what Qwen-Image conditions on. KL and top-1 use the language-model logits, which Qwen-Image does not use.
Not measured: image-level quality. The numbers cover the 252 attention and MLP linears; lm_head and the embeddings were kept bf16 in all three runs.