language: en license: other license_name: see-base-models tags:
You can also use this node in ComfyUI to convert any BF16 diffuser or text encoder! @Aksu on Civitai: https://civitai.red/user/Addictedd/models @Starnodes on Github: https://github.com/Starnodes2024
Single-file ComfyUI text encoders quantized to 3-bit (w3a8) and 5-bit (w5a8) weights using the ConvRot layout from comfy-kitchen.
These are experimental. They are smaller than the published w4a8 and w6a8 files, and the 3-bit ones have not been put through a formal perplexity evaluation yet.
| Folder | File | Format |
|---|---|---|
qwen3.8-27b-abliterated/ |
huihui-qwen38-27b-abl-raw-w3a8_convrot.safetensors |
w3a8 ConvRot |
gemma4/ |
gemma4_12b-w3a8_convrot.safetensors |
w3a8 ConvRot |
Check the file sizes on the Files tab. The Qwen3.8-27B file stages at roughly 16 GB in ComfyUI.
Both formats use comfy-kitchen's AsymW4A8Int8Layout at a different bit width. The width is implied by the packed row size, so the checkpoint needs no extra metadata.
| Format | Weights | Packed row width | Codebook |
|---|---|---|---|
| w3a8 | 3-bit | 3K/8 bytes | 8-entry Lloyd-Max codebook |
| w5a8 | 5-bit | 5K/8 bytes | none (uniform levels) |
Comfy-kitchen can quantize at 3 and 5 bits, but ComfyUI core did not register these formats on my install. Loading fails with:
KeyError: 'w3a8_int8'
The loader already infers bit width from the packed weight shape, so the fix is registering the format names. Back up both files first.
comfy/ops.py
if s.quant_format not in ("int8_tensorwise", "asym_w4a8_int8", "w3a8_int8", "w5a8_int8", "w6a8_int8"):
_GROUPED_INT8_FORMATS = {"asym_w4a8_int8": 4, "w3a8_int8": 3, "w5a8_int8": 5, "w6a8_int8": 6}
comfy/quant_ops.py
Register the formats next to w6a8_int8:
QUANT_ALGOS["w3a8_int8"] = QUANT_ALGOS["w5a8_int8"] = QUANT_ALGOS["w6a8_int8"] = {
"storage_t": torch.int8,
"parameters": {"weight_scale"},
"comfy_tensor_layout": "AsymW4A8Int8Layout",
"quantize_input": False,
}
Restart ComfyUI afterwards. Line numbers will differ between ComfyUI versions, so search for w6a8_int8 and add the new names alongside it. If upstream adds these formats, restore your backups and update instead. A git pull may conflict with local edits.
ComfyUI/models/text_encoders/.CLIPLoader) node and pick the type your workflow uses.Requires a comfy-kitchen build whose AsymW4A8Int8Layout.quantize accepts a bits argument.
Informal only. I ran a captioning task on a night photo and compared outputs by eye.
No perplexity numbers yet. If you run one, I would like to hear the result.
AsymW4A8Int8Layout at bits=3 and bits=5These files are derivative works. Each follows the license of its base model (the Qwen3.8 abliterated base is listed as Apache-2.0; check the Gemma terms for the Gemma file). Review the base model pages before reusing or redistributing.
qwen3.8-27b-abliterated/huihui-qwen38-27b-abl-raw-w3a8_convrot.safetensors 17 GBMinimax-H3_Text_Encoder/qwen3vl_32b_minimax_h3-w3a8_convrot.safetensors 13 GBQwen3.5_9b/qwen3.5_9b_flux3_prompt_upsampler-w5a8_convrot.safetensors 9.7 GBQwen3.5_9b/qwen3.5_9b_flux3_prompt_upsampler-w3a8_convrot.safetensors 8.0 GBGemma4/gemma4_12b-w3a8_convrot.safetensors 6.9 GB