Qwen3-8B Heretic v1.4 — INT8 ConvRot for ComfyUI Flux.2 Klein 9B

This repository contains a ComfyUI-compatible, learned-rounding INT8 ConvRot

conversion of

saidutta69/Qwen3-8B-heretic,

Heretic v1.4.0. It is intended as the Qwen3-8B text encoder for Flux.2 Klein

9B workflows.

Download

| Precision | File | Size | Notes |

| --- | --- | ---: | --- |

| INT8 ConvRot | qwen3-8b-heretic-v1.4_klein9b_int8_convrot.safetensors | 10.21 GB | Hosted here; recommended |

| BF16 | Upstream Heretic repository | 16.38 GB | Original four BF16 shards |

INT8 SHA-256:

0f7097a0c71ecc7739cdcc1955d5a9f158d5331dce5159743e46f26b4c6003b6

The conversion is pinned to upstream revision

a0440e8ed46a519c1b35e6add9ffae000cd9fb51.

Quantization

Built with

silveroxides/convert_to_quant

1.3.1:

• 224 learned-rounding INT8 weights
• Per-row FP32 dequantization scales
• True ConvRot metadata on all 224 quantized matrices
• Fixed ConvRot group size 256
• 175 original weights remain BF16
• The embedding, LM head, and transformer blocks 0, 8, 17, and 26 remain BF16
• Deterministic seed 42
• 847 tensors in the final file

The BF16 blocks were retained to protect the first transformer block and the

hidden-state boundaries used by Flux.2 Klein conditioning.

Conversion command:

ctq \
  -i qwen3-8b-heretic-v1.4_bf16.safetensors \
  -o qwen3-8b-heretic-v1.4_klein9b_int8_convrot.safetensors \
  --int8 \
  --scaling_mode row \
  --convrot \
  --convrot-group-size 256 \
  --comfy_quant \
  --save-quant-metadata \
  --low-memory \
  --device cuda \
  --exclude-layers \
  '(^model\.embed_tokens\.weight$|^lm_head\.weight$|^model\.layers\.(0|8|17|26)\.)' \
  --verbose NORMAL \
  --manual-seed 42

Validation

The safetensors structure was checked against the BF16 source:

• 399 original weight tensors and shapes preserved
• 224 INT8 weights, 224 FP32 scales, and 224 ComfyUI quant descriptors
• 224 matching entries in _quantization_metadata
• No missing quantization companions or shape mismatches
• Exactly 30 two-dimensional weights retained in BF16

ComfyUI detected both the BF16 source and this conversion as QWEN3_8B. GPU

forward tests covered all four Qwen projection shapes:

| Projection | Output shape | Cosine similarity versus BF16 | Relative L2 error |

| --- | ---: | ---: | ---: |

| Attention V | (2, 1024) | 0.9998978 | 1.43038% |

| Attention Q | (2, 4096) | 0.9999223 | 1.24889% |

| MLP up | (2, 12288) | 0.9999169 | 1.28918% |

| MLP down | (2, 4096) | 0.9999068 | 1.36529% |

Every test loaded TensorWiseINT8Layout with convrot=true, group size 256,

and produced finite BF16 outputs.

ComfyUI installation

Place the safetensors file in:

ComfyUI/models/text_encoders/

Select it as the Qwen3-8B text encoder in a Flux.2 Klein 9B workflow.

This file uses ComfyUI mixed quantized operations. The tested/current ComfyUI

runtime pins comfy-kitchen==0.2.22; older releases such as 0.2.10 cannot load

this ConvRot layout. PyTorch builds older than CUDA 13 can use the compatible

fallback path, while CUDA 13 or newer enables the optimized CUDA operations.

This is a single-file ComfyUI text encoder, not a complete Transformers

repository.

Source and credits

• Abliterated BF16 source:

saidutta69/Qwen3-8B-heretic

• Original model:

Qwen/Qwen3-8B

• Quantizer:

silveroxides/convert_to_quant