Qwen3-8B Heretic v1.4 — INT8 ConvRot for ComfyUI Flux.2 Klein 9B

This repository contains a ComfyUI-compatible, learned-rounding INT8 ConvRot conversion of saidutta69/Qwen3-8B-heretic, Heretic v1.4.0. It is intended as the Qwen3-8B text encoder for Flux.2 Klein 9B workflows.

Download

INT8 SHA-256:

0f7097a0c71ecc7739cdcc1955d5a9f158d5331dce5159743e46f26b4c6003b6

The conversion is pinned to upstream revision a0440e8ed46a519c1b35e6add9ffae000cd9fb51.

Quantization

Built with silveroxides/convert_to_quant 1.3.1:

• 224 learned-rounding INT8 weights
• Per-row FP32 dequantization scales
• True ConvRot metadata on all 224 quantized matrices
• Fixed ConvRot group size 256
• 175 original weights remain BF16
• The embedding, LM head, and transformer blocks 0, 8, 17, and 26 remain BF16
• Deterministic seed 42
• 847 tensors in the final file

The BF16 blocks were retained to protect the first transformer block and the hidden-state boundaries used by Flux.2 Klein conditioning.

Conversion command:

ctq \
  -i qwen3-8b-heretic-v1.4_bf16.safetensors \
  -o qwen3-8b-heretic-v1.4_klein9b_int8_convrot.safetensors \
  --int8 \
  --scaling_mode row \
  --convrot \
  --convrot-group-size 256 \
  --comfy_quant \
  --save-quant-metadata \
  --low-memory \
  --device cuda \
  --exclude-layers \
  '(^model\.embed_tokens\.weight$|^lm_head\.weight$|^model\.layers\.(0|8|17|26)\.)' \
  --verbose NORMAL \
  --manual-seed 42

Validation

The safetensors structure was checked against the BF16 source:

• 399 original weight tensors and shapes preserved
• 224 INT8 weights, 224 FP32 scales, and 224 ComfyUI quant descriptors
• 224 matching entries in _quantization_metadata
• No missing quantization companions or shape mismatches
• Exactly 30 two-dimensional weights retained in BF16

ComfyUI detected both the BF16 source and this conversion as QWEN3_8B. GPU forward tests covered all four Qwen projection shapes:

Every test loaded TensorWiseINT8Layout with convrot=true, group size 256, and produced finite BF16 outputs.

ComfyUI installation

Place the safetensors file in:

ComfyUI/models/text_encoders/

Select it as the Qwen3-8B text encoder in a Flux.2 Klein 9B workflow.

This file uses ComfyUI mixed quantized operations. The tested/current ComfyUI runtime pins comfy-kitchen==0.2.22; older releases such as 0.2.10 cannot load this ConvRot layout. PyTorch builds older than CUDA 13 can use the compatible fallback path, while CUDA 13 or newer enables the optimized CUDA operations.

This is a single-file ComfyUI text encoder, not a complete Transformers repository.

Source and credits

• Abliterated BF16 source: saidutta69/Qwen3-8B-heretic
• Original model: Qwen/Qwen3-8B
• Quantizer: silveroxides/convert_to_quant