Qwen3.5-9B-heretic-ara-v1 — NVFP4 + ConvRot INT8

Mixed-precision quantization of rockerBOO/Qwen3.5-9B-heretic-ara-v1 (a Heretic ARA-decensored Qwen/Qwen3.5-9B), as a single safetensors file for ComfyUI.

Files

Source (BF16, single file): 18.8 GB.

Layout

• NVFP4 (96 tensors): MLP gate_proj / up_proj / down_proj in all 32 blocks
• INT8 ConvRot, group size 256 (104 tensors): self_attn q/k/v/o and linear_attn in_proj_qkv / in_proj_z / out_proj
• BF16: embeddings, lm_head, norms, in_proj_a / in_proj_b, conv1d, vision tower

Hardware

NVFP4 layers need an NVIDIA Blackwell GPU (SM ≥ 10.0) for native FP4 compute. Use ComfyUI with quantized-checkpoint (comfy_quant) support.

Usage

Put the file in ComfyUI/models/text_encoders/ and load it with a CLIP/text-encoder loader, or use it with the TextGenerate node.

Quality

Checked for functional correctness only: it loads in ComfyUI and generates coherent text (factual Q&A, math, code, formatting, translation) comparable to the BF16 source on a small prompt set. No benchmark or perplexity evaluation has been run.

Quantization

convert_to_quant, learned rounding (default):

ctq -i Qwen3.5-9B-heretic-ara-v1.safetensors \
    -o qwen3.5-9b-heretic-ara-v1_nvfp4_convrot_int8.safetensors \
    --nvfp4 --fallback fp8 \
    --layer-config qwen35_9b_layer_config.json \
    --comfy_quant --save-quant-metadata

ConvRot: arXiv:2512.03673.

License

Apache License 2.0, inherited from Qwen/Qwen3.5-9B; the license text is included in LICENSE. This is a derivative of the Heretic-modified model linked above.