FLUX.3 Klein Text Encoder (Heretic Qwen3-VL-4B) — DHQ GGUF

FLUX.3 Klein's text encoder is Qwen3-VL-4B-Instruct, unmodified. This repo has a heretic (abliterated) version of it, in bf16 and as a GGUF quantized with DHQ = Dual-Head Quant: every tensor gets the precision that matters to what the FLUX.3 Klein DiT actually reads, measured per tensor with gradients through the DiT's own text path, and checked a second time for prompt writing.

Made mainly for Dual-Head Dragon (one model as both the prompt enhancer and the text encoder), but it works as a plain FLUX.3 Klein text encoder too: the bf16 file with ComfyUI's own CLIPLoader, the GGUF with ComfyUI-GGUF or llama.cpp (see How to load).

• Heretic: refusals 2/100 (original 97/100), KL divergence 0.037 to the original (Heretic, 892-trial search). Only self_attn.o_proj in 31 layers is changed; everything else is bit-identical to Qwen3-VL-4B-Instruct.
• One file, Q4_K_M class, 2.66 GB: the Q4_K_M budget plus 165 MB spent on the tensors FLUX.3's padding depends on (see below). Fewer errors than the standard Q4_K_M on every measure.

About the DHQ series

DHQ files are quantized for use as a diffusion model's text encoder: each tensor's precision is chosen by how much its quantization error changes what the target DiT reads (measured through the DiT's own text-input stage), not by next-token loss. They stay normal LLM GGUFs, so the same file can still write or rewrite prompts: where a model is also used as a prompt enhancer, every file is also checked for that (the PE column) and the tables show where it gets worse.

DHQ repos: Qwen-Image-2.1 · Ideogram 4 · Boogu · Z-Image-Engineer-V6 · Qwen3-4B (Z-Image / FLUX.2 klein 4B) · FLUX.3 Klein

We also built Dual-Head Dragon (ComfyUI-DualHead-Dragon), a ComfyUI node that runs one GGUF as both the text encoder and the prompt enhancer (llama.cpp inside). It is public but still in testing. For FLUX.3 Klein it is used for prompt rewriting only (see Master styles on FLUX.3 Klein); encode with ComfyUI's own loader. The files here work without it (see How to load).

Results

Same heretic bf16 source. Standard = llama.cpp Q4_K_M rules + the same imatrix. TE = relative error of the text conditioning after FLUX.3 Klein's text path (txt_in and the 5 text-only blocks, which run once per prompt, before the image is involved), on 36 held-out prompts never used to build or pick anything, split into the prompt's own tokens and the padding (lower is better). PE = mean KL divergence of next-token predictions vs bf16 on 80 prompt-rewrite samples written with BFL's own t2i system prompt (lower is better).

Why the padding matters. FLUX.3 Klein pads every prompt to a multiple of 80 tokens, and the DiT attends to the padding: a 20-word prompt is three quarters padding. The padding has no content of its own, its hidden states come almost entirely from attention to the real tokens, so quantization errors in attention hit it hard: at plain 4 bit its error per position is ~50× that of the text, and one tensor (layer 0 q_proj) carries a third of it. The uploaded file puts the 32 tensors that matter most to the padding at Q8_0 (+165 MB), which takes the padding error from 11.4% to 6.5%. Raising token_embd (also the output layer here) to Q8_0 as well was tested and changed nothing.

How TE was measured: each GGUF's quantized weights were dequantized exactly into ComfyUI's own FLUX.3 text encoder (templates, padding and the 8 tapped layers as in ComfyUI), then scored through the DiT's text path. This covers the whole weight error but not the activation rounding llama.cpp adds at run time (small at 4–5 bit).

The heretic model itself, vs the original on the same benchmark subsets (bf16, chat template): GSM8K 85% vs 87% (100 problems), IFEval 82% vs 80% (50 prompts) — within sampling noise.

As a prompt enhancer. BFL's model card says FLUX.3 Klein should not be used without a prompt upsampler. This model can rewrite prompts with BFL's system_prompts.json, and it does not refuse. In a small side-by-side (4 prompts, same seed), BFL's own upsampler (a fine-tuned Qwen3.5-9B) followed stated styles and framing better (e.g. it keeps "charcoal drawing" a drawing); the untuned Qwen3.5-9B made the same mistakes as this 4B, so the gap comes from BFL's fine-tune, not the model size. Note: ComfyUI's TextGenerate node does not work with the flux3 text encoder type (it returns #, also with the original bf16 weights); rewrite with llama.cpp instead.

How to load

ComfyUI, bf16: put flux3-klein-te-heretic-bf16.safetensors in models/text_encoders/ and load it with the standard CLIPLoader, type flux3, in place of qwen3vl_4b_bf16.safetensors in BFL's FLUX.3 Klein workflow. No add-on needed. The tensor names are the Hugging Face ones: renamed to model.safetensors next to the source repo's config and tokenizer files, it also loads with transformers.

ComfyUI, GGUF: needs ComfyUI-GGUF and the add-on ComfyUI-GGUF-Qwen3VL-TE (commit 3eea253 or later), and a ComfyUI with FLUX.3 support. Put the GGUF and mmproj-flux3-klein-te-heretic-dhq-f16.gguf (this repo) in models/text_encoders/ without renaming them (the mmproj is found by file-name prefix), then CLIPLoaderGGUF with type flux3, in place of the text encoder loader of BFL's FLUX.3 Klein workflow.

[!WARNING] Without the add-on (or with an older version) ComfyUI does not see the vision tower and builds the Z-Image (Qwen3-4B) text encoder from this file, without an error: one 2560-wide layer instead of FLUX.3's 8 stacked layers. With the add-on the conditioning matches the bf16 safetensors encoder (per-token cos 1.0000 on text, 0.9999 on padding, checked with a Q8_0 GGUF on ComfyUI 0.41.0).

llama.cpp / Dual-Head Dragon: a standard qwen3vl GGUF (text generation, prompt rewriting). The mmproj adds image input.

Master styles on FLUX.3 Klein

The Master Style prompt enhancer (role-play system prompt + style cards) runs on this GGUF in Dual-Head Dragon with its 4B LoRA and DFlash2 draft, and FLUX.3 Klein draws the result. Nothing new to install beyond those two nodes:

The whole graph: klein/workflow-flux3-klein-master-style-api.json (ComfyUI API format; Monet card, seed 42).

• The system prompt is the Master Style one for Z-Image, unchanged (the LoRA was trained on it), followed by a short section with FLUX.3 Klein's caption conventions from BFL's own upsampler prompt: open with the medium (a photograph as a photograph; a painting, print or poster by its own medium), one prose block, every person an adult with an observable profile, frame-relative positions, quoted text verbatim.
• Type lumina2 only picks a Qwen3 chat template for rewriting; Dual-Head Dragon has no FLUX.3 encoder type yet, so the encoding is done by ComfyUI's CLIPLoader. The text encoder is then in memory twice (the 2.66 GB GGUF for rewriting + the 8.88 GB bf16 file for encoding); on a 16 GB card ComfyUI swaps models between steps.
• Speed (RTX 5060 Ti 16 GB): rewrite ~5 s (DFlash2, ~480 tokens), image ~13 s with the swapping above.
• The LoRA was trained on Z-Image-Engineer V6 (Qwen3-4B), not on this model. With this system prompt, 129 rewrites (15 painters, 11 poster styles, 12 photo styles, these samples) never ran away. Long system prompts of your own can make it repeat itself until max_new_tokens; run it again or shorten the prompt.

Samples: the 29 Master Style showcase prompts (same prompts and seeds as the Dual-Head Dragon showcase for the other image models), FLUX.3 Klein NVFP4, settings as above.

Reproducibility

recipes/ has the full recipe: source repo and revision, the heretic parameters (to rebuild the heretic weights from Qwen3-VL-4B-Instruct), the llama.cpp commit, the importance matrix with its calibration text, and the type of every tensor (*.grid.md shows them per layer). Built on aarch64 (GB10).

License

Apache-2.0, same as the base weights (Qwen3-VL-4B-Instruct, Alibaba Cloud / Qwen). See LICENSE and NOTICE. This repo contains no FLUX.3 weights; FLUX.3 Klein is released by Black Forest Labs under its own license.