Qwen3-4B text encoder GGUF, quantized by what the DiT sees (DHQ)

Two 2.9 GB GGUFs of the Qwen3-4B text encoder, quantized with DHQ, a per-tensor mixed-precision allocation that is scored by the diffusion model rather than by next-token KL.

About the DHQ series

DHQ files are quantized for use as a diffusion model's text encoder: each tensor's precision is chosen by how much its quantization error changes what the target DiT reads (measured through the DiT's own text-input stage), not by next-token loss. They stay normal LLM GGUFs, so the same file can still write or rewrite prompts: where a model is also used as a prompt enhancer (Z-Image-Engineer-V6, the heretic Qwen3-VL-8B), every file is also checked for that (the PE column) and the tables show where it gets worse.

DHQ repos: Qwen-Image-2.1 · Ideogram 4 · Boogu · Z-Image-Engineer-V6 · Qwen3-4B (Z-Image / FLUX.2 klein 4B) · FLUX.3 Klein

We also built Dual-Head Dragon (ComfyUI-DualHead-Dragon), a ComfyUI node that runs one GGUF as both the text encoder and the prompt enhancer (llama.cpp inside). It is public but still in testing, including its LoRA support (separate LoRAs for the encoding and the rewriting head) and DFlash speculative decoding for faster rewriting. The files here work without it (see How to load).

Why

A text encoder feeding a DiT is judged by the hidden states the DiT reads (Z-Image reads layer 34; klein reads layers 8/17/26), not by the next token. Standard quants (llama.cpp Q4_K_M, Unsloth Dynamic) spend bits based on LLM output. DHQ measures each tensor's damage through the DiT's input layer and spends bits where the DiT feels it.

Rules for these files: every tensor is at least Q4_K; the size budget is 2.9 GB. Mix: Q4_K / Q5_K / Q6_K / Q8_0 per tensor. The full per-layer table (q/k/v/o/gate/up/down for every layer) is in GRIDS.md.

Measured error (real GGUFs, lower is better)

Metric: relative error of the text-encoder output after the DiT's input layer, compared with BF16, averaged over 18 prompts (12 long, 6 short). All files run through llama.cpp's real kernels.

Qwen3-4B

Z-Image-Engineer-V6

The V6 file uses an importance matrix computed on V6 itself (same calibration text as the Qwen3-4B file). On 36 held-out prompts never used to pick the mix it scores 4.4%.

Formats

Both files use only K-quant types (Q4_K / Q5_K / Q6_K / Q8_0 per tensor), so they decode fast everywhere (GPU, CPU, Metal, Vulkan, and ComfyUI-GGUF on the GPU). There are no IQ files in this repo.

Usage

ComfyUI (needs ComfyUI-GGUF): put the file in models/text_encoders/, load it with CLIPLoaderGGUF, type lumina2 for Z-Image or flux2 for FLUX.2 [klein] 4B. Everything else is the stock workflow.

llama.cpp / LM Studio: both are standard qwen3 GGUFs. The V6 file can write Z-Image prompts the same way the original V6 does.

Reproducibility

recipes/ has one recipe per file: source repo and revision, llama.cpp commit, importance matrix (sha256), and the type of every tensor as built. Rebuilding from a recipe gives a bit-identical file (checked).

sha256

1be4aa21207e48ac7a06a9b5a6656d3aa8b612769a3cbcf9b6960c173860ab42  qwen3_4b_dhq_2p9.gguf
81125bf274d28b0481984ac0da2bacbf9d7505e58d34962c0e6b0da886d48762  zimage_engineer_v6_dhq_2p9.gguf

License

Apache-2.0, same as both source models. Credit to the Qwen team for Qwen3-4B and to BennyDaBall for Z-Image-Engineer-V6.