Model Card (SVDQuant)

Language: English | 中文

Important: This quantized model cannot run with the official upstream nunchaku package (the official repo has not been actively maintained for a long time). Install vitoom-nunchaku prebuilt wheels, or deploy via vitoom for a ready-to-use platform.

Model and upstream

• Quantized weights repo: tonera/Qwen-Image-2512-Lightning-Nunchaku

• Lightning distilled source: lightx2v/Qwen-Image-2512-Lightning

• Official full-precision base: Qwen/Qwen-Image-2512

• Quantized Transformer in this repo: svdq-<precision>_r32-Qwen-Image-2512-Lightning-Nunchaku.safetensors; <precision> is commonly fp4 or int4. Pick the matching file according to your vitoom inference environment and the return value of get_precision(). The repo may also provide variants such as int8

• Optional quantized text encoder: svdq-int4-Qwen2.5vl-Nunchaku.safetensors from tonera/Qwen2.5vl-Nunchaku (same as Qwen-Image-Edit-2511-Lightning-Nunchaku; not svdq-int4-Qwen2.5vl-text-Nunchaku.safetensors)

Qwen-Image-2512-Lightning is a lightweight text-to-image model distilled from Qwen-Image-2512; it typically produces results in 4–8 steps. Hardware and upstream details are on the Lightning model card.

Quantization quality

The Qwen-Image family is sensitive to weight quantization. The FP4 Transformer in this repo scores about 7.5–8 / 10 in subjective evaluation (vs. the full-precision Lightning baseline).

Objective metrics from this repo (data.txt, N=5):

For the companion text encoder (tonera/Qwen2.5vl-Nunchaku), hidden-states metrics are cosine ≈ 0.969 and rel_l2 ≈ 0.247; see that repo for details.

Install vitoom-nunchaku (Option 1: manual Python environment)

Install the prebuilt wheel from tonera/vitoom-nunchaku that matches your platform, Python, and CUDA:

pip install "git+https://github.com/huggingface/diffusers.git"

pip install torch==2.11.* torchvision==0.26.* torchaudio==2.11.* \
  --index-url https://download.pytorch.org/whl/cu130

hf download tonera/vitoom-nunchaku \
  nunchaku-1.3.0.dev20260622+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl \
  --local-dir ./wheels

pip install ./wheels/nunchaku-1.3.0.dev20260622+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl

For cu128, cp310, or ARM64 aarch64 wheels, see the vitoom-nunchaku README.

Verify:

python -c "import nunchaku; from nunchaku import NunchakuQwenImageTransformer2DModel; print(nunchaku.__version__)"

Usage example (text-to-image + quantized Transformer + quantized text encoder)

The example below assumes vitoom-nunchaku is installed. Loading is the same as for Qwen-Image-Edit-2511-Lightning-Nunchaku (QwenImageEditPlusPipeline + NunchakuQwenEncoderModel). Replace REPO / TE_REPO with your local path or Hugging Face ID.

import torch
from diffusers import QwenImageEditPlusPipeline

from nunchaku import NunchakuQwenEncoderModel, NunchakuQwenImageTransformer2DModel
from nunchaku.torch_transfer_utils import pretouch_pipeline_cpu_tensors
from nunchaku.utils import get_precision

REPO = "tonera/Qwen-Image-2512-Lightning-Nunchaku"
TE_REPO = "tonera/Qwen2.5vl-Nunchaku"
NAME = "Qwen-Image-2512-Lightning-Nunchaku"

torch_dtype = torch.bfloat16

# Optional: quantized text encoder for much lower VRAM
text_encoder = NunchakuQwenEncoderModel.from_pretrained(
    f"{TE_REPO}/svdq-int4-Qwen2.5vl-Nunchaku.safetensors"
)

transformer = NunchakuQwenImageTransformer2DModel.from_pretrained(
    f"{REPO}/svdq-{get_precision()}_r32-{NAME}.safetensors"
)

pipe = QwenImageEditPlusPipeline.from_pretrained(
    REPO,
    text_encoder=text_encoder,
    transformer=transformer,
    torch_dtype=torch_dtype,
)
pretouch_pipeline_cpu_tensors(
    pipe, ("text_encoder", "text_encoder_2", "vae", "unet", "transformer")
)
pipe.to("cuda")

result = pipe(
    prompt="A cute cat sitting on a windowsill, sunlight streaming in, high-quality photography, rich detail",
    negative_prompt=" ",
    width=1024,
    height=1024,
    num_inference_steps=8,
    true_cfg_scale=1.0,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]
result.save("qwen_2512_lightning_nunchaku.png")

To use only the quantized Transformer and keep the native text encoder, omit the text_encoder= argument.

CPU offload when VRAM is tight

pipe.transformer.set_offload(True, use_pin_memory=True, num_blocks_on_gpu=1)
pipe._exclude_from_cpu_offload.append("transformer")
pipe.enable_sequential_cpu_offload()

LoRA

This model supports stacking LoRA on the quantized Transformer (e.g. Lightning community LoRA or custom LoRA):

from nunchaku.lora.common.compose import compose_lora

lora_path = "/path/to/your_lora.safetensors"
transformer.update_lora_params(compose_lora([(lora_path, 0.5)]))

Recommended inference settings

How to run

Option 2 (recommended: vitoom platform)

1. Clone and install vitoom: see Quick install or docker-usage-en.md
2. Download and activate this model’s weights under Models in the vitoom Web UI
3. Run text-to-image inference in the Image workspace

The vitoom Visual inference image already includes vitoom-nunchaku—no manual wheel install required.

License and compliance

Quantized weights are derived from Qwen-Image-2512 and the Lightning distilled release. The upstream Qwen model is generally under Apache 2.0; Lightning LoRA / fused weights follow the terms on lightx2v/Qwen-Image-2512-Lightning. Confirm upstream license terms before commercial use or redistribution.