Model Card (SVDQuant / Nunchaku)

Language: English | 中文

Model and upstream

• Quantized weights repo: tonera/FLUX.2-klein-4B-Nunchaku
• Official full-precision source: black-forest-labs/FLUX.2-klein-4B
• Quantized Transformer in this repo: svdq-<precision>_r32-FLUX.2-klein-4B-Nunchaku.safetensors; use nunchaku.utils.get_precision() for <precision> (commonly fp4 or int4) so the file name matches your environment and Nunchaku build

Use in ComfyUI

Install the Vitoom Nunchaku plugin. ComfyUI 0.44 or newer is required, started with --enable-manager. Installing the plugin also installs a matching nunchaku wheel.

1. Open ComfyUI and click Manage Extensions in the top-right corner.
2. Search for Vitoom Nunchaku and click Install.
3. Restart ComfyUI.

Place this repository's transformer in ComfyUI/models/diffusion_models: svdq-fp4_r32-FLUX.2-klein-4B-Nunchaku.safetensors for RTX 50 series, svdq-int4_r32-FLUX.2-klein-4B-Nunchaku.safetensors for RTX 30 and 40 series. Place the text encoder svdq-int4-Qwen3-4B-text-Nunchaku.safetensors in ~/weights or ComfyUI/models/text_encoders. Place the VAE flux2-vae.safetensors in ComfyUI/models/vae.

Drag example_workflows/flux2_klein.json from the plugin onto the canvas, or open it from the workflow templates. Connect Vitoom Nunchaku DiT Loader to KSampler, and Vitoom Nunchaku Text Encoder to the encode nodes. Disconnect the reference image for text-to-image. For editing, connect it through ReferenceLatent.

Option 1 (for users comfortable with coding)

• Engine: vitoom-nunchaku — community-maintained Nunchaku build with FLUX.2 Klein support (FP4/INT4 SVDQuant inference)
• Framework: Diffusers with Flux2KleinPipeline support (official examples install from source):

pip install "git+https://github.com/huggingface/diffusers.git"

Install vitoom-nunchaku (recommended)

Upstream Nunchaku has not merged FLUX.2 Klein support for a long time (PR #926 still pending). Do not copy patch files manually. Install the prebuilt wheel that matches your platform, Python, and CUDA:

1. Install matching PyTorch 2.11 (cu128 or cu130) first.
2. Download and install a wheel from tonera/vitoom-nunchaku.

Example (x86_64, Python 3.11, CUDA 13.0):

pip install torch==2.11.* torchvision==0.26.* torchaudio==2.11.* \
  --index-url https://download.pytorch.org/whl/cu130

hf download tonera/vitoom-nunchaku \
  nunchaku-1.3.0.dev20260622+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl \
  --local-dir ./wheels

pip install ./wheels/nunchaku-1.3.0.dev20260622+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl

For other platforms (cu128, cp310, ARM64 aarch64), see the wheel table in the vitoom-nunchaku README.

Verify:

python -c "import nunchaku; from nunchaku import NunchakuFlux2Transformer2DModel; print(nunchaku.__version__)"

Minimal example (image-to-image + quantized Transformer)

Assumes vitoom-nunchaku is installed and weights are available locally or at tonera/FLUX.2-klein-4B-Nunchaku; set REPO to your directory or Hugging Face model id.

import torch
from diffusers import Flux2KleinPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuFlux2Transformer2DModel
from nunchaku.utils import get_precision

REPO = "tonera/FLUX.2-klein-4B-Nunchaku"  # or local absolute path
NAME = "FLUX.2-klein-4B-Nunchaku"

transformer = NunchakuFlux2Transformer2DModel.from_pretrained(
    f"{REPO}/svdq-{get_precision()}_r32-{NAME}.safetensors",
    torch_dtype=torch.bfloat16,
)
pipe = Flux2KleinPipeline.from_pretrained(
    REPO, torch_dtype=torch.bfloat16, transformer=transformer
)

pipe.to("cuda")

ref = load_image("https://example.com/your_ref.png").convert("RGB")
image = pipe(
    prompt="Describe your edit in English…",
    image=ref,
    guidance_scale=1.0,  # matches official Klein examples; tune if needed
    num_inference_steps=4,  # common for the distilled model; see Diffusers docs otherwise
    generator=torch.Generator("cpu").manual_seed(1),
).images[0]
image.save("flux2_klein_4b_nunchaku.png")

For text-to-image, omit image (behavior per current Diffusers Flux2KleinPipeline docs). Use pipe.enable_model_cpu_offload() or similar if VRAM is tight.

Recommended inference settings

If the steps above feel too difficult, install the vitoom platform instead (see below).

Option 2 (recommended: vitoom)

We recommend deploying and running via vitoom: it ships a complete vitoom-nunchaku runtime, Web UI, multi-reference editing, and LoRA management—no manual wheel install or file copying. See the vitoom repo and docker-usage-en.md for setup.

1. Prepare the environment

You need Docker, Docker Compose, and an NVIDIA GPU for inference (driver must support CUDA 13.0, matching vitoom’s cu130 inference images). Confirm GPU access:

docker run --rm --gpus all nvidia/cuda:13.0.0-base-ubuntu24.04 nvidia-smi

2. Generate config and pull images

In the vitoom repo root, run the setup wizard (recommended):

git clone https://github.com/tonera/vitoom.git
cd vitoom
python scripts/setup_vitoom.py

Or copy .env.example to .env, fill in required fields (VITOOM_BACKEND_URL, VITOOM_WS_URL, VITOOM_INFERENCE_UPLOAD_AUTH_SECRET, etc.), then pull images:

python scripts/load_vitoom_images.py --components backend,visual

3. Start services

Start Backend:

docker compose up -d backend

Open http://127.0.0.1:8888 in your browser (port follows VITOOM_SERVER_PORT in .env).

Start the Visual inference service (FLUX.2 Klein is an image model; this profile is required):

docker compose -f docker-compose.inference.release.yml --profile visual up -d

Optional: run python scripts/download_initial_models.py to batch-download common models; or place weights under the host resources/models directory (override via VITOOM_MODELS_HOST_DIR in .env).

4. Use this model in the Web UI

1. Log in to the vitoom Web UI, open Models, download and activate tonera/FLUX.2-klein-4B-Nunchaku
2. Open the Image workspace, select the activated FLUX.2 Klein 4B model, and run text-to-image or image editing (multi-reference editing supported)

You can also run the Python example above directly inside the vitoom Visual inference container.

LoRA

FLUX.2 Klein supports LoRA on the quantized Transformer when using vitoom-nunchaku (wheel or vitoom Visual image)—no manual file copying required:

lora_path = "/path/to/your_lora.safetensors"
transformer.update_lora_params(lora_path)
transformer.set_lora_strength(0.8)

Acknowledgments

• nunchaku: a high-performance inference engine.
• deepcompressor: a model compression toolbox.
• fluxsd.com: image, video, poster, comic, and picture-book generation.
• createpptx.com: high-quality editable PowerPoint generation.

License and compliance

These quantized weights are derived from FLUX.2-klein-4B. The upstream model is under Apache 2.0; confirm upstream license terms and Black Forest Labs’ acceptable use policy before commercial use or redistribution.