Model Card (SVDQuant)

Language: English | 中文

Model Name

• Model repo: tonera/waiSHUFFLENOOB_vPred03
• Base (Diffusers weights path): tonera/waiSHUFFLENOOB_vPred03 (repo root)
• Quantized UNet weights: tonera/waiSHUFFLENOOB_vPred03/svdq-<precision>_r32-waiSHUFFLENOOB_vPred03.safetensors

Quantization / Inference Tech

• Inference engine: Nunchaku (https://github.com/nunchaku-ai/nunchaku)

Nunchaku is a high-performance inference engine for 4-bit (FP4/INT4) low-bit neural networks. Its goal is to significantly reduce VRAM usage and improve inference speed while preserving generation quality as much as possible. It implements and productionizes post-training quantization methods such as SVDQuant, and reduces the overhead introduced by low-rank branches via operator/kernel fusion and other optimizations.

The SDXL quantized weights in this repository (e.g. svdq-*_r32-*.safetensors) are intended to be used with Nunchaku for efficient inference on supported GPUs.

Quantization Quality (fp8)

PSNR: mean=15.416 p50=14.7882 p90=18.5321 best=19.6476 worst=12.6638 (N=25)
SSIM: mean=0.660081 p50=0.636783 p90=0.780973 best=0.826327 worst=0.530204 (N=25)
LPIPS: mean=0.360286 p50=0.357911 p90=0.490157 best=0.187094 worst=0.636845 (N=25)

Performance

Below is the inference performance comparison (Diffusers vs Nunchaku-UNet).

• Inference config: bf16 / steps=30 / guidance_scale=5.0
• Resolutions (5 images each, batch=5): 1024x1024, 1024x768, 768x1024, 832x1216, 1216x832
• Software versions: torch 2.9 / cuda 12.8 / nunchaku 1.1.0+torch2.9 / diffusers 0.37.0.dev0
• Optimization switches: no torch.compile, no explicit cudnn tuning flags

Cold-start performance (end-to-end for the first image)

Steady-state performance (5 consecutive images after warmup)

Notes:

• The longer load time on RTX 3090 is due to extra one-time processing when loading quantized weights.
• During inference (cold_infer and steady-state), Nunchaku shows clear speedups on both GPUs.

Nunchaku Installation Required

• Official installation docs (recommended source of truth): https://nunchaku.tech/docs/nunchaku/installation/installation.html

(Recommended) Install the official prebuilt wheel

• Prerequisite: PyTorch >= 2.5 (follow the wheel requirements)
• Install Nunchaku wheel: choose a wheel matching your torch/cuda/python versions from GitHub Releases / HuggingFace / ModelScope (note cp311 means Python 3.11):
  • https://github.com/nunchaku-ai/nunchaku/releases

# Example (select the correct wheel URL for your torch/cuda/python versions)
pip install https://github.com/nunchaku-ai/nunchaku/releases/download/vX.Y.Z/nunchaku-X.Y.Z+torch2.9-cp311-cp311-linux_x86_64.whl

• Tip (RTX 50 series): typically prefer CUDA >= 12.8, and prefer FP4 models for compatibility/performance (follow official docs).

Usage Example (Diffusers + Nunchaku UNet)

import torch
from diffusers import StableDiffusionXLPipeline

from nunchaku.models.unets.unet_sdxl import NunchakuSDXLUNet2DConditionModel
from nunchaku.utils import get_precision

MODEL = "waiSHUFFLENOOB_vPred03"  # Replace with the actual model name before publishing (e.g. zavychromaxl_v100)
REPO_ID = f"tonera/{MODEL}"

if __name__ == "__main__":
    unet = NunchakuSDXLUNet2DConditionModel.from_pretrained(
        f"{REPO_ID}/svdq-{get_precision()}_r32-{MODEL}.safetensors"
    )

    pipe = StableDiffusionXLPipeline.from_pretrained(
        f"{REPO_ID}",
        unet=unet,
        torch_dtype=torch.bfloat16,
        use_safetensors=True,
    ).to("cuda")

    prompt = "Make Pikachu hold a sign that says 'Nunchaku is awesome', yarn art style, detailed, vibrant colors"
    image = pipe(prompt=prompt, guidance_scale=5.0, num_inference_steps=30).images[0]
    image.save("sdxl.png")