Vitoom Nunchaku (Extended)

English | 中文

Community-maintained fork of Nunchaku. Nunchaku is an SVDQuant inference engine for 4-bit diffusion models (paper: SVDQuant). The upstream project has not been actively maintained for a long time. This fork keeps the original capabilities while extending model and quantization support, and distributes prebuilt wheels on Hugging Face.

Official docs remain useful as a baseline: Nunchaku Docs · DeepCompressor

---

Supported Models

---

Changelog

1.4.1

Quantized inference for FLUX.3 Image Klein and its prompt upsampler. Wheel platforms match 1.4.0: Linux x86_64, Linux aarch64, and Windows, all built for CUDA 13.0.

flux-3-image-klein uses the existing W4A4 SVDQuant path, with FP4 and INT4. Those weights are already packed at export. flux-3-image-klein-pup is the Qwen3.5 prompt upsampler: language-model linear layers and lm_head are W4 TinyChat (INT4), and the vision tower stays in float.

1.4.0

Quantized inference for three additional models. Published wheels cover Linux x86_64, Linux aarch64, and Windows, all built for CUDA 13.0:

• Linux x86_64: Python 3.11 + PyTorch 2.11; Python 3.11 / 3.12 / 3.13 + PyTorch 2.14
• Linux aarch64: Python 3.11 + PyTorch 2.11
• Windows: Python 3.11 / 3.12 / 3.13 + PyTorch 2.14

The filename rule is below; only the version is 1.4.0.

Anima checkpoints store plain INT8. from_pretrained(..., device="cuda") packs them for the kernel during load. Qwen-Image-2.1-Turbo and Krea2 use the existing W4A4 SVDQuant path, and those weights are already packed at export.

1.3.0

1.3.0.dev20260622 through dev20260629. Adds Chroma, FLUX.2 Klein, Qwen-Image W8A16, and quantized Qwen2.5-VL / Qwen3 text encoders. Those platform wheels are still installable; see below.

---

Extensions Over Upstream

---

Wheel Filenames

Prebuilt wheels are published on Hugging Face: tonera/vitoom-nunchaku. Filenames follow one pattern. Replace each part with your own environment to find the matching file in the repo.

nunchaku-{version}+cu{CUDA}torch{PyTorch}-cp{Python}-cp{Python}-{platform}.whl

Take nunchaku-1.4.1+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl as an example:

Not every combination of version, Python, PyTorch, CUDA, and platform is published. Build the filename from the tables above, then confirm the file exists in the repo before downloading.

---

Use in ComfyUI (Recommended)

If you already use ComfyUI, install the Vitoom Nunchaku plugin.

ComfyUI 0.44 or newer is required, started with --enable-manager. The current 1.4.1 wheels target PyTorch 2.14 and CUDA 13.0, on Linux x86_64 and Windows. RTX 50-series GPUs can only use W4A4 checkpoints whose filenames contain svdq-fp4.

1. Open ComfyUI and click Manage Extensions in the top-right corner.
2. Search for Vitoom Nunchaku and click Install.
3. Restart ComfyUI.

Drag a workflow from the plugin's example_workflows/ onto the canvas, or open one from the workflow templates.

Download quantized models from huggingface.co/tonera. Place DiT checkpoints in ComfyUI/models/diffusion_models.

---

Use via Vitoom (Recommended)

If you only need image generation acceleration in production and prefer not to manage Python/CUDA/wheel versions yourself, deploy Vitoom instead of installing the wheel from this repo by hand.

Vitoom is a Docker-based, locally deployable AIGC platform. Through a browser you can run text, image, audio, and video inference on your own hardware (RTX 30 / 40 / 50, DGX Spark, GB10, etc.). Its visual inference module ships with vitoom-nunchaku pre-integrated. After deployment, Chroma, FLUX.2 Klein, FLUX.3 Image Klein, Qwen-Image quantization, and related features are available in the Web UI.

Quick start (see the Vitoom README for full details):

git clone https://github.com/tonera/vitoom.git
cd vitoom
python scripts/setup_vitoom.py
python scripts/load_vitoom_images.py
docker compose up -d backend
docker compose -f docker-compose.inference.release.yml --profile visual up -d

Then open http://<LAN-IP>:8888 in your browser. Download models from the Web UI or via python scripts/download_initial_models.py.

Deployment guides: docker-usage-en.md · docker-usage-cn.md

---

Installation (Manual Wheel)

After downloading the wheel that matches your platform and Python version from Hugging Face:

pip install nunchaku-1.4.1+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl

Or use the Hugging Face CLI:

hf download tonera/vitoom-nunchaku \
  nunchaku-1.4.1+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl \
  --local-dir ./wheels

pip install ./wheels/nunchaku-1.4.1+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl

Running also requires the latest diffusers and transformers source. The PyPI stable releases do not yet include the models added in 1.4.1:

pip install git+https://github.com/huggingface/diffusers.git
pip install git+https://github.com/huggingface/transformers.git

Verify

python -c "import nunchaku; print(nunchaku.__version__)"

---

Citation

If the inference acceleration in this project helps your research, please cite the original SVDQuant paper:

@inproceedings{
  li2024svdquant,
  title={SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models},
  author={Li*, Muyang and Lin*, Yujun and Zhang*, Zhekai and Cai, Tianle and Li, Xiuyu and Guo, Junxian and Xie, Enze and Meng, Chenlin and Zhu, Jun-Yan and Han, Song},
  booktitle={The Thirteenth International Conference on Learning Representations},
  year={2025}
}

---

Acknowledgments

• nunchaku: a high-performance inference engine.
• deepcompressor: a model compression toolbox.
• fluxsd.com: image, video, poster, comic, and picture-book generation.
• createpptx.com: high-quality editable PowerPoint generation.

---

Disclaimer

This is a community-maintained extension of upstream Nunchaku. Wheels are distributed independently by the Vitoom team and are not an official release from MIT Han Lab / nunchaku-ai. For support, please open an issue on the corresponding Hugging Face repository or project tracker.