FP8 weight-only quantization of NucleusAI/Nucleus-Image. Single 16.94 GB safetensors file at the repo root (was 33.85 GB in BF16). Peak VRAM at 1024² is 17.6 GB with enable_model_cpu_offload() — fits comfortably on 24 GB cards.
import importlib.util, torch
from diffusers import DiffusionPipeline
from huggingface_hub import hf_hub_download
REPO = "D-Squarius-Green-Jr/Nucleus-Image-FP8"
patch_py = hf_hub_download(REPO, "moe_fp8_patch.py")
weights = hf_hub_download(REPO, "Nucleus-Image-FP8.safetensors")
hf_hub_download(REPO, "config.json") # sits next to the weights
spec = importlib.util.spec_from_file_location("moe_fp8_patch", patch_py)
patch = importlib.util.module_from_spec(spec); spec.loader.exec_module(patch)
patch.apply_patch()
transformer = patch.load_fp8_safetensors_transformer(weights)
pipe = DiffusionPipeline.from_pretrained(
"NucleusAI/Nucleus-Image",
transformer=transformer,
torch_dtype=torch.bfloat16,
)
pipe.enable_model_cpu_offload()
image = pipe(
prompt="A quiet alpine lake at sunrise, mist rising off still water, snow-capped peaks reflected, soft pink and gold sky",
width=1024, height=1024,
num_inference_steps=20, guidance_scale=8.0,
generator=torch.Generator(device="cuda").manual_seed(42),
).images[0]
image.save("out.png")
Skip
TextKVCacheConfigif running multiple prompts of different lengths — its state has no public reset (as ofdiffusers0.38.0.dev0). Single prompts are fine.
| Layers | How | Size |
|---|---|---|
354 nn.Linear |
FP8 e4m3, per-output-channel scale | 1.6 GB |
29 SwiGLUExperts (15.3 B params, fused 64-expert tensors) |
FP8 e4m3, per-expert × per-output-channel scale | 15.3 GB |
| Norms / routing logits / embeddings | BF16 (routing decisions stay bit-identical to BF16) | 0.04 GB |
Off-the-shelf quantizers (TorchAO, optimum-quanto, bnb) only walk nn.Linear, missing the 15.3 B inside SwiGLUExperts. The runtime patch handles both.
x): 4.49 %The text below is the verbatim model card from NucleusAI/Nucleus-Image.
🌐 Website | 🖥️ GitHub | 🤗 Hugging Face | 📑 Tech Report
Nucleus-Image is a text-to-image generation model built on a sparse mixture-of-experts (MoE) diffusion transformer architecture. It scales to 17B total parameters across 64 routed experts per layer while activating only ~2B parameters per forward pass, establishing a new Pareto frontier in quality-versus-efficiency. Nucleus-Image matches or exceeds leading models including Qwen-Image, GPT Image 1, Seedream 3.0, and Imagen4 on GenEval, DPG-Bench, and OneIG-Bench. This is a base model released without any post-training optimization (no DPO, no reinforcement learning, no human preference tuning). All reported results reflect pre-training performance only. We release the full model weights, training code, and dataset, making Nucleus-Image the first fully open-source MoE diffusion model at this quality tier.
diffusers pipeline. Simply enable it with TextKVCacheConfig for automatic speedup with no code changes to the inference loopNucleus-Image is a 32-layer diffusion transformer where 29 of the 32 blocks replace the dense FFN with a sparse MoE layer containing 64 routed experts and one shared expert (the first 3 layers use dense FFN for training stability). Image queries attend to concatenated image and text key-value pairs via joint attention. Text tokens are excluded from the transformer backbone entirely, participating only as KV contributors. This eliminates MoE routing overhead for text and enables full text KV caching across denoising steps.
Routing uses Expert-Choice with a decoupled design: the router receives the unmodulated token representation concatenated with the timestep embedding, while expert MLPs receive the fully modulated representation. This prevents the adaptive modulation scale — which varies by an order of magnitude across timesteps — from collapsing expert selection into timestep-dependent routing, preserving spatial and semantic expert specialization.
| Specification | Value |
|---|---|
| Total parameters | 17B |
| Active parameters | ~2B |
| Architecture | Sparse MoE Diffusion Transformer |
| Layers | 32 |
| Hidden dimension | 2048 |
| Attention heads (Q / KV) | 16 / 4 (GQA) |
| Experts per MoE layer | 64 routed + 1 shared |
| Expert hidden dimension | 1344 |
| Text encoder | Qwen3-VL-8B-Instruct |
| Image tokenizer | Qwen-Image VAE (16ch) |
| Training data | 700M images, 1.5B caption pairs |
| Training curriculum | Progressive resolution (256 → 512 → 1024) |
| Total training steps | 1.7M |
Nucleus-Image achieves state-of-the-art or near state-of-the-art results on all three benchmarks despite activating only ~2B of its 17B parameters per forward pass. All results are from the base model at 1024x1024, 50 inference steps, CFG scale 8.0.
| Benchmark | Score | Highlights |
|---|---|---|
| GenEval | 0.87 | Matches Qwen-Image; leads all models on spatial position (0.85) |
| DPG-Bench | 88.79 | #1 overall; leads in entity (93.08), attribute (92.20), and other (93.62) |
| OneIG-Bench | 0.522 | Surpasses Imagen4 (0.515) and Recraft V3 (0.502); strong style (0.430) |
Install the latest version of diffusers:
pip install git+https://github.com/huggingface/diffusers
Generate images with Nucleus-Image:
import torch
from diffusers import DiffusionPipeline
from diffusers import TextKVCacheConfig
model_name = "NucleusAI/Nucleus-Image"
pipe = DiffusionPipeline.from_pretrained(model_name, torch_dtype=torch.bfloat16)
pipe.to("cuda")
# Enable Text KV caching across denoising steps (integrated into diffusers)
config = TextKVCacheConfig()
pipe.transformer.enable_cache(config)
# Supported aspect ratios
aspect_ratios = {
"1:1": (1024, 1024),
"16:9": (1344, 768),
"9:16": (768, 1344),
"4:3": (1184, 896),
"3:4": (896, 1184),
"3:2": (1248, 832),
"2:3": (832, 1248),
}
prompt = "A weathered lighthouse on a rocky coastline at golden hour, waves crashing against the rocks below, seagulls circling overhead, dramatic clouds painted in shades of amber and violet"
width, height = aspect_ratios["16:9"]
image = pipe(
prompt=prompt,
width=width,
height=height,
num_inference_steps=50,
guidance_scale=8.0,
generator=torch.Generator(device="cuda").manual_seed(42),
).images[0]
image.save("nucleus_output.png")
Nucleus-Image generations of human subjects and portraits, spanning diverse cultures, ages, and artistic styles. From expressive character studies to fine-grained close-ups with intricate skin texture and detail.
Nucleus-Image generations spanning fantasy, surrealism, animation, and the natural world.
Nucleus-Image generations across product photography, architecture, typography, food, and world culture, demonstrating versatility in commercial, conceptual, and everyday imagery.
Nucleus-Image is licensed under Apache 2.0.
@misc{nucleusimage2026,
title={Nucleus-Image: Sparse MoE for Image Generation},
author={Nucleus AI Team},
year={2026},
eprint={XXXX.XXXXX},
archivePrefix={arXiv},
primaryClass={cs.CV},
}
NucleusAI/Nucleus-Image