Wan 2.2 T2V-A14B — NF4 for LoRA training
NF4 (4-bit) version of Wan 2.2 T2V-A14B used by AcademiaSD LoRAlab Trainer Studio to train Wan 2.2 video and image LoRAs on 16 GB GPUs. The trainer downloads it automatically, and only the expert you train.
These weights are a derivative of Wan 2.2 and are distributed under the same Apache 2.0 license.
Contents
Wan 2.2 A14B splits the denoising steps between two 14B experts. Each one is 57 GB in FP32 in the original repository and 7.7 GB here.
| Folder | What it is | Size |
|---|---|---|
transformer_nf4/ |
High noise expert (diffusers transformer, steps with t ≥ 875): layout and motion |
7.7 GB |
transformer_2_nf4/ |
Low noise expert (diffusers transformer_2): detail, faces, style |
7.7 GB |
text_encoder/ |
UMT5-XXL, BF16, as in the original repository | 11.4 GB |
vae/, tokenizer/, scheduler/, model_index.json |
Unchanged from the original repository | 0.5 GB |
transformer/, transformer_2/ |
Only the config.json of each expert, used to build the model before loading the NF4 weights |
— |
Format
This is not a standard diffusers or bitsandbytes checkpoint: from_pretrained does not load it. Each *_nf4/ folder holds:
part_01.safetensors…part_12.safetensors, one for each FP32 shard of the original.index.json, which lists every quantized layer with its file and shape.config.json, the expert's diffusers config.
How the weights are stored:
- NF4: the
nn.Linearweights of the 40 transformer blocks, quantized withbitsandbytes.functional.quantize_4bit(blocksize 64, compressed statistics: the same asParams4bit.cuda()). TheQuantStateof each one is saved next to it as<name>.weight.qs.*, fromQuantState.as_dict(packed=True). - BF16: the patch embedding, the timestep and text embeddings, and the output projection.
- FP32: the norms and the modulation tables (
scale_shift_table), as in diffusers'_keep_in_fp32_modules.
The trainer builds each expert empty from its config, replaces the block layers with Linear4bit and fills them from these files (load_expert in scripts/wan22_model.py). The BF16 model is never in memory. The conversion code (ensure_expert in the same file) makes this same folder from the original repository, if you prefer to convert it yourself.
LoRAs trained with it
The trainer saves the LoRAs with the original Wan key names (diffusion_model.blocks.N.…), one for each expert (_high_noise / _low_noise). They load on the normal Wan 2.2 T2V-A14B models in ComfyUI, Wan2GP and diffusers. The NF4 model is only needed for training.