Wan2.2-T2V-A14B — MXFP8 (weight-quantized)

An MXFP8 (OCP microscaling FP8) weight-quantized build of Wan-AI/Wan2.2-T2V-A14B-Diffusers, produced with Intel auto-round (zero-shot RTN). Only the two diffusion transformers (the MoE expert denoisers) are quantized; the text encoder, VAE, tokenizer and scheduler are kept as-is.

TL;DR

• Whole model 118 GB → 39 GB (≈3.0×); each transformer 54 GB (FP32) → 14 GB (≈3.9×).
• Quality is effectively unchanged: VBench imaging_quality 0.6775 (bf16) → 0.6754 (mxfp8), Δ −0.31 % over 93 prompts.
• Stored weights are true MXFP8 (E4M3 + per-32 E8M0 block scales). On H200 they run through a Marlin dequant kernel (correct, memory-saving, not faster); native MXFP8 tensor-core speedups require Blackwell/B300.

⚠️ Research / evaluation artifact. The original is FP32, so the headline compression ratio is large mostly because of the source precision, not because FP8 is unusually small.

---

Showcase — official example, bf16 vs MXFP8

Reproduction of the official Wan2.2 diffusers usage example, generated on our own vLLM-Omni stack with identical settings (1280×720, 81 frames, 40 steps, seed 42, guidance 4.0/3.0, boundary 0.875, flow-shift 5.0) so the only variable is the quantization.

Prompt: "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

Both clips: 1280×720, 16 fps, 81 frames (5.06 s).

How to read the PSNR/SSIM/LPIPS. They are moderate, not near-identical, because the two videos are not pixel-for-pixel copies. A ~2–3 % MXFP8 weight perturbation, compounded over 40 denoising steps, nudges the sampler onto a different but equally valid trajectory — same subjects, composition, lighting and motion, slightly different pixels. The matching VBench imaging_quality (and the 93-prompt result below) confirm there is no systematic quality loss; the reference-metric distance is sampling chaos, not degradation.

---

What was quantized

WanPipeline (A14B) is a dual-expert MoE: two WanTransformer3DModel denoisers (transformer = high-noise expert, transformer_2 = low-noise expert) switched at boundary_ratio=0.875.

Quantized Linears — only blocks.* Linears, 10 per block × 40 = 400 per transformer:

Per transformer this is 400 × F8_E4M3 weights + 400 × U8 E8M0 block scales ⇒ 98.4 % of weight params are MXFP8. Ignored (kept high precision): patch_embedding, condition_embedder.* (time/text embedders), proj_out, all norms (norm_q/norm_k, norm2), scale_shift_table, biases — plus the entire text_encoder and vae.

Quant format (per transformer/config.json): quant_method=auto-round, data_type=mx_fp, bits=8, group_size=32, sym=True; activations act_data_type=mx_fp, act_bits=8, act_dynamic=True, act_group_size=32 (runtime dynamic, so no static calibration is needed); packing_format=auto_round:llm_compressor; autoround_version=0.17.0.

Reproduce the quantization

CUDA_VISIBLE_DEVICES=1,2 \
auto_round \
    --model /path/to/Wan2.2-T2V-A14B-Diffusers \
    --scheme MXFP8 \
    --dataset opens2v --nsamples 32 \
    --iters 0 \
    --batch_size 1 \
    --calib_num_inference_steps 8 \
    --low_gpu_mem_usage \
    --diffusion_tuning_cache_size auto \
    --output_dir ./wan22-mxfp8

• --iters 0 ⇒ zero-shot RTN. MXFP8 activations are dynamic, so --dataset/--nsamples/--calib_num_inference_steps have no effect here; set --iters > 0 for the calibrated path (both paths quantize both transformers).
• Requires the dual-transformer zero-shot fix from auto-round PR #2481. Without it the RTN path quantizes only transformer and copies transformer_2 out in bf16.

---

Size / compression (measured, du)

The whole-model ratio (3.0×) is diluted by the untouched 11 G text encoder. Had the source been bf16 (~57 G), the same 39 G output would be only ~1.5× — the large ratio is driven by the FP32 origin.

---

Quality — VBench imaging_quality (dense93, n = 93)

Both models generated all 93 prompts with the same recipe (1280×720, 81 frames, 40 steps, seed 42, guidance 4.0/3.0, boundary 0.875, flow-shift 5.0, cache-dit DBCache, same negative prompt), then scored with the same VBench imaging_quality (MUSIQ-SPAQ, no-reference, 0–1).

Mean difference is −0.0021. Per-video deltas are near-symmetric (mxfp8 higher on 42/93, lower on 51/93; mean |Δ| ≈ 1.95 on the 0–100 scale) while per-video scores span 25.8–80.4 — i.e. the ±2 jitter is diffusion sampling non-determinism, not a quantization floor. Because the stored weights are genuine MXFP8, this score faithfully reflects the quantized numerics regardless of which GEMM kernel executes them.

---

Usage (vLLM-Omni)

Served exactly like the original model — vLLM-Omni auto-detects the auto-round MXFP8 config:

vllm serve /path/to/wan22-mxfp8 --omni \
  --host 127.0.0.1 --port 8290 \
  --tensor-parallel-size 1 --ulysses-degree 2 --ulysses-mode strict \
  --ring-degree 1 --vae-patch-parallel-size 2 --cfg-parallel-size 1 \
  --cache-backend cache_dit --enable-cache-dit-summary \
  --boundary-ratio 0.875 --flow-shift 5.0

Then POST a text-to-video request to /v1/videos (1280×720, 81 frames, 40 steps). Needs 2 GPUs (ulysses-degree 2); ~66 GB/GPU at load.

---

Hardware note (important)

H200 (Hopper) has no native MXFP8 tensor core (block-scaled MMA is a Blackwell/B300 feature). vLLM-Omni therefore runs these weights through MarlinMxfp8LinearKernel, a weight-only dequant GEMM: functionally correct and memory-saving, but ~3–4× slower than bf16 (~14–22 s/it vs ~6 s/it). Use this build on H200 to validate quantization accuracy; real inference speedups require B300. Dequantization does not change the stored values, so accuracy matches what a native MXFP8 kernel would produce.

Runtime is W8A16, not W8A8. The Marlin kernel is documented as MXFP8 W8A16 GEMM via Marlin: it dequantizes the MXFP8 weight (E4M3 + E8M0 block scale) inside the GEMM and multiplies it against unquantized bf16 activations (a_scales=None; a 1-byte/fp8 activation path explicitly raises "Marlin W8A8 is not supported."). The checkpoint's act_data_type=mx_fp / act_bits=8 / act_dynamic=True is quantization-recipe metadata that is not exercised on H200 — activations stay bf16. True W8A8 (FP8 activations on a native microscaling tensor core) requires Blackwell/B300.

---

Limitations

• Not faster than bf16 on Hopper (see above).
• Text encoder and VAE are unquantized, so VRAM savings come only from the transformers.
• Evaluated with no-reference imaging_quality; other VBench dimensions were not swept.

License & attribution

Inherits the base model's Apache-2.0 license. This is a derivative (quantized) checkpoint of Wan-AI/Wan2.2-T2V-A14B-Diffusers; all model credit belongs to the Wan team. Quantization by Intel auto-round (zero-shot RTN, MXFP8) with PR #2481. Please also comply with the upstream Wan2.2 usage terms.