intel-ai/Wan2.2-T2V-A14B-Diffusers-NVFP4-AutoRound

Model Details

This model is an NVFP4 W4A4 quantized version of Wan-AI/Wan2.2-T2V-A14B-Diffusers, generated by Intel AutoRound using RTN. Please follow the license of the original model.

• Quantized: both diffusion transformer experts (transformer and transformer_2), with 400 quantized linear layers each.
• Format: FP4 E2M1 weights and activations, group size 16, FP8 E4M3 block scales, and FP32 global scales.
• Activation scaling: dynamic block scales with calibrated static global scales.
• Not quantized to NVFP4: text encoder, VAE, embeddings, norms, and excluded projections.

Quantization uses plain RTN with no rounding optimization. OpenS2V calibration is still required to obtain the static global activation scales.

How to Use

Environment

The evaluated setup used NVIDIA B200, Python 3.12.3, PyTorch 2.14.0, vLLM 0.33.0, FlashInfer 0.7.2.post1, and CUDA toolkit 12.8.93, with official vLLM-Omni commit 7465c0f52. Use a compatible environment with support for AutoRound NVFP4 checkpoints.

Inference used TRTLLM_ATTN attention and FlashInfer CUTLASS NVFP4 GEMMs, not an activation quantize-dequantize emulation path. Keep the exported AutoRound quantization metadata unchanged.

vLLM-Omni Usage

DIFFUSION_ATTENTION_BACKEND=TRTLLM_ATTN \
vllm serve intel-ai/Wan2.2-T2V-A14B-Diffusers-NVFP4-AutoRound --omni

Use the /v1/videos endpoint for video generation. The command above is a serving example; the reported evaluation used the Omni Python API with the settings below.

Text-to-Video Example

From the vLLM-Omni checkout above, generate a 720p video with the official offline example:

CUDA_VISIBLE_DEVICES=0 DIFFUSION_ATTENTION_BACKEND=TRTLLM_ATTN \
python examples/offline_inference/text_to_video/text_to_video.py \
  --model intel-ai/Wan2.2-T2V-A14B-Diffusers-NVFP4-AutoRound \
  --model-class-name WanPipeline \
  --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage." \
  --negative-prompt "色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走" \
  --height 720 --width 1280 --num-frames 81 --fps 16 \
  --num-inference-steps 40 \
  --guidance-scale 4.0 --guidance-scale-high 3.0 \
  --boundary-ratio 0.875 --flow-shift 5.0 \
  --seed 42 --enforce-eager \
  --output wan22_nvfp4.mp4

Evaluation

Video Generation Comparison

Prompt: "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage." — 1280 × 720, 81 frames, 40 steps, 16 fps, seed 42, CFG 4.0/3.0. Both videos use the same negative prompt and generation settings.

VBench Results

Evaluated with VBench 0.1.5 on the same 165 prompts for BF16 and NVFP4: 72 from subject_consistency and 93 from overall_consistency. Each metric was computed over all 165 videos of each model, not averaged from separate group scores.

The flow shift was explicitly set to 5.0 for both models, rather than the local scheduler default of 3.0. Evaluation used custom_input, local=True, read_frame=False, and imaging_quality_preprocessing_mode="longer".

Scores are fractions; higher is better. Recovery is NVFP4 divided by BF16.

The mean absolute relative change across these five metrics is 0.65%. This is a one-video-per-prompt subset evaluation, not the standard five-sample full VBench protocol; no VBench total score is reported.

Generate the Model

The checkpoint was exported with AutoRound 0.17.0 using the following settings:

auto-round \
  --model Wan-AI/Wan2.2-T2V-A14B-Diffusers \
  --scheme NVFP4 \
  --algorithm rtn \
  --disable_opt_rtn \
  --iters 0 \
  --dataset opens2v \
  --nsamples 32 \
  --batch_size 1 \
  --calib_num_inference_steps 8 \
  --num_inference_steps 40 \
  --guidance_scale 5.0 \
  --generator_seed 42 \
  --seed 42 \
  --device_map 0 \
  --low_gpu_mem_usage \
  --format auto_round \
  --output_dir Wan2.2-T2V-A14B-Diffusers-NVFP4-AutoRound

Ethical Considerations and Limitations

This model inherits the capabilities and limitations of the base model. Generated videos may contain artifacts, inaccurate content, or bias. Perform appropriate quality and safety testing before deployment. The reported results cover a limited prompt subset and a single seed; small score increases do not establish a general quality improvement.

Caveats and Recommendations

• Intel Neural Compressor
• AutoRound
• vLLM-Omni

Disclaimer

The license on this model does not constitute legal advice. Users are responsible for complying with the original model's license and applicable deployment requirements.

Cite

@article{cheng2023optimize,
  title={Optimize weight rounding via signed gradient descent for the quantization of llms},
  author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao and Liu, Yi},
  journal={arXiv preprint arXiv:2309.05516},
  year={2023}
}