CLIP ViT-L/14 W4A8 ModelOpt Quantization
This is a NVIDIA Model Optimizer W4A8 post-training quantized checkpoint of:
laion/CLIP-ViT-L-14-laion2B-s32B-b82K
The checkpoint was produced with NVIDIA ModelOpt using W4A8/AWQ-style post-training quantization. Patch embedding quantization was disabled because it gave better retrieval quality in this experiment.
Evaluation
Evaluation was run on COCO val retrieval with 5,000 images.
| Metric | W4A8 | Delta vs FP16 |
|---|---|---|
| image-to-text R@1 | 0.6168 | -0.0078 |
| image-to-text R@5 | 0.8372 | -0.0048 |
| image-to-text R@10 | 0.8992 | -0.0068 |
| text-to-image R@1 | 0.4417 | -0.0173 |
| text-to-image R@5 | 0.6959 | -0.0108 |
| text-to-image R@10 | 0.7867 | -0.0060 |
Notes
This is a ModelOpt checkpoint and includes modelopt_state.pth.
To restore it correctly, use NVIDIA ModelOpt with Hugging Face checkpointing enabled.
This experiment measures retrieval quality, not production latency. For real acceleration claims, benchmark an optimized deployment runtime separately.
Base model
Base model: laion/CLIP-ViT-L-14-laion2B-s32B-b82K