MODA-Fashion-Vision-FP16

Compressed vision-only encoder for fast fashion image-to-image retrieval — 4.2x smaller, same quality.

This is the vision tower extracted from MODA-Fashion-Distilled, converted to FP16 half-precision. It strips the unused text encoder (image-to-image tasks never use it) and halves the weight precision, reducing model size from 775 MB to 186 MB with only -0.21 pp quality loss on LookBench.

Key Numbers

LookBench Results (Fine Recall@1)

Inference — Quick Start

A standalone inference.py is included in this directory.

# Single image → 768-d embedding
python inference.py --image query.jpg

# Two images → embeddings + cosine similarity
python inference.py --image img1.jpg img2.jpg --similarity

# Run on GPU (keeps FP16 precision for speed)
python inference.py --image query.jpg --device cuda

Python API

import torch
import open_clip
import torch.nn.functional as F
from safetensors.torch import load_file
from PIL import Image

# Build the ViT-B-16-SigLIP architecture without downloading any pretrained weights.
# The text tower is randomly initialized (we never use it). Only the visual tower
# is overwritten with MODA's fine-tuned weights below. Suppresses the ~775 MB
# pretrained-checkpoint download that would otherwise come with hf-hub:Marqo/marqo-fashionSigLIP.
base_model, _, preprocess = open_clip.create_model_and_transforms(
    "ViT-B-16-SigLIP", pretrained=None
)

# Load MODA's vision-only fp16 weights (186 MB) and overlay onto the visual tower.
vision_sd = load_file("path/to/moda-fashion-vision-fp16/vision_encoder.safetensors")
vision_sd_fp32 = {k: v.float() for k, v in vision_sd.items()}
full_sd = base_model.state_dict()
for k, v in vision_sd_fp32.items():
    full_sd[k] = v
base_model.load_state_dict(full_sd, strict=True)

encoder = base_model.visual
encoder.eval()

image = preprocess(Image.open("fashion_item.jpg")).unsqueeze(0)
with torch.no_grad():
    features = encoder(image)
    features = F.normalize(features, p=2, dim=-1)  # [1, 768]

Requirements

open_clip_torch>=2.20.0
torch>=2.0
Pillow
safetensors

Why Vision-Only?

For image-to-image retrieval (the primary use case), the text encoder is never used. Stripping it provides:

• 54% fewer parameters (92.9M vs 203M) — faster forward pass
• 4.2x smaller on disk (186 MB vs 775 MB) — cheaper to deploy
• Near-zero quality loss (-0.21 pp Fine R@1) — within noise margin

Training

This model inherits its weights from MODA-Fashion-Distilled, which was trained via:

1. Knowledge distillation from a 10-model ensemble (SigLIP, CLIP, EVA-CLIP, MetaCLIP, DFN variants)
2. Stratified contrastive learning on Google Shopping data (10K image pairs, category-balanced)
3. Vision-only export — text tower removed
4. FP16 conversion — weights cast from float32 to float16

Related Models

License

MIT

Citation

If you use this model, please cite:

@software{moda2026,
  title  = {MODA: Open-source benchmark and models for fashion search},
  author = {Hopit AI},
  year   = {2026},
  url    = {https://github.com/hopit-ai/Moda}
}