MODA-Fashion-CrossDomain

Fashion image-to-image retrieval fine-tuned on cross-domain shop↔consumer pairs.

MODA-Fashion-CrossDomain is a vision-encoder fine-tuned ViT-B-16-SigLIP that achieves 66.52% Fine Recall@1 on LookBench, beating FashionSigLIP by +2.68 with just 13.5K training triplets and no distillation.

Highlights

• +2.68 Fine R@1 over FashionSigLIP on LookBench Overall
• +9.37 on AIGen-StreetLook — the hardest cross-domain subset
• Trained on only 13,557 cross-domain triplets (no LookBench data)
• Same architecture as FashionSigLIP — drop-in replacement
• No ensemble or distillation needed

LookBench Results

Per-subset Fine Recall@1

Model Spec

Inference — Quick Start

A standalone inference.py is included in this directory.

# Single image → 768-d embedding
python inference.py --image query.jpg

# Two images → embeddings + cosine similarity
python inference.py --image img1.jpg img2.jpg --similarity

# Run on GPU/MPS
python inference.py --image query.jpg --device cuda

Python API

import open_clip
import torch
import torch.nn.functional as F
from PIL import Image

model, _, preprocess = open_clip.create_model_and_transforms(
    "ViT-B-16-SigLIP",
    pretrained="path/to/moda-fashion-crossdomain/open_clip_model.safetensors",
)
model.eval()

image = preprocess(Image.open("query.jpg")).unsqueeze(0)
with torch.no_grad():
    features = model.encode_image(image)
    features = F.normalize(features, p=2, dim=-1)  # [1, 768]

Requirements

open_clip_torch>=2.20.0
torch>=2.0
Pillow
safetensors

Training Details

• Base model: Marqo-FashionSigLIP (ViT-B-16-SigLIP, webli pretrained)
• Fine-tuned components: Vision encoder only (image tower)
• Training data: cross-domain shop↔consumer image pairs (see the corpus specification in the repo's REPRODUCE.md)
• Triplets: 13,557 train + 714 validation
• Loss: InfoNCE + L2 weight drift regularization
• Temperature: 0.07
• Alignment weight: 0.3
• Optimizer: AdamW, LR=2e-6, batch=24
• Epochs: 4 (best at epoch 3, val triplet accuracy = 99.6%)
• BBox cropping: Uses per-garment bounding box annotations for item-level crops
• Hardware: Apple M-series (MPS)

Why It Works

The key insight is cross-domain contrastive learning. The training corpus contains pairs of the same product photographed in two very different conditions:

• Shop images: Clean studio photos (white background, centered)
• Consumer images: In-the-wild photos (varied backgrounds, angles, lighting)

Training the vision encoder to match these pairs teaches the model to look past domain differences and focus on the product's intrinsic visual features — exactly what LookBench tests.

Related Models

License

MIT

Citation

If you use this model, please cite:

@software{moda2026,
  title  = {MODA: Open-source benchmark and models for fashion search},
  author = {Hopit AI},
  year   = {2026},
  url    = {https://github.com/hopit-ai/Moda}
}