Mind the Tail: Released Artifacts

This repository contains the artifacts of Mind the Tail: Data Selection Through Sparse Concepts: the sparse autoencoder (SAE) used to extract concepts, the selection rankings of every training pool, and the models trained on the selected subsets.

Paper | Project Page | Code

Structure

├── ckpts/
│   ├── sae/dinov3_b_layer11_in1k_topk32/
│   │   └── sae.pt                              # TopK SAE, 16,384 concepts
│   └── downstream/
│       ├── nabirds/<ratio>pct/best.pt          # ResNet-50
│       ├── stanford_cars/<ratio>pct/best.pt    # ResNet-50
│       ├── waterbirds/<ratio>pct/best.pt       # ResNet-50
│       ├── spawrious/<ratio>pct/best.pt        # ResNet-50
│       └── cc12m/<ratio>pct/clip.pt            # CLIP ViT-B/16
└── corsets/
    ├── nabirds/ranked_examples.parquet
    ├── stanford_cars/ranked_examples.parquet
    ├── waterbirds/ranked_examples.parquet
    ├── spawrious/ranked_examples.parquet
    └── cc12m/ranked_examples.parquet

<ratio> is one of 10, 20, 30, 50, and 70.

Sparse Autoencoder

ckpts/sae/dinov3_b_layer11_in1k_topk32/sae.pt is the single SAE used for every dataset: a TopK SAE (k = 32 active concepts per token, 16,384 concepts) trained on 100M layer-11 patch tokens of a frozen DINOv3 ViT-B/16 from ImageNet-1K train. Image-level concept vectors are obtained by max-pooling the patch activations.

The file starts with one JSON line holding the architecture (d_model, d_sae, top_k), followed by a PyTorch state dict with W_enc (768 × 16,384), b_enc, W_dec (16,384 × 768), and b_dec.

Selection Rankings

Each ranked_examples.parquet holds the full ordering of a training pool. A subset at ratio r is the top of this ordering.

Within-class rankings (labelled datasets) have the columns rank_within_class, class_candidate_count, image_path, class_idx, and class_name. The subset at ratio r keeps the top max(1, floor(r · class_candidate_count)) images of every class.

Global ranking (CC12M) has the columns rank_global and image_path. The subset at ratio r keeps the top floor(r · N) images. Paths have the form tar://${DATA_ROOT}/cc12m-wds/<shard>.tar::<key>.jpg, referring to the image of sample <key> in a CC12M WebDataset shard.

All paths are written relative to ${DATA_ROOT}, the folder holding the original datasets, so they resolve against any dataset location.

Models

ResNet-50 (best.pt): ImageNet-pretrained ResNet-50 fine-tuned on the subset of each dataset and ratio (18,700 steps, batch size 128, SGD, lr 1e-3). Each file holds only the weights as {"model_state_dict": ...}. The output classes follow class_idx of the corresponding ranking file.

CLIP ViT-B/16 (clip.pt): trained from scratch on the CC12M subset at each ratio. Each file is a plain state dict of the OpenCLIP ViT-B-16 model.

Datasets

The images themselves are not included; they are available from their original sources: NABirds, Stanford Cars, Waterbirds (waterbird_complete95_forest2water2), Spawrious (one-to-one hard), and CC12M.

Citation

@article{zhang2026mindthetail,
  title   = {Mind the Tail: Data Selection Through Sparse Concepts},
  author  = {Zhang, Ziheng and Hulkund, Neha and Lee, Colin and Mai, Zheda and
             Shen, Hui and Ding, Ying and Berger-Wolf, Tanya and Beery, Sara and
             Chao, Wei-Lun and Gu, Jianyang},
  journal = {arXiv preprint arXiv:2610.21007},
  year    = {2026}
}

License

The artifacts in this repository are released under the MIT License.