mmBERT Hindi Quality Classifier

A text quality classifier for Hindi pretraining data, trained from mmBERT-small.

This model implements the FineWeb2-HQ approach (Messmer et al., 2025) but uses mmBERT as the encoder for improved Hindi understanding.

Usage

from transformers import pipeline

classifier = pipeline("text-classification", model="AdaMLLab/mmBERT-Hindi-Quality-Classifier")
result = classifier("हिंदी पाठ यहाँ")

Citation

@misc{alrashed2025mixminmatch,
      title={Mix, MinHash, and Match: Cross-Source Agreement for Multilingual Pretraining Datasets}, 
      author={Sultan Alrashed and Francesco Orabona},
      year={2025},
      eprint={2512.18834v2},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2512.18834v2}, 
}