ModernBERT-TR MASSIVE Intent

A 150M-parameter Turkish classifier for the 60 intent labels in MASSIVE 1.1.

Results

Our model scores 88.21 +/- 0.16% accuracy and 85.43 +/- 0.14% macro-F1 on the MASSIVE 1.1 tr-TR test set. The released checkpoint scores 88.33% accuracy and 85.69% macro-F1.

The quantized int8 version scores 88.33% accuracy and 85.92% macro-F1.

Usage

from transformers import pipeline

classify = pipeline(
    "text-classification",
    model="ytu-ce-cosmos/modernbert-tr-massive-intent",
)
classify("yarın sabah yedi için alarm kur")

Training

We finetune ytu-ce-cosmos/modernbert-tr-base jointly with a mean-pooled 60-class intent head and a 111-class slot head; this repository contains the exported intent head. The human-localized MASSIVE 1.1 Turkish split has 11,514 training, 2,033 validation, and 2,974 test utterances. We use encoder learning rate 5e-5, head learning rate 1e-4, batch size 64, 15 epochs, linear warmup and decay, weight decay 0.01, intent label smoothing 0.1, and five seeds.

Standalone binaries are available in the encoderfile repo.

License

Apache-2.0. The MASSIVE dataset is distributed under CC BY 4.0.