中文Sign up

sukhrobnurali/modernbert-base-banking77

🤗 Hugging Face sourcetext-classificationapache-2.0150M params599 MBsafetensors✓ 2 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo sukhrobnurali/modernbert-base-banking77 ./model-folder
Needs a seeder →

modernbert-base-banking77

A fine-tune of answerdotai/ModernBERT-base for 77-way English banking-intent classification, trained on PolyAI/banking77.

  • Base model: answerdotai/ModernBERT-base (149M params, Apache-2.0)
  • Task: single-label intent detection over 77 fine-grained banking intents
  • Held-out test: the official Banking77 test split (3,080 queries, 40/intent), disjoint from training
  • Result: 0.9399 accuracy / 0.9401 macro-F1 on the test split
  • Training code: https://github.com/sukhrobnurali/banking77-modernbert

Intended use

Routing short English banking/customer-support queries to one of 77 intents (e.g. card_arrival, lost_or_stolen_card, exchange_rate) for task-oriented dialog and triage.

Out of scope

  • Non-English text and non-banking domains (single-domain training data).
  • Out-of-scope / rejection detection: every input is forced into one of 77 intents.
  • Long documents — trained at max_length=64 on short single-sentence queries.

Evaluation

The same protocol is applied to every row of the table, all on the untouched test split. Two baselines contextualize the fine-tune's gain: a majority-class floor, and a frozen-encoder linear probe (mean-pooled base-model embeddings → logistic regression).

Model Accuracy Macro-F1 Weighted-F1
Majority class 0.0130 0.0003 0.0003
Frozen ModernBERT + linear probe 0.8945 0.8948 0.8948
This model (fine-tuned) 0.9399 0.9401 0.9401

ModernBERT's pretrained representations are already strong on this task — a linear probe on frozen embeddings reaches 0.8948 macro-F1 — so fine-tuning adds +4.5 points of macro-F1 (0.8948 → 0.9401) on top of that, well clear of the 0.0003 majority-class floor.

At ~0.94 accuracy the residual errors stay concentrated among semantically adjacent intents — the closely related transfer / top-up families and look-alike card intents. Full per-class breakdown: results/classification_report.txt; 77×77 confusion matrix: results/confusion_matrix.png.

Usage

from transformers import pipeline

clf = pipeline("text-classification", model="sukhrobnurali/modernbert-base-banking77")
clf("My card still hasn't arrived, when will I get it?")
# -> [{'label': 'card_arrival', 'score': 0.99}]

Training data

PolyAI/banking77 (Casanueva et al. 2020, arXiv:2003.04807): 10,003 train / 3,080 test online-banking queries over 77 intents. License CC-BY-4.0. A stratified 10% of train was held out as a validation set for early stopping; the official test split was used only for the final numbers above.

Reproducibility

Fixed seed=42. Fine-tuned with the HF Trainer, lr=5e-5, up to 4 epochs with early stopping (patience 2) on validation macro-F1, max_length=64, batch size 32, warmup ratio 0.1, weight decay 0.01. Pinned versions in requirements.txt (transformers 5.10.1). eval.py reproduces the test numbers from the published model.

License

Apache-2.0, inherited from the base model answerdotai/ModernBERT-base. Training data PolyAI/banking77 is CC-BY-4.0 (attribution: Casanueva et al. 2020).

Citation

@misc{nurali_modernbert_banking77_2026,
  author       = {Sukhrob Nurali},
  title        = {modernbert-base-banking77: a ModernBERT intent classifier for Banking77},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/sukhrobnurali/modernbert-base-banking77}}
}

Author