mBERT — Armenian Participle-Clause Punctuation
Fine-tuned bert-base-multilingual-cased that restores punctuation around
Eastern Armenian participle clauses, framed as 4-class token labeling.
From the CODASSCA 2026 paper Sequence Labeling for Low-Resource Syntax:
Automatic Punctuation of Armenian Participle Clauses.
Labels
0 O · 1 COMMA_AFTER · 2 BUTH_AFTER · 3 REMOVE_COMMA
Results (macro-F1)
| Benchmark | macro-F1 |
|---|---|
| Gold 2K (noisy web text) | 0.4655 |
| Shtemaran 292 (clean textbook) | 0.5190 |
Training: 15% negative under-sampling of the majority O class; lower 6 layers
frozen; hyper-parameters tuned with Optuna (21 trials). See artifacts/ for the
raw checkpoint, best config, trial log and final results; training/ holds the notebook.
Usage
from transformers import AutoTokenizer, AutoModelForTokenClassification
tok = AutoTokenizer.from_pretrained("AlbertHakobyan/mbert-armenian-participle-punct")
model = AutoModelForTokenClassification.from_pretrained("AlbertHakobyan/mbert-armenian-participle-punct")
Ensemble
Best results come from a soft-vote of this model with the BiLSTM at alpha=0.45:
P = 0.45 * P_bilstm + 0.55 * P_mbert (macro-F1 0.6745 on Shtemaran). See the
bilstm-armenian-participle-punct repo.