Bozorgmehr PuncFixer v0.1: Persian punctuation restoration

A fine-tuned HooshvareLab/albert-fa-zwnj-base-v2 that restores punctuation in unpunctuated Persian text. It labels each word with the mark that follows it (، . ؟ ! ؛ :) and never rewrites the words.

in : سلام حال شما چطور است
out: سلام، حال شما چطور است؟

Illustrative example. Measured results are in the Evaluation section.

Intended use

• Post-processing of speech-to-text output, OCR text, or any Persian text that lacks punctuation.
• A word-preserving alternative to generative models, which can introduce unwanted edits.

Out of scope: spelling correction, paraphrasing, restoring quotes, brackets, dashes or ellipses, and any use that needs guaranteed-correct punctuation.

How to use

The repository contains two small helper files (punc_core.py, punc_inference.py) that handle word parsing, long-text windowing and output reconstruction. Download the whole repository, then:

pip install torch transformers sentencepiece protobuf safetensors huggingface_hub

from huggingface_hub import snapshot_download
import sys

path = snapshot_download("<your-username>/<your-model-name>")
sys.path.insert(0, path)

from punc_inference import PunctuationRestorer

restorer = PunctuationRestorer(path)
print(restorer.restore("آیا فردا به دانشگاه می‌روی"))
print(restorer.restore(["جمله اول", "جمله دوم"]))   # batch, order preserved

Loading the weights alone with AutoModelForTokenClassification returns per-token logits. Turning them into text correctly needs the word-level alignment in punc_core.py, so use PunctuationRestorer unless you reimplement it.

Behaviour

• Existing punctuation in the input is discarded and re-predicted. Numbers such as 3.5 or 10:30 are kept intact.
• Output words always equal input words, verified on every call.
• Long inputs are processed in overlapping windows, so nothing is truncated.
• Runs on CPU. A GPU is used automatically if available.

Labels

ASCII , and ? are normalised to ، and ؟. Quotes, brackets, dashes and ellipses stay inside their word and are not predicted.

Training data

MohammadJRanjbar/persian-punctuation-restoration: 17.3M deduplicated Persian sentences with at least two target punctuation marks, split 16,952,014 train / 17,432 validation / 327,939 test (as published). The dataset is gated: accept its conditions on Hugging Face to access it.

Inputs and targets were both derived from the punctuated reference sentences: punctuation was stripped to form the input, and each removed mark became the label of the preceding word. Sentences that cannot be represented by one label per word were excluded from training and evaluation: consecutive marks (؟!, ...), leading punctuation, and tokens made only of symbols.

Training scope: subset: 640,000 sentences (656,000 stream rows read, 3.87% of the train split), 20,000 optimizer steps, about 34 minutes on one T4.

Training procedure

• Encoder: pretrained ALBERT plus a new linear classification head (7 classes).
• Optimizer: AdamW, learning rate 3e-5, weight decay 0.01, 6% linear warm-up then linear decay.
• Effective batch size 32 (16 × 2 gradient accumulation), max 256 sub-words, fp16 autocast, gradient clipping 1.0, dynamic padding.
• Each word's label is placed on its last sub-word; all other positions are ignored in the loss.
• Model selection by validation macro F1. The test split was used only for the final evaluation.
• Hardware: one NVIDIA Tesla T4 (Google Colab), about 330 sentences/s training throughput, peak 1.8 GB GPU memory.

Evaluation

Measured on the full published test split (327,939 sentences); 1.11% of sentences were excluded because the label scheme cannot represent them.

Validation split (17,432 sentences, 17,254 usable): macro F1 0.8144, punctuation micro F1 0.8520, accuracy 0.9696, exact match 0.6212. Model selection during training used a 5,000-sentence validation slice (macro F1 0.695), which is not comparable to the full-split figures.

The no-punctuation baseline reaches 0.88 accuracy only because 88% of positions carry no mark; macro F1 and punctuation micro F1 are the informative metrics.

Metric definitions: precision, recall and F1 are per class over valid word positions. Macro F1 averages over classes present in the reference. Insertion precision and recall ignore which mark was chosen. Punctuation error rate is (substitutions + insertions + deletions) divided by the number of reference marks.

No published score is claimed. Numbers from other papers use different data and splits and are not directly comparable.

Error analysis

Based on the test confusion matrix (6,778,371 word positions) and a sample of sentence-level errors:

• Commas are under-predicted. About 27% of reference commas receive no mark (recall 0.72, precision 0.83). Comma placement is often a matter of writing style, so some of these misses may be acceptable alternatives.
• ! is mostly confused with .. 56% of reference exclamation marks are predicted as periods (recall 0.43). Sentence-final . versus ! often depends on tone, which the text alone does not settle.
• ؛ is rare and weak. Recall is 0.47 on 4,966 reference marks, and the model rarely over-predicts it (precision 0.94).
• Periods, question marks and colons are strong (F1 0.97, 0.87, 0.88).
• Dialogue and speaker labels (name: utterance) are a recurring error type, with colon, comma and period swapped.

Examples from the held-out error sample (reference vs. prediction):

Many of these are plausible alternatives rather than clear mistakes, so the metrics under-count acceptable output. Only the lexical-change rate is a hard guarantee.

Limitations and bias

• Trained on written Persian from Wikipedia, blogs, news, Telegram and similar sources filtered to well-punctuated sentences. Spoken-style or very informal text may be punctuated less reliably.
• Punctuation is partly subjective: several placements can be correct, so the metrics under-count valid alternatives.
• Only six marks are predicted; untested on sentences containing consecutive marks or symbol-only tokens.
• Trained on about 3.9% of the available data; the rare classes (!, ؛) are the weakest. Further training may help (not tested).
• Exact-string overlap was found between the splits: 66 sentences appear in both validation and test, and 369 of 100,000 sampled training sentences (0.37%) also occur in validation or test, despite the dataset's published deduplication (whitespace normalisation only is applied in this check). Test scores may be slightly optimistic. Only a bounded sample was checked, and the remainder of the train split was not compared.
• Source datasets carry their own licenses and content biases, which the model may reflect.

License

Apache-2.0 for the code and weights (matching the base model). The training data aggregates several datasets with separate licenses; verify them before commercial use.

Citation

If you use this model, please cite the repository:

@software{bozorgmehr-puncfixer-v0.1,
  title = {Bozorgmehr-puncfixer-v0.1},
  author = {Kheradafrouz},
  url = {https://github.com/kheradafrouz/Bozorgmehr-puncfixer-v0.1},
  license = {apache-2.0}
}

Also:

@inproceedings{kalahroodi-etal-2026-persianpunc,
  title     = {{P}ersian{P}unc: A Large-Scale Dataset and {BERT}-Based Approach for {P}ersian Punctuation Restoration},
  author    = {Kalahroodi, Mohammad Javad Ranjbar and Faili, Heshaam and Shakery, Azadeh},
  booktitle = {Proceedings of the First Workshop on {NLP} and {LLM}s for the {I}ranian Language Family},
  year      = {2026},
  publisher = {Association for Computational Linguistics},
  url       = {https://aclanthology.org/2026.silkroadnlp-1.11/}
}

@misc{ALBERTPersian,
  author = {Hooshvare Team},
  title  = {ALBERT-Persian: A Lite BERT for Self-supervised Learning of Language Representations for the Persian Language},
  year   = {2021},
  howpublished = {\url{https://github.com/m3hrdadfi/albert-persian}}
}