trspell-t5-efficient-small

A Turkish T5 model fine-tuned for Turkish spelling correction on NoisyWikiTr. It takes a sentence with spelling errors and generates the corrected sentence.

This is the T5-efficient-TRsmall model from Leveraging Large Language Models for Spelling Correction in Turkish (Ceren Güzel Turhan, PeerJ Computer Science 11:e2889, 2025), the best model in the paper on realistic noise.

• Paper: https://doi.org/10.7717/peerj-cs.2889
• Code: https://github.com/cgturhan/trspell
• Dataset: https://huggingface.co/datasets/cguzel/NoisyWikiTr

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

repo = "cguzel/trspell-t5-efficient-small"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSeq2SeqLM.from_pretrained(repo)

text = "netekim kasım ve temmuz felaketleri bunun sadece acı iki örneğidir"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)
output = model.generate(**inputs, num_beams=5, max_length=256, early_stopping=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Input should be normalised like the training data: lowercase with Turkish rules (I → ı, İ → i), no digits, punctuation or apostrophes. No task prefix is used.

Model details

• Base model: Turkish-NLP/t5-efficient-small-turkish, a T5-efficient model pre-trained on Turkish
• Architecture: encoder-decoder, 32 encoder layers and 6 decoder layers, d_model 512, about 142M parameters
• Tokenizer: Turkish SentencePiece vocabulary (32k), which covers all Turkish letters (ı, ğ, ş, ç, ö, ü, İ)
• Task: sequence-to-sequence, noisy sentence → clean sentence

Training

• Data: NoisyWikiTr training split, 687k Turkish Wikipedia sentences with about 10% of tokens corrupted (deletion, insertion, transposition, substitution, vowel errors, common misspellings, word splits)
• Validation: NoisyWikiTr validation split (85k sentences)

Evaluation

NoisyWikiTr test set, accuracy / word correction rate (Table 5 of the paper):

On the realistic test set with non-Turkish sentences removed, this model reaches 0.9664 accuracy, 0.8142 word correction and 0.9925 Levenshtein similarity.

Limitations

• Trained and evaluated on synthetic errors; results on real typos may differ.
• Deletion errors are the hardest type for the model.
• Corrects spelling only, not grammar.
• Long inputs are truncated at 256 tokens; split long texts into sentences.

Citation

@article{guzelturhan2025trspell,
  title   = {Leveraging large language models for spelling correction in Turkish},
  author  = {G{\"u}zel Turhan, Ceren},
  journal = {PeerJ Computer Science},
  volume  = {11},
  pages   = {e2889},
  year    = {2025}
}