TrOCR-Sin-Printed

A Sinhala text-line recognizer for printed text, released with the SinFUND and SinOCR datasets (SriDoc/datasets). For handwriting, use SriDoc/TrOCR-Sin-Handwritten.

Model

• TrOCR-style vision encoder-decoder (VisionEncoderDecoderModel).
• The encoder was initialized from facebook/deit-base-distilled-patch16-224 (ImageNet weights). The decoder was initialized from NLPC-UOM/SinBERT-large, a RoBERTa model pre-trained on Sinhala text. The cross-attention layers started from random weights.
• No OCR-pretrained TrOCR weights were used.
• Input: one pre-cropped text-line image, resized to 224 x 224.
• Decoding: beam search with 4 beams and a maximum length of 64 tokens.

Training data

The SinOCR printed training split: 90,000 synthetic text-line images of CC100 Sinhala text, rendered in 200 Sinhala fonts. Training ran for 75,000 steps (about 27 epochs), with a batch size of 32 and a learning rate of 5e-5.

Results

Character error rate (CER; lower is better) on the SinOCR test splits:

How the CER is computed:

• Characters are Unicode code points.
• Runs of spaces are collapsed, and leading and trailing spaces are removed.
• The CER is pooled over the whole test split, not averaged per line.

SinOCR has no separate validation split, so the test split was also the evaluation set during training. The lowest test CER occurred at the final checkpoint, so checkpoint selection did not change this result.

Limitations

• The model reads single, pre-cropped lines. It does not find text on a page.
• It was trained on synthetic printed text only.

Usage

from transformers import VisionEncoderDecoderModel, TrOCRProcessor, AutoTokenizer, ViTImageProcessor
from PIL import Image

model = VisionEncoderDecoderModel.from_pretrained("SriDoc/TrOCR-Sin-Printed")
tokenizer = AutoTokenizer.from_pretrained("NLPC-UOM/SinBERT-large")
image_processor = ViTImageProcessor.from_pretrained("google/vit-base-patch16-224")
processor = TrOCRProcessor(image_processor=image_processor, tokenizer=tokenizer)

image = Image.open("line.png").convert("RGB")   # one cropped text line
pixel_values = processor(image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values)
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])

Citation

@article{gunathilaka2025sinfund,
  title   = {SinFUND and SinOCR: Benchmarks for Sinhala Handwritten OCR and Template-Free Form Understanding},
  author  = {Gunathilaka, Kavishka and Hewagama, Danusha and Pushpakumara, Supul and Ambegoda, Thanuja D.},
  journal = {Research Square preprint},
  year    = {2025},
  doi     = {10.21203/rs.3.rs-6976719/v1}
}

Training log

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 75000
• mixed_precision_training: Native AMP

Training results

The "Cer" and "Validation Loss" columns were measured on the SinOCR printed test split.

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.1