Model description
Model Name: estonian-large-handwritten
Model Version: estonian-large-november2026
Model Type: Transformer-based encoder-decoder for OCR
Base Model: microsoft/trocr-large-handwritten
Purpose: Handwritten text recognition
Languages: Estonian
License: Apache 2.0
This is a fine-tuned model for Estonian handwriting recognition, trained with computing resources generously provided by CSC – IT Center for Science on the Puhti and LUMI supercomputers.
This model was developed in the ArchXAI project funded by the Central Baltic Programme.
Model Architecture
The model is based on a Transformer architecture with an encoder-decoder setup, similar to TrOCR from Li et. al. (2023):
- The encoder processes an image of a single line of text into a sequence of hidden states.
- The decoder attends to the hidden states from the encoder using cross-attention, to generate the corresponding text output.
This model is a fine-tuned version of the original trocr-large-handwritten from Li et. al. (2023), for handwritten text recognition in primarily Estonian-language historical documents.
Intended Use
- Document digitization (e.g., archival work, historical manuscripts)
- Handwritten notes transcription
- Data rescue (handwritten or typed numerical and letter codes)
Training data
The training data consists of human-annotated samples of mainly handwritten text lines from historical documents in the collections of the National Archives of Estonia, with some printed and typed text lines included. The training set and validation set (but not the test set) were expanded with empty lines containing no text, from the collections of the National Archives of Finland. The training set was also expanded with additional mainly numerical table cells as explained below.
Training set: 231 086 text lines (including 7106 empty lines)
Validation set: 29 412 text lines (including 888 empty lines)
Test set: 30 409 text lines
UoS Data Rescue: Training set (above) expanded by 6.66% with table cells sampled randomly (fresh samples per epoch) from UoS Data Rescue by Middleton and Singh (2025), a set of 768 074 table cells including some text cells in other languages such as English and French.
Randomly generated numerical table cells: Training set (above) expanded by 3.33% with digit strings randomly generated (per epoch) with our generator by concatenating individual digits from NIST SD19 (Grother and Hanaoka, 2016) and DIDA (Kusetogullari et. al., 2021).
Evaluation
The following metrics were calculated on the test set (in-domain evaluation) using the evaluate library with default settings:
CER (character error rate): 0.0271
WER (word error rate): 0.1257
Used Hyperparameters
Train batch size per device: 8
Number of devices: 128
Learning rate: 5e-5
Scheduler: linear
Optimizer: AdamW
Number of epochs: 400
FP16 mixed precision training: False
Input image size: 192 x 1024
How to Use the Model
For production inference, use our nafrec package. You can also use the model for inference by loading the processor and model manually.
from transformers.models.vit.modeling_vit import ViTPatchEmbeddings, ViTEmbeddings
from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image
def load_custom_trocr_model():
"""Load a TrOCR model with custom image size support"""
original_embeddings_forward = ViTEmbeddings.forward
# Always apply patches for models saved with custom image sizes
def universal_patch_forward(self, *args, **kwargs):
pixel_values = args[0] if args else kwargs['pixel_values']
embeddings = self.projection(pixel_values).flatten(2).transpose(1, 2)
return embeddings
def universal_embeddings_forward(self, *args, **kwargs):
kwargs['interpolate_pos_encoding'] = True
return original_embeddings_forward(self, *args, **kwargs)
# Apply patches
ViTPatchEmbeddings.forward = universal_patch_forward
ViTEmbeddings.forward = universal_embeddings_forward
# Load model and processor
processor = TrOCRProcessor.from_pretrained("Kansallisarkisto/estonian-large-handwritten",
use_fast=True,
do_resize=True,
size={'height': 192,'width': 1024})
model = VisionEncoderDecoderModel.from_pretrained("Kansallisarkisto/estonian-large-handwritten")
return processor, model
# Load model and processor
processor, model = load_custom_trocr_model()
# Open an image of handwritten text
image = Image.open("path_to_image.jpg")
# Preprocess and predict
pixel_values = processor(image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(generated_text)
Limitations and Biases
The model was trained primarily on handwritten text that uses basic Latin characters and Estonian special characters. It has not been trained on non-Latin alphabets, such as Chinese characters or other writing systems like Arabic or Hebrew. The model may not generalize well to any other language than Estonian.
Future Work
Potential improvements for this model include:
- Expanding training data: incorporating more ground truth data
- Optimizing for specific domains: fine-tuning the model on domain-specific handwriting
- Pretraining: pre-training a fully Estonian-specific model instead of starting the fine-tuning from a model trained on English
- Out-of-domain generalization: studying how pre-training and fine-tuning could be optimized to maximize out-of-domain generalization of the fine-tuned model
Citation
If you use this model in your work, please cite it as:
@misc{estonian-large-handwritten,
author = {Kansallisarkisto},
title = {NAF Estonian HTR model},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/Kansallisarkisto/estonian-large-handwritten/}},
}
References
P. Grother and K. Hanaoka, NIST special database 19 handprinted forms and characters 2nd Edition, National Institute of Standards and Technology, Tech. Rep., 2016. DOI: http://doi.org/10.18434/T4H01C
Huseyin Kusetogullari, Amir Yavariabdi, Johan Hall, Niklas Lavesson, “DIGITNET: A Deep Handwritten Digit Detection and Recognition Method Using a New Historical Handwritten Digit Dataset”, Big Data Research, 2020, DOI: http://doi.org/10.1016/j.bdr.2020.100182.
Huseyin Kusetogullari, Amir Yavariabdi, Johan Hall, Niklas Lavesson, DIDA: The largest historical handwritten digit dataset with 250k digits, June 2021. Accessed on: June 13, 2021. Available: https://github.com/didadataset/DIDA/.
Li, M., Lv, T., Chen, J., Cui, L., Lu, Y., Florencio, D., Zhang, C., Li, Z. and Wei, F. 2023. TrOCR: Transformer-Based Optical Character Recognition with Pre-trained Models. Proceedings of the AAAI Conference on Artificial Intelligence. 37, 11 (Jun. 2023), 13094-13102. DOI:https://doi.org/10.1609/aaai.v37i11.26538.
Middleton, S. E., & Singh, L. G. (2025). UoS Data Rescue [Dataset]. In International Journal on Document Analysis and Recognition (Version 1.0). Zenodo. https://doi.org/10.5281/zenodo.15730546
Model Card Authors
Author: Kansallisarkisto
Contact Information: john.makela@kansallisarkisto.fi, ilkka.jokipii@kansallisarkisto.fi