LiLT-RE (Sinhala form understanding: relation extraction)

This model links each answer in a Sinhala form to its question. It was released with the SinFUND dataset (SriDoc/datasets).

Model

• LiLT with an XLM-R base text encoder, plus a biaffine relation-extraction head (LiLTRobertaLikeForRelationExtraction).
• This head comes from the LiLT reference implementation. It is not part of the transformers library, so loading the model requires that repository's code.
• Input: the words, word boxes and entities of a form (spans labelled QUESTION or ANSWER).
• The model scores every question-answer pair within each 512-token segment.

Training

Sequential cross-lingual fine-tuning in two stages:

1. On the Italian part of XFUND (kavg/LiLT-RE-IT).
2. On the SinFUND training split (80 forms) for 10,000 steps, with a batch size of 8, a learning rate of 1e-5 and 10% warm-up.

Results

Results on the SinFUND test split (396 question-answer links). The input was ground-truth words, boxes and entities. A predicted link counts as correct only if it connects exactly the annotated question and answer. Scores are micro-averaged over all links:

Important: SinFUND has no separate validation split. The "validation" scores in the training log below were measured on the SinFUND test split, and the released checkpoint was chosen by the test loss.

Limitations

• The scores assume perfect entities. With entities predicted by SriDoc/LiLT-SER, or with OCR output, accuracy will be lower.
• Precision is much lower than recall, so the model predicts many incorrect links.

Citation

@article{gunathilaka2025sinfund,
  title   = {SinFUND and SinOCR: Benchmarks for Sinhala Handwritten OCR and Template-Free Form Understanding},
  author  = {Gunathilaka, Kavishka and Hewagama, Danusha and Pushpakumara, Supul and Ambegoda, Thanuja D.},
  journal = {Research Square preprint},
  year    = {2025},
  doi     = {10.21203/rs.3.rs-6976719/v1}
}

Training log

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 10000

Training results

The "Validation" columns were measured on the SinFUND test split (see above). From step 8,500 onwards the training was resumed, and the numbers in each row are in the order Precision, Recall, F1, Validation Loss, not in the order of the column headings.

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.1