Finetuning

This model is a fine-tuned version of Varsha00/finetuned-opusmt-en-to-ta on the samanantar & WMT News dataset. source group: English target group: Gujarati model: transformer

Model description

This model is a sequentially finetuned version of the Helsinki-NLP/opus-mt-en-mul model, designed for translating between English and Gujarati. The model was initially finetuned on the Tamil language using a substantial dataset and subsequently finetuned on Gujarati using a smaller dataset. This approach, known as sequential finetuning or cascaded finetuning, allows the model to leverage the knowledge gained from Hindi to improve its performance on Gujarati translations, despite the limited data available for the latter.

Training and evaluation data

ai4bharath/samanantar WMT-News

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-5
• warmup_steps: 500
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• num_epochs: 10

Benchamark Evaluation

• BLEU score on Tatoeba: 26.2690989442415
• BLUE score on IN-22: 5.81105590832792

Framework versions

• Transformers 4.42.3
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.19.1