IndicTrans2 Indic→English — multilingual fine-tune (Tamil, Hindi, Telugu, Kannada, Malayalam, Bengali)
Fine-tuned from ai4bharat/indictrans2-indic-en-dist-200M on quality-filtered
Samanantar pairs, with all languages mixed in every batch.
Final weights are a WiSE-FT blend of pretrained and fine-tuned weights (alpha = 0.25), chosen on validation data.
Training setup
- Languages: Tamil, Hindi, Telugu, Kannada, Malayalam, Bengali (→ English)
- Examples per language: up to 6000, epochs: 2, LR: 1e-05, label smoothing: 0.1
- Cosine schedule with warm-up, weight decay 0.01, fp16 mixed precision
Results (average over languages; identical decoding, beam=5, fp16)
| set | model | BLEU | chrF++ | COMET |
|---|---|---|---|---|
| FLORES devtest | Fine-tuned | 37.9426 | 62.006 | 0.8817 |
| FLORES devtest | Pretrained | 37.4825 | 61.7819 | 0.8738 |
| Samanantar held-out | Fine-tuned | 27.2057 | 51.2517 | 0.8508 |
| Samanantar held-out | Pretrained | 25.834 | 50.5189 | 0.8412 |
Test sets: FLORES-200 devtest (clean, human translated) and a held-out Samanantar sample. Note: the base model was trained on Samanantar-style data, so FLORES is the more trustworthy comparison.
Usage
import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from IndicTransToolkit.processor import IndicProcessor # pip install IndicTransToolkit
repo = "vijaysarathy/indictrans2-indic-en-multilingual-ft"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained(repo, trust_remote_code=True, dtype=torch.float16, low_cpu_mem_usage=False).to("cuda").eval()
ip = IndicProcessor(inference=True)
src_lang, tgt_lang = "tam_Taml", "eng_Latn" # hin_Deva, tel_Telu, kan_Knda, mal_Mlym, ben_Beng ...
sents = ["நான் சென்னையில் வசிக்கிறேன்."]
batch = ip.preprocess_batch(sents, src_lang=src_lang, tgt_lang=tgt_lang)
inputs = tok(batch, padding="longest", truncation=True, max_length=256, return_tensors="pt").to("cuda")
with torch.no_grad():
out = model.generate(**inputs, max_length=256, num_beams=5, use_cache=False) # use_cache=False avoids a cache bug with new transformers
text = tok.batch_decode(out.cpu().tolist(), skip_special_tokens=True)
print(ip.postprocess_batch(text, lang=tgt_lang))
Tested with transformers==4.57.1.