A CTranslate2 int8 conversion of ai4bharat/indictrans2-en-indic-dist-200M (MIT, AI4Bharat), for the bhashantar package: English to 22 Indian languages on CPU, no torch, no fairseq.
Source weights: pytorch_model.bin, sha256 52671a26e4e5f06a86b2f890dea7163db28c01fc1449a3da80d7cc990c41fef0 (see provenance.json). AI4Bharat's MIT LICENSE is included unchanged.
FLORES-200 devtest, first 100 sentences, beam 5. Reference: the upstream model in fp32 (transformers 4.50.3 + IndicTransToolkit 1.1.1). Pass bar fixed before running: |delta| <= 1.0 chrF.
| pair | chrF reference | chrF this model | delta |
|---|---|---|---|
| eng_Latn-hin_Deva | 65.69 | 65.17 | -0.52 |
| eng_Latn-ben_Beng | 54.52 | 54.62 | +0.10 |
| eng_Latn-tam_Taml | 61.33 | 61.59 | +0.27 |
About 0.3 s per sentence and 707 MB peak memory on a laptop CPU (4 threads). Only three of the 22 targets were scored; Santali output mixes scripts in some sentences, as the original does.
pip install bhashantar
bhashantar translate --tgt hin_Deva "The train leaves at 6:45 from platform 3."
Please cite the IndicTrans2 paper (Gala et al., TMLR 2023) when you use this model.