afro-xlmr-large-61L

AfroXLMR-large was created by MLM adaptation of XLM-R-large model on 61 languages widely spoken in Africa including 4 high-resource languages.

Pre-training corpus

A mix of mC4, Wikipedia and OPUS data

Languages

There are 61 languages available :

• English (eng)
• Amharic (amh)
• Arabic (ara)
• Somali (som)
• Kiswahili (swa)
• Portuguese (por)
• Afrikaans (afr)
• French (fra)
• isiZulu (zul)
• Malagasy (mlg)
• Hausa (hau)
• chiShona (sna)
• Egyptian Arabic (arz)
• Chichewa (nya)
• Igbo (ibo)
• isiXhosa (xho)
• Yorùbá (yor)
• Sesotho (sot)
• Kinyarwanda (kin)
• Tigrinya (tir)
• Tsonga (tso)
• Oromo (orm)
• Rundi (run)
• Northern Sotho (nso)
• Ewe (ewe)
• Lingala (lin)
• Twi (twi)
• Nigerian Pidgin (pcm)
• Ga (gaa)
• Lozi (loz)
• Luganda (lug)
• Gun (guw)
• Bemba (bem)
• Efik (efi)
• Luvale (lue)
• Luba-Lulua (lua)
• Tonga (toi)
• Tshivenḓa (ven)
• Tumbuka (tum)
• Tetela (tll)
• Isoko (iso)
• Kaonde (kqn)
• Zande (zne)
• Umbundu (umb)
• Mossi (mos)
• Tiv (tiv)
• Luba-Katanga (lub)
• Fula (fuv)
• San Salvador Kongo (kwy)
• Baoulé (bci)
• Ruund (rnd)
• Luo (luo)
• Wolaitta (wal)
• Swazi (ssw)
• Lunda (lun)
• Wolof (wol)
• Nyaneka (nyk)
• Kwanyama (kua)
• Kikuyu (kik)
• Fon (fon)

Acknowledgment

We would like to thank Google Cloud for providing us access to TPU v3-8 through the free cloud credits. Model trained using flax, before converted to pytorch.

BibTeX entry and citation info.

@misc{adelani2023sib200,
      title={SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects}, 
      author={David Ifeoluwa Adelani and Hannah Liu and Xiaoyu Shen and Nikita Vassilyev and Jesujoba O. Alabi and Yanke Mao and Haonan Gao and Annie En-Shiun Lee},
      year={2023},
      eprint={2309.07445},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}