google/madlad400-10b-mt converted to the
CTranslate2 format and quantized to int8.
MADLAD-400-10B-MT is a multilingual machine translation model covering over 400 languages, trained on 250 billion tokens. This repo changes the runtime format.
| File | Size | Purpose |
|---|---|---|
model.bin |
~10.7 GB | CTranslate2 model weights (int8) |
config.json |
— | CTranslate2 runtime config |
shared_vocabulary.json |
— | CTranslate2 vocabulary |
spiece.model, tokenizer.json, tokenizer_config.json |
— | SentencePiece / T5 tokenizer |
Apache 2.0, inherited from the original model. Please cite the original work:
@misc{kudugunta2023madlad400,
title={MADLAD-400: A Multilingual And Document-Level Large Audited Dataset},
author={Sneha Kudugunta and Isaac Caswell and Biao Zhang and Xavier Garcia and
Christopher A. Choquette-Choo and Katherine Lee and Derrick Xin and
Aditya Kusupati and Romi Stella and Ankur Bapna and Orhan Firat},
year={2023},
eprint={2309.04662},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Refer to the original model card for intended use, limitations, and evaluation details.
google/madlad400-10b-mt