opus-mt-it-en, ONNX int8 (merged decoder)
Helsinki-NLP/opus-mt-it-en (revision 42556a0), exported to ONNX for on-device translation.
- Exported with Optimum (
--task text2text-generation-with-past): one encoder and one merged decoder that handles both the first step and the cached steps (use_cache_branchinput), so the decoder weights are stored once. - Dynamic int8 weight quantisation (ONNX Runtime
quantize_dynamic,EnableSubgraphso the layers inside the merged decoder's branches are quantised too). - About 173 MB in total.
| File | Purpose |
|---|---|
encoder_model.onnx |
Encoder |
decoder_model_merged.onnx |
Decoder with KV cache |
source.spm, target.spm |
SentencePiece models |
vocab.json |
Shared vocabulary |
config.json, generation_config.json |
Token IDs (eos, pad, decoder start) and generation settings |
With greedy decoding, the int8 files give the same output as the original PyTorch model on about 5 in 6 short test phrases; the rest differ in wording (quantisation), not in meaning.
Licence and credit
The model is by the Language Technology Research Group at the University of Helsinki (OPUS-MT, Tiedemann & Thottingal, 2020), released under Apache-2.0; this conversion is released under the same licence.