opus-mt-en-fr, ONNX int8 (merged decoder)

Helsinki-NLP/opus-mt-en-fr (revision dd7f654), exported to ONNX for on-device translation.

• Exported with Optimum (--task text2text-generation-with-past): one encoder and one merged decoder that handles both the first step and the cached steps (use_cache_branch input), so the decoder weights are stored once.
• Dynamic int8 weight quantisation (ONNX Runtime quantize_dynamic, EnableSubgraph so the layers inside the merged decoder's branches are quantised too).
• About 140 MB in total.

With greedy decoding, the int8 files give the same output as the original PyTorch model on 24 of 30 short test phrases; the rest mostly differ in wording (quantisation).

Licence and credit

The model is by the Language Technology Research Group at the University of Helsinki (OPUS-MT, Tiedemann & Thottingal, 2020), released under Apache-2.0; this conversion is released under the same licence.