distiluse-base-multilingual-cased-v2

This repository hosts the distiluse-base-multilingual-cased-v2 models exported for the React Native ExecuTorch library as ExecuTorch .pte programs, ready to run on device.

Upstream model: distiluse-base-multilingual-cased-v2

Variants

Repository structure

config.json                                                     58 B
coreml/config.json                                              1.0 kB
coreml/distiluse_base_multilingual_cased_v2_coreml_fp16.pte     258 MB
mlx/config.json                                                 1.0 kB
mlx/distiluse_base_multilingual_cased_v2_mlx_int8.pte           133 MB
tokenizer.json                                                  2.8 MB
tokenizer_config.json                                           531 B
vulkan/config.json                                              1.0 kB
vulkan/distiluse_base_multilingual_cased_v2_vulkan_fp16.pte     258 MB
xnnpack/config.json                                             1.7 kB
xnnpack/distiluse_base_multilingual_cased_v2_xnnpack_8da4w.pte  375 MB
xnnpack/distiluse_base_multilingual_cased_v2_xnnpack_fp32.pte   516 MB

Compatibility

These files are published for the ExecuTorch v1.4.1 runtime. ExecuTorch gives no forward compatibility guarantee, so an older runtime may fail to load them.

To use them in React Native ExecuTorch, pass the model constant shipped in the library's model registry to the corresponding task pipeline. See the documentation.

To load these files in your own ExecuTorch runtime, read the compatibility note first.

Model details

• Architecture: DistilBERT multilingual cased + mean pooling + Dense (768→512, Tanh) + L2 norm.
• Output dimension: 512.
• Max sequence length: 126 tokens (128 − 2 for [CLS] / [SEP]).
• Languages: 50+ (multilingual).
• Typical strength: cross-lingual sentence similarity and medium-length sentence retrieval. Short single-word queries in non-English languages are this model's weakest case — for those, longer sentences and/or English inputs give markedly better ranking.

Export notes

The exported program skips HuggingFace's internal attention-mask-to-4D conversion because the RNE runtime never pads at inference (single sentence, no batching). This preserves bit-exactness with the PyTorch reference (RMSE 0 on fp32 random input) while trimming ~27% off the XNNPACK forward wall-time and keeping XNNPACK delegation around 89–91% of graph runtime.

Unsupported combinations (rejected by the exporter, documented for reference):

• XNNPACK + fp16 — model.to(torch.float16) causes softmax / LayerNorm overflow and the runtime output is NaN. XNNPACK's size wins come from quantization, not fp16.
• CoreML + 8da4w — coremltools has no MIL mapping for the torch.int8 tensors torchao emits (KeyError: torch.int8). The CoreML-native way to shrink further is ct.optimize.coreml palette/linear quantization, not torchao source transforms.