multilingual-e5-small · Core ML

intfloat/multilingual-e5-small converted to a Core ML ML Program (fp16) for on-device semantic search in the DND Master app (iPad / iPhone / Mac).

What is inside

Pad token id is 1 (<pad>), attention mask is 1 for real tokens. Use the e5 prefixes: query: for queries, passage: for documents.

Conversion notes

• coremltools 9.0, torch 2.7.0, transformers 4.46.3, minimum_deployment_target = iOS18.
• Enumerated input shapes instead of a RangeDim: with a flexible range Core ML returned NaN (“Data-dependent shapes were disabled”).
• The additive attention mask was replaced by -1e4 before tracing: the default finfo(float32).min overflows to -inf in fp16 and the softmax produces NaN.
• Parity with the PyTorch reference: cosine 1.000000 on test sentences; parity with an MLX fp16 embedding of a 12 270-chunk Russian corpus: min cosine 0.9997.
• Recommended compute units: .cpuAndNeuralEngine. The GPU path is ~25× slower for this graph on Apple silicon.

The conversion script lives in the app repository (BestiaryEmbedder/convert-e5-coreml.py).

License

The weights are derived from intfloat/multilingual-e5-small (MIT).