Unfold search embedder, French and English
Version 2026.10.30, built 2026-10-30T01:21:06Z.
The embedder that Unfold Note runs offline, on the device, in its search over the user's own Obsidian notes for the chat.
It turns note excerpts and questions into 384-dimension vectors (mean pooling, L2-normalised); a question is prefixed with query: , an excerpt with passage: .
- Base model:
intfloat/multilingual-e5-smallat commit614241f622f53c4eeff9890bdc4f31cfecc418b3(mit). This repository redistributes it under the base model's licence; nothing else is added. - Model file:
model.int8.onnx, quantised to INT8 (onnxruntime dynamic, per channel). Vocabulary: pruned to French and English (fren): embedding rows sliced and tokenizer ids remapped. - Settings: max_len 512 tokens, dims 384, mean pooling.
- Runs on CPU with onnxruntime. Minimum app version: 2.8.0.
Training data (from the base model's card)
- and continually trained on a mixture of multilingual datasets.
- Initialization: microsoft/Multilingual-MiniLM-L12-H384
- First stage: contrastive pre-training with weak supervision
- Filtered mC4
- CC News
- NLLB
- Wikipedia
- Filtered Reddit
- S2ORC
- Stackexchange
- xP3
- Miscellaneous unsupervised SBERT data
- Second stage: supervised fine-tuning
- MS MARCO
- NQ
- Trivia QA
- NLI from SimCSE
- ELI5
- DuReader Retrieval
- KILT Fever
- KILT HotpotQA
- SQuAD
- Quora
- Mr. TyDi
- MIRACL
- For all labeled datasets, we only use its training set for fine-tuning.
- For other training details, please refer to our paper at https://arxiv.org/pdf/2402.05672.
Licence note: training data lists MS MARCO: "MS MARCO". Published with the owner's explicit approval (--allow-licence).
Gates
Measured with the app's own search code (word search, meaning search, reranking) on a synthetic French and English vault of 60 notes with 40 questions, and on the owner's real vault (scores only, no note content published).
| gate | bar | value | result |
|---|---|---|---|
| hits8_vs_word_synthetic | hits@8 above word search alone (synthetic set) | {'word': 0.85, 'e5': 0.95, 'delta': 0.1, 'ci95': [0.025, 0.2]} | pass |
| hits8_vs_word_real | hits@8 above word search alone (real set) | {'word': 0.769, 'e5': 0.865, 'delta': 0.096, 'ci95': [0.019, 0.173]} | pass |
| search_latency_median | local search median <= 300 ms per request (word plus dense) | {'synthetic_p50_ms': 18, 'synthetic_p95_ms': 23, 'real_p50_ms': 25, 'real_p95_ms': 29, 'real_planner_4_requests_p50_ms': 63} | pass |
Integrity
embedder-model.json lists every model file with its size and sha256. The app verifies them after download.