multilingual-reranker-small v1.2 — LiteRT (int8 dynamic)
A LiteRT model of Horizon-Labs/multilingual-reranker-small
(v1.2, commit 23139fec6f1f448cdf89a5ac2d2c6bf5e8c7b398): a multilingual cross-encoder (141M, mmBERT-small) that
scores how well a passage answers a question. Q-P1NG uses it on the phone's CPU to order the passages its archive
finds, in place of the GGUF on llama.cpp.
| File | Size | SHA-256 |
|---|---|---|
multilingual-reranker-small-v1.2-L128-L512-int8dyn.tflite (two signatures, recommended) |
152,184,768 bytes | e062475fcf329e910ab360ceadb872dc5c069b29eb8fbbc9647b73ce3b0a0716 |
multilingual-reranker-small-v1.2-L512-int8dyn.tflite (512 tokens only) |
146,999,712 bytes | 1c2e5b70d7c3aa75b87e0e692919bc2f83b5ad569c891748aafa2f2f5a063ebb |
multilingual-reranker-small-v1.2-vocab.gguf |
14,866,112 bytes | b6682a2bddab4974b0a6d52acf6f6f976f89225b4fa125ef1ab76ccdd19ebd90 |
Use
LiteRT 2.3 (CompiledModel, CPU). Inputs: input_ids and attention_mask, int32; output: the pair's score, a logit
(higher: more relevant; sigmoid gives 0-1). The pair is [BOS] question [EOS] passage [EOS], the passage cut to fit,
padded with id 0 (masked out).
The two-signature model has a signature per length, L128 ([1, 128]) and L512 ([1, 512]), over one copy of the
weights: a pair that fits in 128 tokens (a sentence) runs in L128, with the same tokens and so the same score, at a
fraction of the work of L512 (a page). The other model has the 512-token shape only.
The vocabulary file is a GGUF without tensors (the metadata of the Q8_0 GGUF): llama.cpp loads it with vocab_only
and tokenizes the pairs exactly as the original tokenizer does, so no Python or Rust tokenizer is needed on the device.
Measured
38 Italian and English questions about two PDFs, 10 candidate passages each (the passage holding the answer and the 9 most similar to the question), against the original model in FP32 (LiteRT 2.3, laptop CPU):
| FP32 | this model, L512 |
this model, L128 (pairs cut at 128) |
|
|---|---|---|---|
| Same best passage as FP32 (same length) | — | 37 / 38 | 37 / 38 |
| Passage with the answer first | 31 | 32 | 18 (FP32 at 128: 18) |
| Passage with the answer in the first three | 34 | 34 | — |
llama.cpp's tokens from the vocabulary file: identical to transformers' on all 380 pairs, at 512 and at 128 tokens. In Q-P1NG's archive search (68 questions), the passage holding the answer reaches the model as often with the two signatures as with the 512-token one alone.
Conversion
See multilingual-reranker-small-CONVERSION.txt; the tools are in the Q-P1NG repository (tools/litert).
Licence
Apache-2.0, as the original model: see multilingual-reranker-small-LICENSE.txt and
multilingual-reranker-small-NOTICE.txt (mmBERT-small, MIT). The original was trained on openly licensed web text
(FineWeb, FineWeb-2: ODC-BY), MIRACL (Apache-2.0) and WebFAQ (CC-BY-4.0), with labels from Qwen3-Reranker-4B
(Apache-2.0). The changes made here are listed in multilingual-reranker-small-CONVERSION.txt.