bekko-embedding-v1-a25m — weight-only int8 ONNX
A weight-only 8-bit copy of hotchpotch/bekko-embedding-v1-a25m at revision
44f0b8af0f487acd0ccf1a7cb7ae7a29a6dfc09c, made from its official fp32 ONNX export onnx/model_fp32_not_recommended.onnx.
Same tokenizer, same MEAN pooling, same embedding dimension; L2-normalize the pooled vector.
How it was made
Every MatMul with a constant weight was replaced by ONNX Runtime's MatMulNBits
(bits=8, block_size=32, symmetric, accuracy_level unset). Activations stay fp32, so a
text's vector does not depend on the other texts in its batch. The token embedding table (Gather) stays
fp32. Weights are stored in the external-data sidecar onnx/model.onnx_data.
uv run scripts/prepare-embedding-w8.py --repo hotchpotch/bekko-embedding-v1-a25m --revision 44f0b8af0f487acd0ccf1a7cb7ae7a29a6dfc09c \
--onnx onnx/model_fp32_not_recommended.onnx --pooling mean --out <dir>
The script is scripts/prepare-embedding-w8.py in the Slipwiki repository. Dependencies:
onnx 1.23.2, onnxruntime 1.24.4, numpy 2.3.5,
tokenizers 0.22.2, onnx-ir 1.1.0, CPython 3.12.14. Re-running it reproduces the
model files byte for byte.
Fidelity against the fp32 export
On 13 built-in texts (English, Chinese, Japanese, French, German, very short and truncated): minimum cosine to fp32 0.999615; minimum cosine between batched and one-by-one encoding 1.000000.
Files
| File | Bytes | SHA-256 |
|---|---|---|
1_Pooling/config.json |
312 | 029891eaab443d10165e66e6f3d5d9008a16b712481f71ae7c938234e8fdeca1 |
LICENSE |
1395 | eddadd645883bb447653fd61247716b8a3aafa382042bc0d27123949481d66e6 |
NOTICE |
570 | 3b5ab795ab202cfe0618a49cbcbef24c96aa9fb0a84971bc89e279780150e028 |
config.json |
1749 | 1719a7afae7032fd3b1320af2673740a17ac5fa7e6a12bb5b09e2611bfe8305a |
modules.json |
229 | 8f4b264b80206c830bebbdcae377e137925650a433b689343a63bdc9b3145460 |
onnx/model.onnx |
266176 | fea309d925a5a50a1b01c6492fabc594ff120799533fb8ea0e37ed110cde5757 |
onnx/model.onnx_data |
421292544 | 203908ee3c3b192b3b82213e6b03c32ff3d7ccf26bb51f6beb48af9d5012f1da |
sentence_bert_config.json |
58 | 9ee3935137f87c9ce6662ae17fe7afb7c9c64744a41247359fd228dc37e627b3 |
special_tokens_map.json |
1051 | 6e3204c5a4004719c185007a745d1940471a6fb02521cfa0a00306dbb6bc5903 |
tokenizer.json |
34363442 | 8bd47075711f75a143d1b78e01a41cc65c1c591b00d3cfeffc23db07adce1392 |
tokenizer_config.json |
46634 | 2ee40d1066ac855e1ea38ac422dbe1f3ddf84c9d415e79272adce569668f3de4 |
License
mit, as the base model; see LICENSE and NOTICE. This is a modified (quantized) version of the
base model.
The tokenizer is carried over unchanged from the base model, which reuses the Gemma 2 tokenizer (via mmBERT); its terms are as stated by the upstream model.