opensearch-neural-sparse-encoding-doc-v3-distill · GGUF
GGUF conversion of opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill
(67M-parameter inference-free sparse retrieval encoder, current v3 generation), packaged for sqlite-sparse:
semantic search in a single SQLite file with no model at query time. Served as the
base tier.
llama.cpp's converter drops the masked-LM head that sparse encoders need, so this repo
ships it as a sidecar: the .sprs file carries the MLM transform, the query-weight
lookup table (the "inference-free" query side), and the WordPiece vocabulary.
This v3 model uses double-log saturation (log1p(log1p(relu(x)))); the sidecar's
header carries an activation flag so runtimes apply the right formula.
Files
| file | what |
|---|---|
base_f16.gguf |
encoder body, f16 (llama.cpp-loadable, per-token hidden states) |
base_q8.gguf |
encoder body, Q8_0 |
base.sprs |
MLM head + query weight table + vocab (format v1) |
Fidelity
Differentially validated against the original SentenceTransformers implementation:
- encoder hidden states, worst token: cosine 0.999995 (f16), 0.999823 (Q8_0)
- end-to-end term weights through the sidecar head: max delta 4.8e-04
- end to end through the compiled sqlite-sparse extension at the card's 512-token setting: SciFact NDCG@10 0.7066 vs 0.708 on the original model card (Q8_0 encoder, u8 storage)
Provenance
Weights are unmodified from the Apache-2.0 original by the OpenSearch project; this repo only changes container format. All retrieval-quality credit belongs to the original model.