Glint-Embed

An 8.41M-parameter static text embedding model for retrieval. It is 33.6 MB on disk, encodes about 17,800 documents a second on a laptop CPU, and with hybrid scoring it ties Qwen3-Embedding-8B on SciQ Recall@1 (88.9 against 88.9) at roughly one nine-hundredth of the parameters.

A smaller sibling, Glint-Embed-Flash (1.07M parameters, 64 dimensions, 4.3 MB), is released separately and is compared against this model throughout. This is the one to reach for by default. Take Flash instead if you are storing 64 dimensions anyway, running on constrained hardware, or indexing very large corpora on CPU.

Quick start

pip install -r requirements.txt

python infer.py --similarity "how do i set up postgres" "postgresql installation guide"
python infer.py --encode "a sentence to embed"
python infer.py --folder ./my-notes --query "what were the deployment steps"

Everything needed to run the model is in this directory. infer.py imports only torch and tokenizers, and carries its own copy of the model definition, so there is no project to clone and no package to install.

from infer import Encoder, Retriever

encoder = Encoder()
corpus = ["Postgres tuning notes ...", "Kubernetes ingress setup ...", "..."]

retriever = Retriever(corpus, encoder)
for score, index, text in retriever.search("how do i tune postgres", top_k=5):
    print(score, text[:80])

vectors = encoder.encode(corpus)                  # (n, 512), L2 normalised
queries = encoder.encode(["a query"], is_query=True)
small = encoder.encode(corpus, dim=64)            # Matryoshka prefix, 8x smaller

Accuracy

Every number below was measured on the same machine and the same benchmark. None are quoted from a leaderboard, and each reference model was run with the pooling and prompt format its authors specify.

The benchmark is 1,000 SQuAD-dev questions and 884 SciQ-test questions searching a shared corpus of 6,951 documents: their gold passages plus 4,000 FineWeb-Edu distractors.

Two things are worth reading carefully here. First, this model matches the 8B model on SciQ and beats every other transformer in the table on it, because those queries are short science questions where term overlap carries most of the signal. Second, it loses SQuAD by 14.6 points to Qwen3-8B, because SQuAD passages are long and full of near-miss distractors, and a static model has no attention and so no representation of word order. That limit is structural. It will not close with a longer training run.

Results on a real corpus

Benchmarks are built to be measured against. To check the model on prose written for other reasons, we indexed a personal Obsidian vault of 3,477 notes (10,355 chunks) and derived queries from the notes' own structure: a note title, a natural question templated from a section heading, and a sentence held out of the chunk it came from.

The split is consistent enough to plan around. Where the query shares wording with the target, this model is at the top of the table. Where the query is a natural question phrased differently from the passage, the transformers lead by 20 to 25 points. For a RAG stack over documents like these, the recommended shape is two stages: this model as the first-stage filter over the whole corpus, and a transformer reranking the top 20 to 50 candidates for question-shaped queries.

Hybrid retrieval is the default

Retriever blends the dense score with Okapi BM25, both z-scored, with alpha weighting the dense side. This is what the published numbers use, and the margin is large: hybrid is worth 12.9 Recall@1 over dense alone on SQuAD (58.2 against 45.3) and 10.4 on SciQ. Anything between alpha=0.5 and alpha=0.6 performs the same.

Dense-only and BM25-only modes are available for comparison:

python infer.py --folder ./notes --query "..." --mode dense
python infer.py --folder ./notes --query "..." --mode bm25 --top-k 10
python infer.py --folder ./notes --query "..." --alpha 0.7

BM25 and the blend both live in infer.py. There is no search service to run.

Matryoshka dimensions

The model was trained so that any prefix of the output vector is a working embedding. Storing 64 dimensions instead of 512 cuts index size by 8x and costs 14.7 Recall@1 on SQuAD. Pick the width against your storage budget with this table in hand.

These are dense-only figures, without the BM25 blend. Note the crossover: at 64 dimensions this model scores 30.6 SQuAD while Glint-Embed-Flash scores 31.3 at its native width. If your storage budget is 64 dimensions, ship Flash instead and save 29 MB.

Throughput

Measured on one desktop CPU, batches of 6,951 documents at a 128-token window. The 10,355-chunk vault above indexes in under a second. The transformer rows are shown for scale, with the two Qwen rows measured on a GPU and the rest on the same CPU.

How the model works

A static embedder looks up one vector per token, weights each token by a learned scalar initialised from corpus IDF, takes the masked mean, applies LayerNorm, and L2 normalises. There is no attention, no positional encoding, and no layer stack, so cost scales with the number of tokens rather than with their square, and almost all of the parameter count sits in a lookup table that never runs a matmul. This is the same family as Model2Vec and potion.

Training took nine CPU minutes against 600,000 query-passage pairs from SQuAD, SciQ, and inverse-cloze crops of FineWeb-Edu, with a symmetric in-batch InfoNCE loss averaged over the Matryoshka prefixes. The embedding table was initialised by embedding each token of the vocabulary on its own with all-MiniLM-L6-v2, and the model additionally regressed its query-passage similarity geometry toward Qwen3-Embedding-0.6B.

No evaluation data was trained on. The benchmark corpus and the vault were held out of every training pass.

Files

infer.py                        standalone inference, BM25, and hybrid retrieval
requirements.txt                torch and tokenizers
models/glint-embed.pt           weights and config
models/tokenizer.json           16,384-token BPE vocabulary
charts/                         the figures above

The checkpoint holds plain tensors and a JSON-safe config dict, loadable with torch.load(..., weights_only=True). The model definition is about 20 lines at the top of infer.py if you would rather port it than import it.

Limits

• English only. The vocabulary is a 16,384-token BPE trained on English text.
• 128-token document window and a 32-token query window. Chunk longer documents, which is what read_folder in infer.py does.
• No instruction prefixes or task prompts. This is a single-purpose retrieval encoder.
• Symmetric tasks such as clustering and deduplication were not measured. The training objective was asymmetric retrieval.
• The word-order limit described under Accuracy is structural to the architecture.