minishlab/blink-8M

blink-8M is a fast English static embedding model by Minish: a single 30,000 × 256 token embedding table (7.7M parameters, 31MB in float32) with mean pooling. It outperforms every static model of the same size, and, depending on the task, often much larger ones. It fully supersedes potion-base-8m, being the same size but better on every task. Static models are models without any active parameters; all model parameters are just embedding parameters. Because of this, they are many orders of magnitude faster than transformers. Although this comes at a cost, this cost is much lower than you would expect.

Usage

With model2vec:

from model2vec import StaticModel

model = StaticModel.from_pretrained("minishlab/blink-8M")
embeddings = model.encode(["a first sentence", "a second sentence"])

With model2vec-rs

use anyhow::Result;
use model2vec_rs::model::StaticModel;

fn main() -> Result<()> {
    let model = StaticModel::from_pretrained(
        "minishlab/blink-8M",
    )?;

    let sentences = vec![
        "Hello world".to_string(),
        "Rust is awesome".to_string(),
    ];

    let embeddings = model.encode(&sentences);
    println!("Generated {} embeddings.", embeddings.len());

    Ok(())
}

With Sentence Transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("minishlab/blink-8M")
embeddings = model.encode(["a first sentence", "a second sentence"])

Results

MTEB

MTEB(eng, v2), all 41 tasks (mteb 2.27.1). Every model is evaluated at 256 dimensions, see Compared models.

Summary

• Mean: blink-8M has the highest mean (55.35), just ahead of lattice (55.24) and bekko (55.21).
• Retrieval-type tasks: blink-8M is best on retrieval, reranking and pair classification. On retrieval it scores 38.86, against 38.01 for lattice and 31.22 for potion-base-8M. Note that this task has significant overlap with BEIR, see below.
• Other tasks: the potion-base models are best on classification (70.48 and 70.41) and potion-base-8M on summarization (25.14), with blink-8M at 67.88 and 22.48. Bekko is best on STS and lattice on clustering.
• Against potion-base-8M: at roughly the same size, blink-8M is 2.05 points higher on the mean and 7.64 points higher on retrieval, but 2.53 points lower on classification.

Compared models

We compare blink-8M to six other static embedding models:

• potion-base-8M: our general-purpose model2vec model, 30k vocab, 256 dimensions.
• potion-base-32M: a larger version of potion-base-8M, 65k vocab, 512 dimensions.
• potion-retrieval-32M: potion-base-32M fine-tuned for retrieval, 65k vocab, 512 dimensions.
• static-retrieval-mrl-en-v1: Sentence Transformers' static retrieval model, 30k vocab, 1024 dimensions.
• bekko-static-embedding-v1: A multilingual model trained on retrieval data, 200k vocab, 1024 dimensions.
• lattice-retrieval: A monolingual model, trained on retrieval data, 30k vocab, 1024 dimensions.

Every model is evaluated at 256 dimensions. potion-base-32M, potion-retrieval-32M and static-retrieval-mrl-en-v1 are truncated to their first 256 dimensions and use L2 normalized embeddings, even if the original model did not have normalization. Applying normalization uniformly improved performance across all models.

We also evaluate many other datasets and tasks below.

RTEB: blink-8M 30.84 mean nDCG@10 (best)

RTEB (RTEB(eng, beta) in MTEB): retrieval across specialized domains, namely legal, finance, healthcare and code. We use the 15 public English tasks, including only the English subset of CUREv1. The private tasks are left out because their data is not available. All scores are nDCG@10, and the last row is the mean over the 15 tasks.

Note that FreshStack is also present as a separate eval, see below, although there it uses Recall@50 and α-nDCG@10.

Summary

• Mean: blink-8M has the highest mean (30.84), but bekko is close (30.32). Blink-8M is best on 8 of 15 tasks, bekko on 6 and lattice on 1 (AILACasedocs).
• By domain: blink-8M leads on legal (36.82 against 35.96 for bekko) and finance (39.59 against 37.50). Bekko leads on healthcare (29.84 against 29.15), and code is a tie (25.01 against 24.95).
• Code: bekko is best on the code-generation tasks (Apps, HumanEval, MBPP and DS1000, the last by 0.02), while blink-8M has the largest win of the table on WikiSQL (37.29 against at most 32.18).
• Overall: Apps and SWE-bench are hard for every model, with nDCG@10 below 9.

BrowseComp-Plus: blink-8M 2.13 evidence nDCG@10 (best)

BrowseComp-Plus: 830 hard, multi-hop BrowseComp questions over a fixed corpus of 100,195 web documents. Queries are the original questions, and documents are embedded in full, without truncation. We report two sets of relevance labels: evidence documents (all documents needed to answer the question) and gold documents (the documents that contain the answer).

Evidence

Gold

Blink has the best nDCG@10 on both label sets and the best gold Recall@5. Bekko has the best evidence Recall@5 and the best Recall@100 and Recall@1000 on both label sets.

Decontaminated BEIR: blink-8M 46.70 mean nDCG@10 (best)

We evaluate on LightOn's decontaminated version of BEIR (lightonai/<dataset>-decontaminated). These are 12 BEIR datasets with every query and document found in the mgte-en pre-training data removed. Contamination was found in two passes: exact matches on normalized text hashes, and 13-gram containment ≥ 0.5. Relevance judgments that point to a removed query or document were dropped as well.

We use the test split and only keep queries that have at least one relevant document left. Documents are encoded as title + text, and retrieval uses cosine similarity over the full corpus. We report nDCG@10 (ranking quality in the top 10) and Recall@100 (fraction of relevant documents found in the top 100). Following LightOn and Lattice, we leave out FEVER and ClimateFEVER.

nDCG@10

Recall@100

Summary

• nDCG@10: blink-8M has the highest mean (46.70) and is best on 6 of 12 datasets: FiQA, HotpotQA, MS MARCO, NFCorpus, SciFact and TREC-COVID. Lattice is best on 3 (DBPedia, NQ, SCIDOCS), bekko on 2 (ArguAna, Quora) and static-retrieval-mrl on 1 (Touché-2020).
• Recall@100: the top three are close: blink-8M 64.96, lattice 64.71, bekko 64.48. Bekko is best on 6 datasets, blink-8M on 3 (one shared with potion-retrieval-32M).
• TREC-COVID: blink-8M, lattice and bekko score 66–68 nDCG@10, while the potion models and static-retrieval-mrl score 45–51.

BRIGHT: blink-8M 10.51 mean nDCG@10 (best)

BRIGHT (Su et al., 2024) is a retrieval benchmark that needs reasoning: relevant documents often share few keywords or little surface topic with the query. For example, a query may describe a coding problem, and the relevant document is another problem that uses the same algorithm. BRIGHT has 12 tasks in three groups:

• StackExchange (7): Biology, Earth Science, Economics, Psychology, Robotics, Stack Overflow and Sustainable Living. Queries are user posts, and relevant documents are web pages linked from accepted answers.
• Coding (2): LeetCode (problems that share a solution technique) and Pony (documentation for a low-resource programming language).
• Math (3): AoPS (competition problems with a similar solution), TheoremQA-Q (problems that use the same theorem) and TheoremQA-T (the theorem statement needed to solve a problem).

We use the original queries, not the LLM-generated reasoning traces, and the standard (short) document split. For each query we drop the documents BRIGHT marks as excluded, then retrieve the top 1000 documents by cosine similarity. Metrics are computed with pytrec_eval: nDCG@10 (ranking quality in the top 10, BRIGHT's main metric) and Recall@100 (fraction of relevant documents found in the top 100).

Because these need reasoning, we expect low scores: these are all static models and can't reason. Nevertheless, it's an interesting held-out task to try.

nDCG@10

Recall@100

Summary

• nDCG@10: blink-8M has the highest mean (10.51, against 9.81 for lattice) and is best on 9 of 12 tasks. The other three go to potion-retrieval-32M (Robotics), potion-base-32M (Sustainable Living) and lattice (AoPS).
• Recall@100: blink-8M has the highest mean (33.71, against 31.28 for lattice) and is best on 8 of 12 tasks. The largest gaps are on Pony (17.03 against at most 14.30 for the others) and Stack Overflow (40.20 against at most 34.06).

FreshStack: blink-8M 15.72 mean α-nDCG@10 (best)

FreshStack (Thakur et al., 2025) tests retrieval on niche, recent technical topics. The queries are Stack Overflow questions, and the corpus is code and documentation chunks from the GitHub repositories of each topic, taken from an October 2024 snapshot. Each question's accepted answer is split into atomic facts ("nuggets"), and corpus chunks are labeled with the nuggets they support. A good ranking therefore has to cover all the facts needed to answer the question, not just find one relevant chunk.

We evaluate on the test split of the 5 topics. Documents are encoded as title + text. Queries and documents are truncated to 2048 tokens, and we retrieve the top 50 documents by cosine similarity. Metrics come from the official freshstack evaluation package:

• α-nDCG@10: nDCG over the top 10 where each document only gets credit for nuggets that higher-ranked documents have not already covered (α = 0.5). It rewards rankings that cover many different nuggets.
• Recall@50: the fraction of a query's relevant documents found in the top 50.

α-nDCG@10

Recall@50

Summary

• α-nDCG@10: blink-8M has the highest mean (15.72, against 14.81 for bekko) and is best on 4 of 5 topics. Bekko is best on Godot. The largest gap is on Laravel: 21.27, against 17.11 for the next best model.
• Recall@50: blink-8M has the highest mean (25.39) but is best on only 2 of 5 topics, Godot (by 0.01) and Laravel. Bekko is best on LangChain and YOLO, and potion-base-8M on Angular.
• potion-base-8M: it is the smallest model here, yet it is competitive. It is 4th on mean α-nDCG@10, and on mean Recall@50 it is roughly tied with lattice (22.83 vs 22.81).

Training

We followed a four-step procedure.

1. Tokenizer and potion initialization

We first train a 30k WordPiece tokenizer on LightOn's English pre-training data, and a distillation stage (potion) from bge-base-en-v1.5 on 1.1M samples of the Tokenlearn C4 data. This led to results comparable to potion-base-8m on MTEB and the retrieval tasks.

2. Contrastive training

Following, Erik Kaunismäki's work on training Lattice, we train on 96M text pairs from lightonai/embeddings-pre-training-curated, using three epochs instead of 1. We ablated hyperparameters in this phase on NanoBEIR. This led us to disable MRL, which had a slightly negative effect, but also did not lead to better performance at lower dimensionalities. Similarly, and unlike previous static models, we also found that training with a learning rate scheduler is unnecessary. However, like Hotchpotch's bekko recipe, we found that drastically lowering temperature works much better. Other hyperparameters and regularization schemes, e.g., dropping tokens, had little effect or were negative.

3. Hard negative fine tuning

Again following Erik's work, we then perform hard-negative fine-tuning on lightonai/embeddings-fine-tuning-filtered-en (1.57M queries with 10 mined hard negatives each), training for 3 epochs, again with a 0.2 constant learning rate.

4. Weight averaging

We found that weight averaging then caused the model to outperform either of the stages separately: the final model is 30% of the contrastive model from step 2 and 70% of the hard-negative model from step 3, giving us a boost on MTEB without sacrificing retrieval quality.

Library Authors

Model2Vec was developed by the Minish team consisting of Stephan Tulkens and Thomas van Dongen.

Citation

Please cite the Model2Vec repository if you use this model in your work.

@software{minishlab2024model2vec,
  author       = {Stephan Tulkens and {van Dongen}, Thomas},
  title        = {Model2Vec: Fast State-of-the-Art Static Embeddings},
  year         = {2024},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.17270888},
  url          = {https://github.com/MinishLab/model2vec},
  license      = {MIT}
}