potion-mxbai-256d-v2

A compact static embedding model that achieves near-SOTA quality at 8.2x smaller size. Only 0.68 points behind our 512D baseline on the full MTEB English suite while being 8.2x smaller (7.5MB int8 vs ~62MB).

Highlights

• 71.45 avg on full MTEB English (STS + Classification + PairClassification, 25 tasks)
• 7.5MB with int8 quantization (8.2x smaller than 512D baseline)
• 28.8MB at full precision (float32)
• 80-88x faster than all-MiniLM-L6-v2 on CPU (~15K vs ~200 sentences/sec)
• Pure numpy inference — no GPU needed
• Native int8 support via model2vec v0.7 — zero quality loss

How It Was Made

1. Teacher: mixedbread-ai/mxbai-embed-large-v1 (335M params, BERT-large architecture)
2. Distillation: model2vec distillation with 256-dim PCA and corpus-informed vocabulary
3. Tokenlearn pre-training: Contrastive loss training on ~217K C4 English sentences using tokenlearn
4. Born-again self-distillation: A second round of contrastive training using the model's own sentence embeddings as targets instead of the teacher's. This closes the "representation gap" between what a static model can learn and what the transformer teacher produces, yielding +0.49 avg improvement.

Key insight

The original teacher's representations are too complex for a static lookup table to fully capture. By self-distilling — training a fresh model to match our own model's outputs — we create "easier" targets that a static model can actually learn well. This simple technique improved all three benchmark categories.

Benchmark Results (Full MTEB English Suite)

Evaluated on 25 tasks (10 STS, 12 Classification, 3 PairClassification), English subsets only, identical eval code across all models.

Usage

from model2vec import StaticModel

# Full precision (28.8MB)
model = StaticModel.from_pretrained("blobbybob/potion-mxbai-256d-v2")

# INT8 quantized (7.5MB, same quality)
model = StaticModel.from_pretrained("blobbybob/potion-mxbai-256d-v2", quantize_to="int8")

embeddings = model.encode(["Hello world", "Static embeddings are fast"])

With Sentence Transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("blobbybob/potion-mxbai-256d-v2")
embeddings = model.encode(["Hello world", "Static embeddings are fast"])

When to use this model

• You need fast, lightweight embeddings for search, clustering, or classification
• You're deploying on resource-constrained environments (mobile, edge, serverless)
• You want sub-millisecond inference without GPU
• You need embeddings in a small binary (7.5MB vs 100MB+ for transformer models)

Model Family

Training Details

• Featurization: ~217K C4 sentences encoded by mxbai-embed-large-v1
• Training: Tokenlearn contrastive loss + born-again self-distillation, batch size 256
• Vocabulary: 29,524 tokens (corpus-informed vocabulary from mxbai teacher tokenizer)
• Dimensions: 256 (via PCA reduction from teacher's 1024-dim output)
• Compute: Local RTX 2070

Citation

@article{minishlab2024model2vec,
  author = {Tulkens, Stephan and {van Dongen}, Thomas},
  title = {Model2Vec: Fast State-of-the-Art Static Embeddings},
  year = {2024},
  url = {https://github.com/MinishLab/model2vec}
}