m2v-gte-256-edu-v2
A Model2Vec static embedding model for German education content. It is the successor of JanSchachtschabel/m2v-gte-256-edu and was fine-tuned on German educational data to find learning resources by keywords and short queries.
Like every Model2Vec model it needs no GPU and no transformer at inference time: a text is embedded by looking up and averaging token vectors, which makes it very fast on a CPU.
Usage
from model2vec import StaticModel
model = StaticModel.from_pretrained("JanSchachtschabel/m2v-gte-256-edu-v2")
embeddings = model.encode(["Bruchrechnung für die 6. Klasse", "Photosynthese"])
The model also works with sentence-transformers (StaticEmbedding).
Tip for search queries: users often type in lowercase. The model is strongest on lowercase queries when queries and documents are both reduced to their nouns in base form first (e.g. "brüche kürzen" -> "Bruch"). The vocabulary contains base forms, inflected forms and lowercase variants, so it works with and without this preprocessing. For classification and similarity of well-formed text, use the raw text.
Model details
- Teacher: Alibaba-NLP/gte-multilingual-base
- Dimensions: 256
- Vocabulary: about 424,000 tokens (teacher vocabulary plus German education terms, their base forms and lowercase variants)
- Weights: float16, about 224 MB
Training
- Distillation from
gte-multilingual-basewith a vocabulary extended by German education terms and keywords. - Fine-tuning on German educational texts (Tokenlearn): the static model learns to reproduce the teacher's embeddings of texts from the education domain.
- Pairwise fine-tuning (contrastive, in-batch negatives): keyword -> learning resource and title -> description pairs from the education domain.
Evaluation
All models were evaluated on the same held-out German education test set; its documents were not used for training. Higher is better.
| Model | Keyword search nDCG@10 | Keyword search Recall@10 | Lowercase keywords nDCG@10 | Title -> description MRR@10 | Title -> description Recall@10 | Subject classification macro-F1 |
|---|---|---|---|---|---|---|
| m2v-gte-256-edu-v2 (raw text) | 0.323 | 0.371 | 0.212 | 0.648 | 0.794 | 0.500 |
| m2v-gte-256-edu-v2 (noun base forms) | 0.297 | 0.352 | 0.291 | 0.618 | 0.768 | 0.462 |
| m2v-gte-256-edu (predecessor) | 0.084 | 0.103 | 0.059 | 0.344 | 0.467 | 0.372 |
| potion-multilingual-128M | 0.122 | 0.148 | 0.106 | 0.481 | 0.641 | 0.393 |
| paraphrase-multilingual-MiniLM-L12-v2 (transformer) | 0.076 | 0.096 | 0.060 | 0.460 | 0.620 | 0.419 |
| TF-IDF (reference) | 0.220 | 0.235 | 0.220 | 0.577 | 0.685 | 0.435 |
Compared models use raw text input.
- Keyword search: a keyword is the query; relevant are the resources tagged with it. The resources are embedded without their keywords.
- Title -> description: find a resource's description from its title.
- Subject classification: a logistic-regression probe on the frozen embeddings.
Compared with its predecessor, v2 almost quadruples keyword search quality and also beats the transformer MiniLM-L12 and a TF-IDF baseline on all three tasks.
Limitations
- Made for German education content; quality on other domains and languages is not measured.
- Part of the lead over the compared models comes from adaptation to the education domain, which they were not trained on.
- A static model cannot match exact terms as precisely as BM25. For search, combining both (hybrid search) is recommended.
License
Apache 2.0, like the teacher model.