SauerkrautLM-MoEdernColBERT-1.5M

A 1.5M-parameter multilingual ColBERT retriever: under 1 MB at 4 bit, 43,255 documents per second on one GPU. Built for German and six other European languages, with documents of up to 32k tokens.

The SauerkrautLM-MoEdernColBERT family introduces MoEdernBERT, a new ModernBERT-style encoder architecture with a mixture of experts and hash embeddings. MoEdernColBERT-1.5M has 1.47M parameters, of which only 0.88M are active per token.

Highlights

• Ahead of 66× larger granite-97m-multilingual-r2 in English and German question answering. NanoBEIR-en 0.591 vs. 0.589, GermanQuAD 0.976 vs. 0.918.
• 6.2× the throughput of granite-107m-multilingual and 300× that of Qwen3-Embedding-0.6B: 43,255 documents per second on one GPU.
• German retrieval ahead of models up to 400× larger. GerDaLIR 0.354 vs. 0.207 for Qwen3-Embedding-0.6B.
• Dutch retrieval ahead of models up to 400× larger. LegalQA-NL 0.690 vs. 0.679† for Qwen3-Embedding-0.6B.
• German legal texts: ahead of 8B models and commercial APIs. LegalQuAD 0.794 vs. 0.772† for Nemotron-3-Embed-8B.
• Long documents up to 32k tokens. LongEmbed 85.2, ahead of the 7B–14B models on the public LongEmbed leaderboard.
• 0.87 MB at 4 bit with the same quality — small enough for phones and edge devices.

† public MTEB result, not measured by us.

Architecture: MoEdernBERT

2 transformer layers, hidden size 128.

• Hash embedding. A factorized embedding: every token id owns a short row of 16 values, which a shared layer projects to the hidden size. This needs 0.48M parameters instead of 3.8M.
• Mixture-of-experts MLP. In every transformer layer, a router sends each token to 2 of 8 GeGLU experts, and one shared expert always runs.
• Attention and long context. Global attention with RoPE, trained on 512 tokens. Above 513 tokens, a ±256-token sliding window keeps every token within the trained range, so documents of up to 32k tokens work.
• ColBERT head. Every token is mapped to 128 dimensions. Queries are padded to 32 tokens with [MASK], and relevance is scored with MaxSim.

Training: Stella-style embedding distillation, then knowledge-distillation pre-training and multilingual fine-tuning.

Speed

Sparse activation makes MoEdernColBERT-1.5M fast on a GPU and on a laptop.

One NVIDIA RTX 6000 Ada, bf16, 512-token documents, bundled Triton MoE kernel.

Results

German

Other languages

German legal texts

Long documents

The MoEdernColBERT family

NDCG@10 unless noted, higher is better. Throughput measured as under Speed.

Usage

The model ships its own modeling code, so trust_remote_code=True is required. Tested with sentence-transformers 6.1 and transformers 5.

pip install -U "sentence-transformers>=6.1"
pip install -U triton   # Linux + NVIDIA GPU: enables the bundled fused MoE kernel

Fast GPU path. On NVIDIA GPUs, the bundled Triton MoE kernel switches on automatically during inference; without a CUDA GPU or without Triton, the model falls back to plain PyTorch. Set MOEDERN_TRITON=0 to turn the kernel off.

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("VAGOsolutions/SauerkrautLM-MoEdernColBERT-1.5M", trust_remote_code=True)

queries = ["Wie hoch ist die Zugspitze?", "What is the capital of France?"]
documents = [
    "Die Zugspitze ist mit 2962 Metern der höchste Berg Deutschlands.",
    "Paris is the capital and largest city of France.",
    "Berlin ist die Hauptstadt Deutschlands.",
]

query_embeddings = model.encode_query(queries)          # list of (32, 128) tensors
document_embeddings = model.encode_document(documents)  # list of (num_tokens, 128) tensors

scores = model.similarity(query_embeddings, document_embeddings)  # MaxSim, shape (2, 3)
print(scores.argmax(dim=1))  # tensor([0, 1])

Long documents. Documents are cut at 512 tokens by default. Raise the limit, and the ±256 window switches on automatically:

model[0].document_length = 32768

Precision. Use fp32 or bf16. Do not call .half(): fp16 overflows and produces NaN embeddings.

GGUF and CoreML. Quantized versions run on CPUs with our ggml-based engine (GGUF) and on Apple GPUs and the Neural Engine (CoreML). The 4-bit CoreML file is 0.87 MB and scores 0.590 on NanoBEIR English (fp32: 0.591).

Evaluation

All numbers without † come from our own runs with sentence-transformers 6, each model with its own configuration. On the German tasks, our models use the better of 512 or 8,192 document tokens per task. The paper has the full protocol and all results.

Languages and limitations. German, English, Spanish, Dutch, French, Italian and Portuguese. Arabic, Japanese and Korean are not supported.

License

Apache 2.0, see LICENSE.

Citation

@misc{sauerkrautlm_moederncolbert_2026,
  title={MoEdernBERT: Hash-Embedded Mixture-of-Experts Encoders for Tiny Multilingual Late-Interaction Retrieval},
  author={David Golchinfar},
  organization={VAGO Solutions},
  url={https://huggingface.co/VAGOsolutions/SauerkrautLM-MoEdernColBERT-1.5M},
  year={2026}
}

Contact

VAGO solutions