SciEmbed-FULL

Headline model. Stage 1 DAPT + contrastive on the ~30M-pair Signal A+B pool (1 epoch). The SciRepEval number in the paper is the mean over three seeds; this repo ships the seed-123 weights.

A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the SciEmbed release (Findings of EMNLP 2026).

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("J0nasW/sciembed-full")
emb = model.encode(["citation-context supervision for scientific embeddings"],
                   normalize_embeddings=True)

• Context length: 512 tokens
• Pooling: mean · Output dim: 768 (Matryoshka-truncatable to 512/256/128)
• License: MIT

SciRepEval (4-category macro)

Citation

See the repository README. Paper: SciEmbed: Citation-Context Supervision for Scientific Document Embeddings, Findings of the Association for Computational Linguistics: EMNLP 2026.