SciEmbed-CTX-2048

Intermediate long-context variant (max_seq_length=2048). Point on the 512→2K→8K context-length scan.

A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the SciEmbed release (paper under double-blind review; author info omitted).

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("anon-nlp/sciembed-ctx-2048")
emb = model.encode(["citation-context supervision for scientific embeddings"],
                   normalize_embeddings=True)

• Context length: 2048 tokens
• Pooling: mean · Output dim: 768 (Matryoshka-truncatable to 512/256/128)
• License: MIT

Citation

See the repository README. Paper: SciEmbed: Citation-Context Supervision for Scientific Document Embeddings (under review).