English / Japanese
Sentence-Qwen2.5-Scratch-Bi-EN-0.5B fine-tunes iamtatsuki05/Sentence-Qwen2.5-Scratch-Bi-EN-0.5B-PT with supervised contrastive learning on the supervised subsets of sentence-transformers/embedding-training-data (AllNLI, MS MARCO, Quora duplicate questions, NQ, SQuAD, and TriviaQA), yielding 896-dimensional English sentence embeddings.
sentence-transformers>=4.1.0
transformers>=4.51.0
accelerate>=1.6.0
sentencepiece>=0.2.0
flash-attn>=2.7.3
import torch
from sentence_transformers import SentenceTransformer
model_name = "iamtatsuki05/Sentence-Qwen2.5-Scratch-Bi-EN-0.5B"
model_kwargs = {
"torch_dtype": torch.bfloat16,
"attn_implementation": "flash_attention_2",
}
model = SentenceTransformer(model_name, model_kwargs=model_kwargs, trust_remote_code=True)
queries = ["What kind of character is Hachiware?"]
docs = [
"Hachiware is a cat-like character from Chiikawa with a bright, sociable, and positive personality. Hachiware enjoys everyday life with Chiikawa and friends and eagerly joins adventures such as subjugation missions.",
"Usagi is a carefree, free-spirited character whose bold actions and sharp intuition pull Chiikawa and friends along, while still caring deeply about friends.",
]
q_emb = model.encode(queries, normalize_embeddings=True)
d_emb = model.encode(docs, normalize_embeddings=True)
scores = model.similarity(q_emb, d_emb)
print(scores)
These English 0.5B encoders start from weakly supervised checkpoints and undergo supervised fine-tuning on the supervised subsets of sentence-transformers/embedding-training-data. The first three rows are the main Encoder / Decoder / Decoder-PT comparison; the remaining rows vary the decoder configuration (Mistral, Qwen2.5 trained from scratch) or the pre-trained source model (Qwen2.5-0.5B, InstructLM-500M).
| ID | Architecture | #Param. | #Param. w/o Emb. |
MTEB-Avg | MTEB-Retrieval | MTEB-STS | MTEB-Classification | MTEB-Reranking | MTEB-Clustering |
|---|---|---|---|---|---|---|---|---|---|
| iamtatsuki05/Sentence-ModernBERT-EN-0.5B | ModernBERT | 677M | 546M | 53.66 | 40.32 | 76.04 | 67.31 | 42.93 | 41.69 |
| iamtatsuki05/Sentence-Llama-Bi-EN-0.5B | Llama | 661M | 530M | 52.12 | 37.31 | 74.10 | 66.28 | 42.29 | 40.63 |
| iamtatsuki05/Sentence-Sarashina-Bi-EN-0.5B | Llama | 661M | 530M | 56.54 | 42.62 | 78.91 | 72.45 | 45.39 | 43.35 |
| iamtatsuki05/Sentence-Mistral-Bi-EN-0.5B | Mistral | 518M | 469M | 52.70 | 38.44 | 75.04 | 66.60 | 42.53 | 40.87 |
| iamtatsuki05/Sentence-Qwen2.5-Scratch-Bi-EN-0.5B (this model) |
Qwen2.5 | 494M | 357M | 51.73 | 36.70 | 73.86 | 65.20 | 42.00 | 40.90 |
| iamtatsuki05/Sentence-Qwen2.5-Bi-EN-0.5B | Qwen2.5 | 494M | 357M | 57.19 | 44.18 | 79.48 | 71.92 | 45.23 | 45.15 |
| iamtatsuki05/Sentence-InstructLM-Bi-EN-0.5B | Mistral | 518M | 469M | 57.73 | 46.11 | 80.01 | 73.19 | 45.71 | 43.62 |
Scores are measured on the 37-task English subset of MTEB (eng, v2) that covers the same five task families as JMTEB. Each task-family score averages all datasets, splits, and subsets in the family, and MTEB-Avg is the macro average over the five families.
This model is distributed under the MIT License.
@article{MIREI
title={同一条件下における Encoder/Decoder アーキテクチャによる文埋め込みの性能分析},
author={岡田 龍樹 and 杉本 徹},
journal={言語処理学会第 32 回年次大会 (NLP2026)},
year={2026}
}