English / Japanese
Sentence-InstructLM-Bi-EN-0.5B-PT wraps iamtatsuki05/InstructLM-Bi-EN-0.5B with weakly supervised contrastive training on the weakly supervised subsets of sentence-transformers/embedding-training-data, producing 1,536-dimensional English sentence embeddings.
sentence-transformers>=4.1.0
transformers>=4.51.0
accelerate>=1.6.0
sentencepiece>=0.2.0
flash-attn>=2.7.3
import torch
from sentence_transformers import SentenceTransformer
model_name = "iamtatsuki05/Sentence-InstructLM-Bi-EN-0.5B-PT"
model_kwargs = {
"torch_dtype": torch.bfloat16,
"attn_implementation": "flash_attention_2",
}
model = SentenceTransformer(model_name, model_kwargs=model_kwargs, trust_remote_code=True)
queries = ["What kind of character is Hachiware?"]
docs = [
"Hachiware is a cat-like character from Chiikawa with a bright, sociable, and positive personality. Hachiware enjoys everyday life with Chiikawa and friends and eagerly joins adventures such as subjugation missions.",
"Usagi is a carefree, free-spirited character whose bold actions and sharp intuition pull Chiikawa and friends along, while still caring deeply about friends.",
]
q_emb = model.encode(queries, normalize_embeddings=True)
d_emb = model.encode(docs, normalize_embeddings=True)
scores = model.similarity(q_emb, d_emb)
print(scores)
The following English 0.5B encoders share the same weakly supervised recipe on the weakly supervised subsets of sentence-transformers/embedding-training-data (about 16.1M training pairs).
| ID | Architecture | #Param. | #Param. w/o Emb. |
|---|---|---|---|
| iamtatsuki05/Sentence-ModernBERT-EN-0.5B-PT | ModernBERT | 677M | 546M |
| iamtatsuki05/Sentence-Llama-Bi-EN-0.5B-PT | Llama | 661M | 530M |
| iamtatsuki05/Sentence-Sarashina-Bi-EN-0.5B-PT | Llama | 661M | 530M |
| iamtatsuki05/Sentence-Mistral-Bi-EN-0.5B-PT | Mistral | 518M | 469M |
| iamtatsuki05/Sentence-Qwen2.5-Scratch-Bi-EN-0.5B-PT | Qwen2.5 | 494M | 357M |
| iamtatsuki05/Sentence-Qwen2.5-Bi-EN-0.5B-PT | Qwen2.5 | 494M | 357M |
| iamtatsuki05/Sentence-InstructLM-Bi-EN-0.5B-PT (this model) |
Mistral | 518M | 469M |
This model is distributed under the MIT License.
@article{MIREI
title={同一条件下における Encoder/Decoder アーキテクチャによる文埋め込みの性能分析},
author={岡田 龍樹 and 杉本 徹},
journal={言語処理学会第 32 回年次大会 (NLP2026)},
year={2026}
}
iamtatsuki05/InstructLM-Bi-EN-0.5B