semantic-lite

Lightweight multilingual embedding model for retrieval and semantic similarity, built from Qwen3-0.6B via layer pruning (28 → 6 layers). 2.4× smaller than the base model while matching or beating dedicated embedding models on retrieval.

• Dimensions: 1024
• Parameters: 250M (pruned from 596M)
• Languages: inherits Qwen3's 100+ language coverage
• Max sequence: 8192 tokens

Installation

pip install sentence-transformers

Quick Start

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("ukung/semantic-lite")
embeddings = model.encode(["Hello world", "Halo dunia"])
# shape: (2, 1024)

---

Use Cases

1. Semantic Similarity

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("ukung/semantic-lite")

sentences = [
    "A man is playing a guitar.",
    "A person is playing a musical instrument.",
    "The stock market crashed today.",
]
embeddings = model.encode(sentences, normalize_embeddings=True)
similarity = embeddings @ embeddings.T
print(similarity[0, 1])  # 0.96  (similar)
print(similarity[0, 2])  # 0.79  (different)

2. Semantic Search

from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("ukung/semantic-lite")

corpus = [
    "How to cook fried rice",
    "Guide to stock market investing",
    "Tips for caring for your cat",
    "Python programming tutorial for beginners",
    "How to grow chili peppers in pots",
]
corpus_embeddings = model.encode(corpus, normalize_embeddings=True)

query = "Easy home cooking recipes"
query_embedding = model.encode(query, normalize_embeddings=True)

scores = query_embedding @ corpus_embeddings.T
best = int(np.argmax(scores))
print(corpus[best])  # "How to cook fried rice"

Tip: for short, abstract queries (e.g. "I want to learn coding"), use an asymmetric prompt for better precision:

query_emb = model.encode(query, prompt="query: ", normalize_embeddings=True)
doc_emb = model.encode(corpus, prompt="passage: ", normalize_embeddings=True)

3. Cross-Lingual Retrieval

model = SentenceTransformer("ukung/semantic-lite")

documents = [
    "How to bake a chocolate cake",
    "Cara membuat kue cokelat",
    "Comment faire un gâteau au chocolat",
    "Cómo hacer un pastel de chocolate",
    "The weather forecast for tomorrow",
]
doc_embeddings = model.encode(documents, normalize_embeddings=True)

query = "Resep kue cokelat"  # Indonesian query
query_embedding = model.encode(query, normalize_embeddings=True)

scores = query_embedding @ doc_embeddings.T
print(documents[int(np.argmax(scores))])  # "Cara membuat kue cokelat"

4. Clustering

from sentence_transformers import SentenceTransformer
from sklearn.cluster import KMeans

model = SentenceTransformer("ukung/semantic-lite")

texts = [
    "I love pizza and pasta",
    "Italian food is delicious",
    "Python is a great programming language",
    "I code in Python daily",
    "The cat is sleeping on the sofa",
    "My dog loves to play fetch",
]
embeddings = model.encode(texts, normalize_embeddings=True)
labels = KMeans(n_clusters=3, n_init=10, random_state=42).fit(embeddings).labels_
print(labels)  # [2 2 0 0 1 1]

5. Duplicate Detection

model = SentenceTransformer("ukung/semantic-lite")

sentences = [
    "How do I reset my password?",
    "What is the process to reset a password?",
    "Where can I buy a new laptop?",
]
embeddings = model.encode(sentences, normalize_embeddings=True)
similarity = embeddings @ embeddings.T
print(similarity[0, 1])  # 0.91  (duplicate)
print(similarity[0, 2])  # 0.85  (not duplicate)

6. Paraphrase Mining

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("ukung/semantic-lite")

corpus = [
    "What is the capital of France?",
    "Paris is the capital of France.",
    "How do I learn Python?",
    "Python is a programming language.",
    "The capital city of France is Paris.",
]
pairs = util.paraphrase_mining(model, corpus, top_k=3)
for score, i, j in pairs:
    print(f"{score:.3f}: {corpus[i]} <-> {corpus[j]}")

7. Retrieval-Augmented Generation (RAG)

model = SentenceTransformer("ukung/semantic-lite")

knowledge_base = [
    "Qwen3 is a family of large language models by Alibaba.",
    "Sentence-transformers is a Python library for embeddings.",
    "The Eiffel Tower is located in Paris, France.",
]
kb_embeddings = model.encode(knowledge_base, normalize_embeddings=True)

query = "What is Qwen3?"
query_embedding = model.encode(query, normalize_embeddings=True)

scores = query_embedding @ kb_embeddings.T
context = knowledge_base[int(np.argmax(scores))]
print(context)  # "Qwen3 is a family of large language models by Alibaba."

8. Recommendation

model = SentenceTransformer("ukung/semantic-lite")

items = [
    "Action movie with explosions",
    "Romantic comedy film",
    "Sci-fi space adventure",
    "Horror ghost story",
]
item_embeddings = model.encode(items, normalize_embeddings=True)

user_preference = "I enjoy science fiction and space"
user_embedding = model.encode(user_preference, normalize_embeddings=True)

scores = user_embedding @ item_embeddings.T
print(items[int(np.argmax(scores))])  # "Sci-fi space adventure"

9. Batch Encoding

model = SentenceTransformer("ukung/semantic-lite")

documents = [f"Document {i} about topic {i % 5}" for i in range(1000)]
embeddings = model.encode(
    documents,
    batch_size=32,
    normalize_embeddings=True,
    show_progress_bar=True,
)
print(embeddings.shape)  # (1000, 1024)

---

Multilingual Support

Inherits Qwen3's 100+ language coverage. Mean cosine similarity to English across 22 tested languages: 0.786 (21/22 above 0.6).

---

Specifications

Evaluation

Retrieval benchmark on an internal Indonesian corpus (20 documents, 8 queries):

Limitations

• Zero-shot classification is not supported. The model's embeddings are not calibrated for label-matching tasks. Use a dedicated zero-shot classifier instead.
• Short, abstract queries may require an asymmetric prompt (query: / passage:) for optimal retrieval precision.
• Evaluation is on a small internal corpus. Results on large public benchmarks (MTEB, BEIR) are not yet validated.

License

Apache 2.0 (inherited from Qwen3).