Octen-Embedding-0.6B for MLX (4-bit)

Octen/Octen-Embedding-0.6B converted to MLX so it runs natively on Apple Silicon: 4-bit affine quantization, group size 64, 0.3 GB on disk. The model embeds text in 100+ languages into 1024 dimensions, for search, retrieval, clustering and similarity. It is a 0.6B-parameter Qwen3 model fine-tuned for retrieval, with a 32K-token context.

Quality note. This is the smallest variant and it is visibly less faithful: embeddings move to a mean cosine of 0.95 from the original, and top-1 retrieval drops by 3.5 points in the test below. Use it when memory is the constraint, check it on your own data, and prefer the 8-bit or bf16 variant otherwise.

This is a community conversion, not an official Octen release. The weights are Octen's, fine-tuned from Qwen/Qwen3-Embedding-0.6B. Both are Apache 2.0, and so is this conversion. Credit for the model goes to the Octen and Qwen teams.

The three variants

The three sizes

The last column is the time to embed the 178 tickets and 140 articles of the test below, 8 texts per batch, up to 1,024 tokens each, on an M3 Max with 48 GB, measured on 21 October 2026 and given as a range over the three variants. Quantization saved disk and memory and did not make embedding faster in these runs. Larger models retrieve better on the test below: top 1 is 67.1% at 0.6B, 72.4% at 4B and 78.2% at 8B (MLX bf16).

Quick start

pip install mlx-embeddings

import mlx.core as mx
from mlx_embeddings import load

model, tokenizer = load("juliensimon/Octen-Embedding-0.6B-mlx-4bit")

texts = [
    "How many days of webhook logs can I see on the Business plan?",
    "Business accounts can view up to 90 days of webhook logs.",
    "Wie lange werden Webhook-Protokolle gespeichert?",
    "Pasta carbonara needs eggs, pecorino and guanciale.",
]
inputs = tokenizer(texts, padding=True, truncation=True, max_length=1024, return_tensors="mlx")
embeddings = model(inputs["input_ids"], attention_mask=inputs["attention_mask"]).text_embeds  # (4, 1024)
embeddings = embeddings.astype(mx.float32)
embeddings = embeddings / mx.linalg.norm(embeddings, axis=1, keepdims=True)  # the model computes in bf16: renormalize in float32
mx.eval(embeddings)
print((embeddings @ embeddings.T).round(3))  # cosine similarities: related texts score high, the carbonara line scores about 0.1

The embedding is the last token's hidden state, normalized, as in the original model. The original model card asks for no instruction prefix, so queries and documents are embedded the same way.

Search across languages

import mlx.core as mx
from mlx_embeddings import load

model, tokenizer = load("juliensimon/Octen-Embedding-0.6B-mlx-4bit")


def embed(texts):
    inputs = tokenizer(texts, padding=True, truncation=True, max_length=1024, return_tensors="mlx")
    e = model(inputs["input_ids"], attention_mask=inputs["attention_mask"]).text_embeds.astype(mx.float32)
    e = e / mx.linalg.norm(e, axis=1, keepdims=True)
    mx.eval(e)
    return e


articles = [
    "Webhook logs: Starter accounts can view up to 7 days of logs, Team up to 30 days, Business up to 90 days.",
    "To create an invoice, select Invoices, click New invoice and choose the customer.",
    "Harborline retries a failed webhook delivery three times, after 10 seconds, 1 minute and 5 minutes.",
]
queries = [
    "Combien de jours de journaux de webhook puis-je consulter avec le plan Business ?",
    "Hoe maak ik een nieuwe factuur aan?",
]
scores = embed(queries) @ embed(articles).T  # (2, 3): one row per query
for q, row in zip(queries, scores.tolist(), strict=True):
    best = max(range(len(articles)), key=row.__getitem__)
    print(f"{q}\n  -> {articles[best][:60]}... ({row[best]:.2f})")

A French question finds the English webhook article, and a Dutch question finds the invoice article.

How faithful is the conversion

Run on 20 October 2026. The test: 170 support tickets in English, French, Dutch and German, each written from one of 140 English help articles (the development set of a workshop on fine-tuning small language models). The question is whether the ticket's own article is among the top k articles retrieved. The reference is the original PyTorch model in bf16 on CPU.

bf16 and 8-bit match the original within one ticket at every k. One benchmark on one language mix is not a general claim: check retrieval quality on your own data before you rely on it. If you store embeddings, embed documents and queries with the same variant.

Reproduce the conversion

mlx_embeddings.convert (mlx-embeddings 0.1.0, MLX 0.32.4) crashes on this model while re-saving tokenizer.json, a read-only file it copied from the Hub cache. This short wrapper copies files without their read-only mode:

import importlib, os, shutil

c = importlib.import_module("mlx_embeddings.convert")
_copyfile = shutil.copyfile


def plain_copy(src, dst, **_):
    dst = os.path.join(dst, os.path.basename(src)) if os.path.isdir(dst) else dst
    return _copyfile(src, dst)


c.shutil.copy = plain_copy
c.convert("Octen/Octen-Embedding-0.6B", mlx_path="out", dtype="bfloat16", quantize=True, q_bits=4, q_group_size=64)

The sentence-transformers files of the original (modules.json and the like) are left out, because this repository holds MLX weights.

Limitations

• The same limits as the original model apply. See its card.
• Quantization changes embeddings a little, and more at 4 bits. Do not mix variants inside one index.
• The test above is mine and small. It measures fidelity to the original and retrieval on one kind of text.

Citation

Please cite the Qwen3 Embedding work this model builds on:

@article{qwen3embedding,
  title={Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models},
  author={Zhang, Yanzhao and Li, Mingxin and Long, Dingkun and Zhang, Xin and Lin, Huan and Yang, Baosong and Xie, Pengjun and Yang, An and Liu, Dayiheng and Lin, Junyang and Huang, Fei and Zhou, Jingren},
  journal={arXiv preprint arXiv:2506.05176},
  year={2025}
}