Vela-1.0-Encoder-307M-Embedding — Apple Core AI

A Core AI (.aimodel) build of vllm-sr/Vela-1.0-Encoder-307M-Embedding, the most-downloaded decision model in the vLLM Semantic Router family, for on-device inference on Apple Silicon.

Upstream shipped safetensors and onnx only. This is the Core AI build.

Verified equivalence

Measured against the original PyTorch model on four fixed inputs at the exported shape — not sampled, not estimated:

Worst absolute difference across all four: 2.31e-07 — float32 rounding, not a modelling difference. Cosine is exactly 1.0 to eight decimal places.

A semantic check the numbers alone cannot give: cosine between the two password sentences is 0.8124, between the password sentence and the weather sentence 0.0547. The embeddings are meaningful, not merely identical.

Arabic

The model card advertises ar. The first validation pass tested none of it, so a second pass added three Arabic sentences at the same fixed shape:

Equivalence holds in Arabic exactly as it does in English — worst 3.24e-07, float32 rounding.

And the model genuinely uses the language rather than merely being stable on it:

The embedding space is shared across the two languages, which is what makes cross-lingual search work: the English "How do I reset my password?" sits at 0.6922 from its Arabic counterpart and at 0.0547 from an unrelated English sentence.

What is in here

Use

import asyncio, numpy as np
from coreai.runtime import AIModel, NDArray, SpecializationOptions
from transformers import AutoTokenizer

SEQUENCE_LENGTH = 512          # the graph is fixed-shape; pad to it and read the vector back

async def main():
    tok = AutoTokenizer.from_pretrained(".")
    model = await AIModel.load("vela-embedding.aimodel", SpecializationOptions.default())
    fn = model.load_function("main")
    enc = tok(["How do I reset my password?"], padding="max_length",
              truncation=True, max_length=SEQUENCE_LENGTH, return_tensors="np")
    out = await fn({"input_ids": NDArray(enc["input_ids"].astype("int32")),
                    "attention_mask": NDArray(enc["attention_mask"].astype("int32"))})
    return np.asarray(list(out.values())[0].numpy())

print(asyncio.run(main()).shape)   # (1, 768)

Padding to the exported length is required: the graph is built for a fixed 512 tokens. Pad and slice rather than passing a shorter sequence.

How it was converted

The pipeline is mirrored from the model's own sentence-transformers config, which is the only way the result can be claimed equivalent:

modules.json -> Transformer -> 1_Pooling (mean tokens) -> 2_Normalize
1_Pooling/config.json -> pooling_mode_mean_tokens: true, word_embedding_dimension: 768

so embedding = L2_normalize( mean(hidden_states * attention_mask) ), computed in fp32 exactly as upstream ships it.

Export path: torch.export at a fixed (1, 512) shape, run_decompositions, then coreai-torch 0.4.3 to Core AI. The graph takes int32 inputs — the runtime accepts no other integer type.

Upstream's modernbert_sdpa_layout.py is deliberately not used: its own docstring states it changes storage layout only, to work around a ROCm efficient-attention miscompute, and that CPU and MATH paths are untouched by it.

Limits, stated plainly

• The exported graph is fixed at 512 tokens. Prompts of other lengths must be padded.
• Only text was measured, but both English and Arabic now are. Chinese, Japanese and Spanish, also advertised upstream, remain untested.
• No speed or memory benchmark was run. Only equivalence was measured.
• Matryoshka dimensions upstream supports were not exercised.

Licence and attribution

Apache-2.0, inherited from upstream. This conversion is a derivative work of vllm-sr/Vela-1.0-Encoder-307M-Embedding; all credit for the model is theirs.