Vela 3.0 (vela3-fast, 0.4B) · Core ML

vllm-sr/vela3-fast (vLLM Semantic Router × KR Labs, Vela 3.0 family) converted to Core ML for Apple silicon: decisions (choice / yes-no over options you name) and text spans (17 types of personal information, words of an answer its context does not support) from one UltraBERT-380M encoder, on device.

from vela3_coreml import Vela3CoreML
m = Vela3CoreML(".")  # GPU by default
m.decide("My card was charged twice and I need this fixed today.",
         {"intent": {"type": "choice", "instructions": "What does the customer want?",
                     "criteria": {"refund": "Money back for the duplicate charge", "cancel": "Cancel the order"}},
          "urgent": {"type": "noul", "instructions": "The customer needs a fast response."}})
m.pii("Hi, I'm Dana Whitfield (dana@example.com, 312-555-0110).")   # [{label, start, end, text, score}, ...]
m.unsupported(context="Adults may take up to 4 g of paracetamol in 24 hours.",
              answer="Adults may take up to 6 g of paracetamol in 24 hours.")  # -> "6 g"

Fidelity

Against the fp32 PyTorch release on the same inputs (reports/verify.txt):

• Decisions: typed-decisions TEST, all 1,200 choice and yes/no questions: 0 answers differ, max probability difference 0.0026.
• Personal information: 25 varied passages (support tickets, clinic notes, HR reports, bank chats; 183 spans): identical span sets.
• Unsupported claims: identical spans on every context / answer pair tested.

On a 20,000-word synthetic case file with 2,971 planted values, the Core ML model catches 99.5 % (2,956), with 30 other flags.

Speed (MacBook Pro M5 Pro, GPU)

One encoder pass answers one question. The question (options, or the 17 PII label descriptions, ~250 tokens) counts toward the length, so a PII pass over a paragraph usually lands in L512: about 0.5 ms per word end to end. The Neural Engine is not used: it was slower at every length and drifted further from the fp32 answers (15 / 1,200 decisions flipped).

Scope and limits

• Covered: decisions on the decision backbone, and the two label sets of the span_sets adapter (17 PII types, unsupported). The release's other adapters (open-label NER, relations, evidence, router signals, domain, fact-check, feedback) are not converted; each retrains every layer, so each would be its own encoder package.
• The release's score question type is not part of the decision backbone and is not supported (as upstream).
• Inputs are limited to 1,024 tokens per encoder pass. Longer decision inputs raise Unsupported; longer span inputs are windowed at 1,024 tokens (the release uses 8,192), so long-document spans can differ.

Files and licences

Apache-2.0, as the source release (LICENSE). upstream/ is the release's own code, unmodified. The conversion: a fixed-length export of the encoder (local layers' 128-token sliding window as a band mask, the two RoPE bases precomputed, fp16-safe masks), the adapter's LoRA merged into plain weights, heads kept in fp32 on the host. conversion/ holds the export, packaging and verification scripts.