vllm-sr/vela3-fast

Name it. Vela finds it.
_{vLLM Semantic Router × KR Labs · Vela 3.0 family · vela3-fast · 0.4B · UltraBERT-380M encoder · Apache-2.0}

What it does · Quick start · Evaluation · The family · 📡 Collection

---

vela3-fast is the 380M encoder of the Vela 3.0 family. Each question is one encoder pass, and it gives decisions, router signals and the exact spans you name (entities, relations, personal data, unsupported claims, evidence), with probabilities and character offsets and no generated text.

Summary

vela3-fast is the default router of the Vela 3.0 family: a 380M UltraBERT encoder with its own decision backbone, distilled from the d3 decision models, and gated adapters on top for spans, entities, relations, evidence and router signals. Its decisions have the highest public Decision Index of any model up to 500M parameters: 50.36 (our run of the unmodified kit, edition 0.3). One encoder pass per question: 14 ms for a decision at 200 tokens. English only, up to 32K tokens.

Highlights

• Decisions: the highest public Decision Index of any model up to 500M parameters: 50.36 (our run of the unmodified kit, edition 0.3, public index). Next up to 500M: Amber 0.4B 49.74 and GLiClass v4 Edge (0.4B) 47.49 (board entries, Decision Index 0.4.1, public index); 15.6 points above GLiClass v4 Edge on Tools & Automation.
• Router signals: better than Vela 2.0 0.3B on 13 of the 24 test files of the public router signal suite, level on 6, worse on 5 (paired 95% CI).
• Zero-shot classification: 74.4 on the 13-set GLiFormer classification suite, above GLiFormer-base (264M): 72.36 and Qwen3.5-9B: 71.4 (reported, GLiFormer paper Tab 5).
• Named entities: 57.4 on the 26 NER sets of the GLiFormer paper (not all zero-shot), above GLiFormer-large (576M): 50.91 (reported, its card); 62.3 on OOD-7 zero-shot.
• Unsupported claims: 0.889 example-F1 on the 10,698-answer LettuceDetect v2 test set, above LettuceDetect v2 mmBERT-base (~307M): 0.869 (reported, its card).
• Speed: 14 ms for a decision and 18 ms for a span question at 200 tokens (batch 1, one AMD Instinct MI325X).

_{vela3-fast (0.4B) against reported numbers. Accent = vela3-fast; grey = other models (reported).}

At a glance

What it does

Quick start

# pip install "transformers==5.17.0" torch safetensors tokenizers huggingface_hub   (as requirements.txt)
import sys
from huggingface_hub import snapshot_download

path = snapshot_download("vllm-sr/vela3-fast")
sys.path.insert(0, path)
from vela3_encoder import Vela3Encoder

m = Vela3Encoder.load(path)  # CUDA if available, else CPU
out = m.system_one(
    "Steve Jobs founded Apple in Cupertino. My card was charged twice.",
    {"intent": {"type": "choice", "instructions": "What does the customer want?",
                "criteria": {"refund": "Money back for the duplicate charge", "cancel": "Cancel the order"}},
     "urgent": {"type": "noul", "instructions": "The customer needs a fast response."},
     "ents": {"type": "span", "instructions": "Which spans are named entities of these types?",
              "criteria": {"person": None, "organization": None}},
     "founder": {"type": "span", "instructions": "Which spans are the founder of Apple?", "criteria": {"founder": None}}})
# out["answers"]["intent"]: choice + probabilities; ["urgent"]: p(yes); ["ents"] / ["founder"]["spans"]: label, start, end, text, score

Over HTTP (/v1/systemone, request body {"state", "questions"}; a question over 32,768 tokens gets a 422, nothing is cut):

python vela3_server.py --model vllm-sr/vela3-fast --port 8000

Decision Index (decision_index_engine:VelaEncoderEngine, the decision backbone only):

hf download vllm-sr/vela3-fast --local-dir model
pip install -r model/requirements.txt "decision-index @ git+https://github.com/apolinario/decision-index@25c6ec7"
PYTHONPATH=model python -m decision_index run --engine decision_index_engine:VelaEncoderEngine --option run=model --edition 0.3 --out runs/vela3-fast --compact

Package size and decisions-only use

The whole package is 1.65B parameters and 6.6 GB. Each question type is its own pass, and an adapter switches on only for its own question, so a full router call is 6-7 passes. For decisions only, skip the adapters; the output is identical to the full Vela3Encoder:

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("vllm-sr/vela3-fast", ignore_patterns=["*_adapter.safetensors"])
sys.path.insert(0, path)
from vela3_encoder import DecisionEncoder

m = DecisionEncoder.load(path)  # decisions only: 0.38B, 1.5 GB

The vllm_srun runtime currently loads the whole package.

Evaluation

All numbers on public test sets, measured through this package with system_one, one request at a time and the shipped thresholds. Other models' numbers are as reported by their authors (or scored in the same harness where marked). Bold: above every comparator in the row. ✦: the best reported number at ≤500M. Full tables: EVALUATION.md.

Named entities

Label names are the only input (no definition writer). Training holds no CrossNER or MIT text, though many generic CrossNER type names occur as training labels.

Relations Evidence

QASPER is in-domain: the evidence adapter trains on QASPER's train split (with CUAD, ContractNLI and Natural Questions); the test papers are held out.

Unsupported claims (hallucination spans) Zero-shot classification Router signals (public router signal suite; above Vela 2.0 0.3B on 4 of 8)

vllm-sr/router-signal-suite (public), the test file of each signal, scored with the dataset's own scoring code. Each signal is a router question this package was trained on (supervised, as in Vela 2.0), asked one request at a time with the shipped thresholds.

Vela 2.0 0.3B was run by us through its public release on the same rows. Decision-1.0-Route-0.6B is scored from the suite's published predictions; it was trained on the suite's own train split, so these test files are in-distribution for it.

Bold: the paired 95% bootstrap CI against Vela 2.0 excludes a tie. The last column drops the test rows that share text (exact or a 13-gram) with the adapters' training data and the earlier decision backbone's data; the public train splits and synthetic data added to this decision backbone were not in that scan. Safety and jailbreak are scored on the rows whose text the suite publishes (2,230 of 3,003 and 1,878 of 3,000).

Feedback is also trained on the suite's own train split (with the public router data Vela 2.0 used; the suite's test and hold rows, and anything sharing an exact text or a 13-gram with them, are kept out), as Decision-1.0-Route is, so its test files are in-distribution for both. The suite's domain classes are drawn differently from the trained ones (religion, other). Feedback has NO_FEEDBACK as a trained option.

Decisions (the encoder's own distilled decision backbone)

Full score (the board's ranking): 60% private same-skill tests and 40% private new-domain tasks, which only the board maintainers run; public index: the 37 public benchmarks, chance-corrected. All rows are board entries on the Decision Index 0.4.1 board (multimodalart/jev-decision-index, data generated 2026-10-28T09:02Z); the rank is the position in the board's Full-score order, and tied rank marks entries the board shows as tied. Among board entries up to 500M parameters with a published size, vela3-fast has the highest public index and the second Full score, after GLiClass v4 Edge. Its board public index equals our own run of the unmodified kit (apolinario/decision-index, edition 0.3; engine decision_index_engine:VelaEncoderEngine, 32K context), which gives the per-benchmark numbers below. Above 500M, StartLux-Decision v1.1 0.8B (873M) has a higher public index, 53.79 (Full score 28.88).

Latency

Decisions run under bf16 autocast; adapter passes run in fp32, as evaluated.

Benchmarks

• OOD-7 (CrossNER AI, literature, music, politics, science; MIT movie, restaurant): find the entities of the named types in text from domains the model was not trained on (strict entity F1).
• NER, 26 datasets of the GLiFormer paper (IE_INSTRUCTIONS): entities of named types across 26 public NER sets, not all zero-shot (strict entity F1).
• CoNLL04, head entity given: given one entity of a sentence, find the entities it is related to by the named relation (F1 over those tail entities).
• Relations end to end, GLiFormer Table 4 protocol (CoNLL04, DocRED, FewRel, CrossRE): find the entity pairs and their relation from raw text (strict triplet F1).
• QASPER: find the sentences of a research paper that answer a question about it (word-F1).
• ACL-Verbatim (KRLabsOrg/acl-verbatim-spans): find the passage of an ACL paper that supports an answer (word-F1).
• LettuceDetect v2 test set (10,698 answers, as in the LettuceDetect v2 paper (GroundLM 2026 workshop at EMNLP 2026) and card): does an answer contain claims its context does not support (example-F1: the answer is flagged or not).
• RAGTruth: does a retrieval-augmented answer contain claims its retrieved context does not support (example-F1).
• MASSIVE, three unseen scenarios (iot, transport, cooking): pick the intent and the scenario of a user command the model never saw in training (accuracy).
• Zero-shot classification, 13 sets (the GLiFormer paper's suite: SST-2, IMDB, AG News, Banking77, and more): pick the label of a text from label names only (mean macro-F1).
• Decision Index 0.4.1 (board, 2026-10-28): 37 public decision benchmarks plus private tests; the Full score ranks on the private same-skill and new-domain tests (chance-corrected skill, 0 = guessing). The public part is the suite of kit edition 0.3, which we also ran ourselves.
• Router signal suite (vllm-sr/router-signal-suite): the vLLM Semantic Router's public test of its request signals (harmful requests, jailbreaks, personal data, unsupported answers, fact-check need, modality, domain, feedback); each label comes from public corpora (AUC or accuracy).

The family

How it works

• Frozen backbone. UltraBERT-380M, fine-tuned as a decision model: 24 layers, hidden size 1024, sliding-window and global attention, 32K context. Each decision is one encoder input, [Q] instructions [O] option … [SEP_SCHEMA] [SEG_user] state, scored at its [O] markers.
• Gated adapters. LoRA adapters with their own top layers and heads run on the frozen backbone and share its weights. Each switches on only for its own questions, and every answer names the one it used (head). The adapters cover:
  • span: described labels and the entity step of the relation pipeline;
  • evidence: a routed adapter for evidence questions over long documents;
  • a second span adapter for unsupported claims and PII;
  • NER (names-only entity labels);
  • relations;
  • router (safety, modality, MASSIVE);
  • task adapters for domain, fact-check and feedback.
• Spans as decisions. The labels are presented as options. A span head scores every word against every label, and words above the threshold form a span with character offsets. Evidence questions read the whole document in one window of up to 32K tokens, through the evidence adapter.
• Specification. Text or JSON state, English, up to 32K tokens; label names only. Full evaluation: EVALUATION.md.

Trade-offs

• Router signals below Vela 2.0 0.3B on the public suite: fact-check needed 0.657 against 0.755; harmful requests 0.933 against 0.945 (same rows).
• English only: MASSIVE intent accuracy (test split, three unseen scenarios) falls from 0.820 in en-US to 0.44–0.54 in German, French, Spanish and Italian, and lower in other languages. For multilingual routing use vela3-lite.
• Label names only: no definition writer, so OOD-7 (62.3) sits just below gliner-multitask-large-v0.5 (62.76, reported).

Citation

@misc{vela3_fast,
  title        = {vela3-fast (Vela 3.0, 0.4B): decisions, spans and router signals in one UltraBERT-380M encoder},
  author       = {{KR Labs} and {vLLM Semantic Router}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/vllm-sr/vela3-fast}}
}