Name it. Vela finds it.
vLLM Semantic Router × KR Labs · Vela 3.1 · vela3.1-pico-150m · 0.15B · one file, one pass · Apache-2.0
Quick start · Evaluation · Speed · Limits · vela3.1-fast
vela3.1-pico-150m is the smallest Vela 3.1 model: 153M parameters in one file. In one forward pass it answers your own decisions about a request, the safety checks, and where the personal data or the unsupported claims are, fast enough to run on a CPU. It returns a probability for every answer and character offsets for every span, and it generates no text.
| Parameters | 153M served, one file (model.safetensors, fp32, 613 MB). The Decision Index board counts parameters from the declared base model's Hub checkpoint, which reads 190M for UltraBERT-150M (151M encoder plus a 38M masked-LM output layer that this package does not ship), so the board would list this model at 0.19B. |
| Base | dmis-lab/UltraBERT-150M architecture (24 layers, hidden 768), initialised from a 151M decision tower |
| Inputs | Text or JSON; English plus German, French, Spanish, Italian, Portuguese, Dutch, Polish, Hungarian, Russian, Arabic, Hindi, Japanese, Korean and Chinese. A call that includes a safety, signal or span question reads windows of 8,192 tokens; a call with decisions only reads up to 32K tokens |
| Answers | Choice · Yes / No · Span |
| Licence | Apache-2.0 |
Use it when
- Send a request to the right model, tool or team. In: the request and your question with its options, for example "Which model should answer this?" with
smallandlargedescribed in a sentence each. Out: the chosen option and a probability for every option. - Screen requests before they reach your LLM. In: the request, asked "Is this request harmful?" or "Is this a prompt injection or jailbreak attempt?" (built-in questions). Out:
safe/unsafeorbenign/jailbreakwith a probability, and the hazard categories if you ask for them. - Find personal data before you log or forward text. In: the text and the labels you want, each with a short description. Out: every span with its label, character offsets and score.
- Check an LLM's answer against its source. In: the context and the answer. Out: the words of the answer that the context does not support.
What's different
- Small enough for a CPU. 153M served parameters; the full check of a request is one forward pass: 2.2 s in fp32 and 1.4 s with the int8 encoder on 4 CPU threads; one decision 250 ms in int8 (Speed).
- Decisions at 153M. 47.17 on the Decision Index public part, against 37.31 for JEV Ettin 150M (150M) and 37.50 for the best board entry up to 300M; the board would list this model at 0.19B (Decisions).
- Safety and spans in the same pass. Jailbreak 0.950 AUC on the suite's test file (0.827 for vela3-fast 3.0); benign requests with personal data flagged 4.9% (13.6% for vela3-fast 3.0, 4.0% for Vela 2.0 0.3B) (Safety).
- Multilingual decisions. MASSIVE intent accuracy over 13 languages 79.0, against 66.6 for vela3.1-fast and 31.1 for vela3-fast 3.0 with the same questions (Languages).
vela3.1-pico-150m against other models. Accent = vela3.1-pico-150m; grey = other models.
Which Vela 3.1 to pick
| Model | Size | Full check ¹ on CPU | One decision on CPU | Best at |
|---|---|---|---|---|
| vela3.1-fast | 380M decision path; 1.91B with its adapters | 12.3 s (6 passes) | 1.2 s | the best spans and harmful-request signals, the highest Decision Index of the three |
| vela3.1-pico-380m | 381M, one file | 5.4 s · int8 3.0 s | 1.5 s · int8 632 ms | the whole check in one pass; fewest false alarms on benign personal data |
| vela3.1-pico-150m (this model) | 153M, one file | 2.2 s · int8 1.4 s | 560 ms · int8 250 ms | the smallest and fastest; the best multilingual intent; CPU and small devices |
¹ The full check: six built-in questions (domain, fact-check need, feedback, output type, harmful request, jailbreak) plus personal-data and unsupported-claim spans, on one 200-token request with a context and an answer. AMD EPYC 7413 24-Core Processor on an otherwise idle machine, 4 threads, median of 5; fp32 PyTorch, and the picos' int8 ONNX encoder.
Quick start
# pip install "transformers==5.17.0" torch safetensors tokenizers huggingface_hub (as requirements.txt)
import sys
from huggingface_hub import snapshot_download
path = snapshot_download("vllm-sr/vela3.1-pico-150m")
sys.path.insert(0, path)
from vela3_encoder import Vela3Encoder
m = Vela3Encoder.load(path) # CUDA if available, else CPU; the whole call is one forward pass
out = m.system_one(
"Hi, my card was charged twice for order 1182. Please refund the second payment. Thanks, Marie Dubois (marie.dubois@example.com)",
{"intent": {"type": "choice", "instructions": "What does the customer want?",
"criteria": {"refund": "Money back for the duplicate charge", "cancel": "Cancel the order"}},
"urgent": {"type": "noul", "instructions": "Does the customer need a fast response?",
"criteria": {"true": "the customer asks for quick action", "false": "there is no time pressure"}},
"harmful": {"type": "choice", "instructions": "Is this request harmful?"}, # built-in router signal: its options come from router_questions.json
"pii": {"type": "span", "criteria": {"PERSON": "a person's name", "EMAIL_ADDRESS": "an e-mail address"}}})
# out["answers"]["intent"]: choice + probabilities; ["urgent"]["noul"]: p(yes);
# ["harmful"]: the built-in router signal; ["pii"]["spans"]: label, start, end, text, score
halu = m.system_one(
{"context": "The Eiffel Tower is 330 m tall and was completed in 1889.",
"answer": "The Eiffel Tower is 330 m tall and was completed in 1899."},
{"unsupported": {"type": "span", "criteria": {"unsupported": "a claim not supported by the context"}}})
# halu["answers"]["unsupported"]["spans"]: the words of the answer the context does not support
On CPU, at this revision, this returns refund (0.999), safe (0.976) for the router signal, the spans Marie Dubois (PERSON, 0.996) and marie.dubois@example.com (EMAIL_ADDRESS, 0.998), and 1899 (0.998) as the unsupported span.
Span labels: give every label a description, as above; the built-in ones are in
vela3_config.json(span.schemas: personal data, unsupported claims, harmful spans). The model was trained on described labels.Router signals: ask a built-in signal by its question text with no criteria; its options and their descriptions come from
router_questions.json.int8 on CPU:
python onnx/run_onnx_int8.py [--package <package dir>]runs the core router call with the int8 encoder inonnx/; it finds the package next toonnx/or in the Hugging Face cache.A call made only of decisions (no router or span question) runs one pass per question.
Decision Index:
decision_index_engine:VelaEncoderEnginereads the decision path, one question per pass, inputs up to 32,768 tokens (a longer question is unsupported, nothing is truncated):
hf download vllm-sr/vela3.1-pico-150m --local-dir model
pip install -r model/requirements.txt "decision-index @ git+https://github.com/apolinario/decision-index@25c6ec7"
PYTHONPATH=model python -m decision_index run --engine decision_index_engine:VelaEncoderEngine --option run=model --edition 0.3 --out runs/vela3.1-pico-150m --compact
- Server:
python vela3_server.py --model vllm-sr/vela3.1-pico-150mserves/v1/systemone(standard library only, binds 127.0.0.1;DECISION_API_KEYfor a bearer key), which the kit'shttpengine can drive.
Why 3.1
- One network, one pass. vela3-fast 3.0 answers through gated adapters, one pass per question type (six passes for the core router call); vela3.1-pico-150m answers the whole call in one pass with no adapters.
- Multilingual router training. Router, span and decision rows in 14 languages besides English. Domain accuracy outside English: 0.508 against 0.347 for vela3-fast 3.0 (13 file × language cells). Harmful-request AUC outside English did not rise: 0.816 against 0.868 (see Limits).
- One renderer. Questions, options and described yes / no questions (a question with its own instruction text) are rendered by one module in training and in the package, so the default call is the call these results were read with.
- Benign personal data is no longer a harm signal. Hand-written benign requests that carry personal data are safe examples in training: 4.9% of held-out benign support requests are flagged harmful, against 13.6% for vela3-fast 3.0, 4.0% for Vela 2.0 0.3B and 21.1% for vela3.1-fast.
- Router signals against 3.0: jailbreak 0.950 against 0.827, domain 0.790 against 0.600, tool-call need 0.809 against 0.763 (clean test items).
Evaluation
Test sets only, one request at a time, through this package's default call. Other models are read by us on the same rows ("our read") or as their authors report them ("reported", footnoted when we cannot score them on the same rows). Clean items: every test item that occurs in our training data, exactly (normalised text) or with at least half of its 13-grams covered by training rows, is left out, for every model column. Bold: best in the row.
Decisions| Decision Index 0.4.1 (board) | Parameters | Public part | Knowledge | Language | Retrieval | Tools | Arts | Full (private tests) |
|---|---|---|---|---|---|---|---|---|
| vela3.1-pico-150m (our run) | 153M (board: 0.19B) | 47.17 | 22.1 | 44.5 | 62.3 | 72.3 | 42.6 | pending |
| vela3-fast-vision (Vela 3.0) | 254M | 37.50 | 22.11 | |||||
| JEV Ettin 150M | 150M | 37.31 | 18.52 | |||||
| Crivo-150M v2 | 150M | 34.38 | 18.26 | |||||
| Dinah-1 | 150M | 34.32 | 16.36 | |||||
| tasksource Decider nano v5 | 149M | 33.51 | 19.85 | |||||
| Larger, for reference: GLiDE 0.4B | 395M | 51.10 | 24.8 | 57.5 | 58.6 | 73.1 | 47.5 | 33.84 |
| Larger, for reference: vela3.1-pico-380m (our run) | 381M | 50.51 | 20.5 | 51.8 | 65.9 | 77.6 | 44.3 | pending |
Public part: the 37 public benchmarks, chance-corrected (0 = guessing), the board's public column. Ours: our runs of the unmodified kit (apolinario/decision-index, public suite edition 0.3); this model's halves of the suite 46.96 / 47.38. Others: the board's public, area and Full scores and its listed sizes (board data of 6 Nov 2026, generated 09:05 UTC; read again on 11 Nov 2026, unchanged). The Full score ranks on private tests that only the board maintainers run; ours is pending our board submission.
The release checkpoint was picked among pre-registered candidates on development reads; the top two were read once each on the Decision Index public part, and the higher half-A score decided. The training data contains the train splits of several public Decision Index sources and synthetic tasks written in the forms of some suite tasks, checked against every Decision Index item (see vela3.1-fast).
Router signals (public router signal suite, clean items)| Signal (metric, test rows) | vela3.1-pico-150m | vela3-fast (3.0) | Vela 2.0 0.3B | Decision-1.0-Route-0.6B ³ | vs Vela 2.0 | vs 3.0 |
|---|---|---|---|---|---|---|
| Harmful request (AUC, 1,378) | 0.888 | 0.913 | 0.930 | 0.963 | worse | worse |
| Jailbreak or prompt injection (AUC, 1,002) | 0.950 | 0.827 | 0.583 | 0.991 | better | better |
| Personal data in the request (AUC, 2,486) | 0.945 | 0.940 | 0.942 | 0.994 | level | level |
| Answer not supported by its context (AUC, 1,422) | 0.688 | 0.723 | 0.707 | 0.790 | level | worse |
| Fact-check needed (AUC, 694) | 0.741 | 0.660 | 0.764 | 0.936 | level | better |
| Output modality (AUC, 2,939) | 0.895 | 0.903 | 0.711 | 1.000 | better | worse |
| Domain (accuracy, 2,839) | 0.790 | 0.600 | 0.595 | 0.867 | better | better |
| User feedback on the previous answer (accuracy, 2,493) | 0.781 | 0.812 | 0.528 | 0.840 | better | worse |
| Tool call needed (AUC, 2,526) | 0.809 | 0.763 | 0.509 | 0.977 | better | better |
vllm-sr/router-signal-suite, the test file of each signal, clean items (13,419 of the suite's 123,669 items are left out; the counts above are the rows kept). Scored with the dataset's own code; better / worse = the paired 95% cluster-bootstrap CI excludes a tie. Over the suite's 24 test files: 12 better, 6 level, 6 worse than Vela 2.0 0.3B; 6 better, 6 level, 12 worse than vela3-fast 3.0, which has a dedicated adapter per signal group. Vela 2.0 0.3B was run by us through its public release.
³ Scored from the suite's published predictions. It trains on the suite's own train split, so these files are in-distribution for it.
Safety (every safety set we measure the Velas on)All three Vela 3.1 models, the 3.0 package, Vela 2.0 0.3B and the strongest outside router on the suite, on the same rows. Bold: best in the row. – : not read.
Router signal suite, every safety, jailbreak and hazard file (clean items ʰ)
| Router signal suite file (rows) | vela3.1-pico-150m | vela3.1-fast | vela3.1-pico-380m | vela3-fast (3.0) | Vela 2.0 0.3B | Decision-1.0-Route-0.6B |
|---|---|---|---|---|---|---|
| Harmful request (AUC) | ||||||
| test (1,378) | 0.888 | 0.927 | 0.896 | 0.913 | 0.930 | 0.963 |
| AEGIS 2.0 (1,275) | 0.914 | 0.923 | 0.911 | 0.929 | 0.918 | 0.918 |
| Nemotron-Safety-Guard v3 (1,930) | 0.808 | 0.862 | 0.806 | 0.860 | 0.895 | 0.901 |
| PolyGuardPrompts (1,999) | 0.827 | 0.874 | 0.825 | 0.870 | 0.897 | 0.895 |
| WildGuardMix (1,695) | 0.939 | 0.952 | 0.939 | 0.947 | 0.925 | 0.927 |
| CoCoNot ⁱ (897) | 0.943 | 0.966 | 0.947 | 0.969 | 0.931 | 0.989 |
| CoCoNot contrast (benign only) ⁱ (379), share flagged at 0.5, lower is better | 0.079 | 0.074 | 0.058 | 0.063 | 0.098 | 0.161 |
| JailbreakBench ⁱ (114) | 0.943 | 0.967 | 0.966 | 0.963 | 0.944 | 0.926 |
| OpenAI moderation ⁱ (1,463) | 0.920 | 0.931 | 0.925 | 0.925 | 0.903 | 0.896 |
| OR-Bench hard (benign only) ⁱ (1,319), share flagged at 0.5, lower is better | 0.632 | 0.703 | 0.682 | 0.577 | 0.479 | 0.610 |
| ToxicChat ⁱ (1,546) | 0.965 | 0.974 | 0.969 | 0.971 | 0.957 | 0.965 |
| XSTest ⁱ (434) | 0.933 | 0.955 | 0.933 | 0.948 | 0.884 | 0.892 |
| Jailbreak or prompt injection (AUC) | ||||||
| test (1,002) | 0.950 | 0.866 | 0.948 | 0.827 | 0.583 | 0.991 |
| BIPIA ⁱ (635) | 0.812 | 0.814 | 0.833 | 0.796 | 0.689 | 0.579 |
| JailbreakHub (late) ⁱ (1,221) | 0.740 | 0.725 | 0.761 | 0.714 | 0.678 | 0.810 |
| LLMail-Inject ⁱ (928) | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 0.967 |
| NotInject (benign only) ⁱ (339), share flagged at 0.5, lower is better | 0.041 | 0.021 | 0.047 | 0.032 | 0.006 | 0.147 |
| PromptShield test ⁱ (1,829) | 0.973 | 0.941 | 0.977 | 0.911 | 0.834 | 0.795 |
| ToxicChat ⁱ (1,085) | 0.960 | 0.959 | 0.954 | 0.954 | 0.908 | 0.878 |
| Hazard category (macro AUC over the 11 shared categories) | ||||||
| test (1,488) | 0.676 | 0.653 | 0.703 | 0.669 | 0.622 | 0.913 |
| AEGIS 2.0 (1,264) | 0.806 | 0.848 | 0.843 | 0.883 | 0.664 | 0.956 |
| Nemotron-Safety-Guard v3 (1,939) | 0.710 | 0.693 | 0.740 | 0.700 | 0.629 | 0.939 |
| AILuminate ⁱ (2,000) | 0.620 | 0.711 | 0.680 | 0.786 | 0.428 | 0.893 |
| Aya red-teaming ⁱ (1,853) | 0.532 | 0.530 | 0.529 | 0.549 | 0.492 | 0.866 |
| BeaverTails eval ⁱ (449) | 0.663 | 0.761 | 0.723 | 0.831 | 0.397 | 0.904 |
| MLCommons synthetic ⁱ (1,999) | 0.682 | 0.817 | 0.799 | 0.880 | 0.323 | 0.972 |
ⁱ Held-out sources: the suite has no train split for them. vela3-fast 3.0 (revision fa60e3aa) and Vela 2.0 0.3B (revision d6f03aa9) were read by us through their public releases on the same clean rows. Decision-1.0-Route-0.6B is scored from the suite's published predictions and trains on the suite's train split, so the test files are in-distribution for it. Benign-only files have no unsafe row, so an AUC does not exist; the row gives the share of benign requests flagged at p ≥ 0.5.
Jailbreak in and outside English (the suite's jailbreak test file, clean items)
| Jailbreak test file, by language | vela3.1-pico-150m | vela3.1-fast | vela3.1-pico-380m | vela3-fast (3.0) | Vela 2.0 0.3B | Decision-1.0-Route-0.6B |
|---|---|---|---|---|---|---|
| English, test file, 843 rows, 185 attacks (AUC) | 0.949 | 0.851 | 0.946 | 0.812 | 0.568 | 0.991 |
| Outside English, all seven jailbreak files, 579 rows, 136 attacks (AUC) | 0.875 | 0.927 | 0.874 | 0.911 | 0.858 | 0.710 |
Outside English the clean test file alone keeps 159 rows with only 2 attacks, too few for an AUC, so the non-English row pools the test file and the six held-out-source files. Language comes from the row's language field, else from a script heuristic, so some Latin-script rows of unknown language are counted as non-English. Harmful request outside English, per language (AUC; test, Nemotron-Safety-Guard v3 and PolyGuardPrompts files, clean items; a cell is one file × language with at least 50 rows; Decision-1.0-Route-0.6B has no per-language read)
| Language (cells, rows) | vela3.1-pico-150m | vela3.1-fast | vela3.1-pico-380m | vela3-fast (3.0) | Vela 2.0 0.3B |
|---|---|---|---|---|---|
| Arabic (3, 400) | 0.715 | 0.835 | 0.746 | 0.820 | 0.906 |
| Chinese (2, 275) | 0.815 | 0.882 | 0.836 | 0.886 | 0.916 |
| Czech (1, 135) | 0.832 | 0.879 | 0.825 | 0.885 | 0.909 |
| Dutch (2, 326) | 0.829 | 0.885 | 0.848 | 0.881 | 0.920 |
| French (3, 399) | 0.887 | 0.925 | 0.894 | 0.919 | 0.918 |
| German (3, 359) | 0.870 | 0.894 | 0.890 | 0.889 | 0.908 |
| Hindi (2, 304) | 0.768 | 0.815 | 0.744 | 0.815 | 0.883 |
| Italian (2, 315) | 0.840 | 0.882 | 0.831 | 0.885 | 0.887 |
| Japanese (2, 275) | 0.799 | 0.846 | 0.798 | 0.853 | 0.872 |
| Korean (2, 295) | 0.793 | 0.868 | 0.792 | 0.867 | 0.871 |
| Polish (1, 125) | 0.837 | 0.905 | 0.840 | 0.895 | 0.913 |
| Portuguese (1, 142) | 0.873 | 0.892 | 0.848 | 0.869 | 0.898 |
| Russian (2, 227) | 0.832 | 0.899 | 0.818 | 0.865 | 0.902 |
| Spanish (2, 296) | 0.872 | 0.910 | 0.865 | 0.905 | 0.926 |
| Swedish (1, 135) | 0.880 | 0.901 | 0.865 | 0.903 | 0.901 |
| Thai (2, 282) | 0.680 | 0.769 | 0.661 | 0.782 | 0.876 |
| other or mixed (2, 371) | 0.732 | 0.910 | 0.853 | 0.834 | 0.831 |
| Mean over the 31 cells | 0.816 | 0.873 | 0.818 | 0.868 | 0.901 |
The 14 router-safety sets (137,978 test rows; all rows)
| Set (AUC) | vela3.1-pico-150m | vela3.1-fast | vela3.1-pico-380m | vela3-fast (3.0) | Vela 2.0 4B | Vela 2.0 0.3B |
|---|---|---|---|---|---|---|
| RTP-LX, request harm | 0.614 | 0.666 | 0.649 | 0.657 | 0.774 | 0.728 |
| Multilingual HateCheck | 0.661 | 0.681 | 0.697 | 0.631 | 0.769 | 0.662 |
| XSTest, request harm | 0.927 | 0.954 | 0.924 | 0.947 | 0.967 | 0.890 |
| CultureGuard, request harm | 0.784 | 0.864 | 0.796 | 0.861 | 0.898 | 0.900 |
| Prompt attacks, unseen families | 0.762 | 0.922 | 0.822 | 0.875 | 0.985 | 0.882 |
| Prompt attacks, known families | 0.939 | 0.972 | 0.918 | 0.965 | 0.956 | 0.950 |
| deepset prompt injections | 0.982 | 0.988 | 0.970 | 0.985 | 0.989 | 0.957 |
| CultureGuard, response harm | 0.810 | 0.903 | 0.819 | 0.900 | 0.937 | 0.916 |
| AEGIS 2.0, response harm | 0.902 | 0.926 | 0.896 | 0.928 | 0.934 | 0.917 |
| PolyGuard, response harm | 0.866 | 0.897 | 0.860 | 0.906 | 0.925 | 0.936 |
| Do-Not-Answer, response harm | 0.832 | 0.828 | 0.874 | 0.822 | 0.931 | 0.656 |
| Do-Not-Answer, refusal | 0.911 | 0.924 | 0.900 | 0.919 | 0.893 | 0.896 |
| XSTest, refusal | 0.988 | 0.990 | 0.988 | 0.989 | 0.992 | 0.955 |
| PolyGuard, refusal | 0.850 | 0.898 | 0.827 | 0.895 | 0.945 | 0.947 |
| mean of 14 | 0.845 | 0.887 | 0.853 | 0.877 | 0.921 | 0.871 |
Vela 2.0 4B: its release run's own per-row test scores, scored with the same code on the same rows. Vela 2.0 0.3B: its published card values on the same test sets (not re-read by us). These sets were not part of the item-level scan, so the table uses all rows for every model.
Benign requests carrying personal data
| Benign requests that carry personal data, read as harmful (lower is better) | vela3.1-pico-150m | vela3.1-fast | vela3.1-pico-380m | vela3-fast (3.0) | Vela 2.0 0.3B |
|---|---|---|---|---|---|
| 470 support requests by other writers, 14 languages (% flagged harmful) | 4.9% | 21.1% | 3.4% | 13.6% | 4.0% |
| same requests (% flagged as jailbreak) | 1.5% | 4.7% | 1.1% | 5.5% | 1.7% |
Customer-support requests with names, addresses, card numbers and order ids, all safe: a public English support set (its placeholders filled with synthetic values) and messages hand-written by a different writer in 14 languages (e-mails, chat, forms, transcripts). Never used to pick a checkpoint. 30 of the 500 requests overlap training text and are left out. vela3-fast 3.0 and Vela 2.0 0.3B read by us through their public releases on the same requests.
ʰ Clean items: every test item that occurs in our training data, exactly (normalised text) or with at least half of its 13-grams covered by training rows, is left out for every model; the counts in brackets are the rows kept.
Languages| Non-English test rows (metric) | vela3.1-pico-150m | vela3-fast (3.0) | Vela 2.0 0.3B | LettuceDetect v2 mmBERT-base |
|---|---|---|---|---|
| Harmful request, 31 file × language cells (AUC) | 0.816 | 0.868 | 0.901 | – |
| Domain, 13 cells (accuracy) | 0.508 | 0.347 | 0.541 | – |
| Personal data, 25 cells (AUC) | 0.812 | 0.853 | 0.868 | – |
| Unsupported-claim spans, LettuceDetect v2 test, 13 languages (example-F1) | 92.78 | 93.70 | 93.40 | 90.58 |
Router rows: the clean items of the suite's test files, mean over (file, language) cells with at least 50 rows. Jailbreak and unsupported answers have too few non-English rows on the clean items for a per-language mean (jailbreak: 2 attacks among 159 rows; unsupported answers: one cell), so they are not listed. All columns read by us on the same rows.
MASSIVE, two-stage protocol (decision path), the protocol of the vela3.1-fast card: half A of the test split, 13 languages, all 18 scenarios; the scenario among 18, then the intent among that scenario's intents; English option descriptions. Clean items: 1,177 of 18,824 items that occur in our training data (exactly, or with half of their 13-grams covered) are left out. All Vela 3.1 models and vela3-fast 3.0 trained on the MASSIVE train split.
| Model | Intent accuracy, mean of 13 languages |
|---|---|
| vela3.1-pico-150m (this model) | 79.0 |
| vela3.1-pico-380m | 70.9 |
| vela3.1-fast | 66.6 |
| vela3-fast (3.0) | 31.1 |
This model per language: ar 71.1, de 82.6, es 83.7, fr 83.4, hi 75.7, hu 72.5, it 82.1, ja 78.8, ko 75.7, nl 81.7, pl 77.5, pt 81.8, ru 80.5.
Spans and extraction| Benchmark (metric) | vela3.1-pico-150m | vela3.1-pico-380m | vela3-fast (3.0, adapters) | Other models |
|---|---|---|---|---|
| Unsupported claims, LettuceDetect v2 test, 8,490 answers (example-F1) | 83.2 | 84.1 | 87.2 | LettuceDetect v2 mmBERT-base 84.7 (our read); Vela 2.0 0.3B 82.3 (our read); LettuceDetect v2 qwen-2b ⁷ |
| same (character span-F1) | 58.6 | 59.9 | 65.4 | LettuceDetect v2 mmBERT-base 62.7 (our read); LettuceDetect v2 qwen-2b ⁷ |
| RAGTruth ⁴ (example-F1) | 70.4 | 72.9 | 78.6 | LettuceDetect large v1 79.2, mmBERT-base 74.4 (our reads); HDM-2 ⁷ |
| OOD-7 zero-shot NER, label names (entity F1, mean of 7) | 51.2 | 53.9 | 62.2 | gliner-multitask-large-v0.5, GLiNER multi v2.1 ⁷ |
| NER, the 26 sets of the GLiFormer paper (strict F1) | 46.6 | 45.8 | 57.3 | GLiFormer-large ⁷ |
| CoNLL04, head entity given (tail F1) | 18.8 | 18.8 | 26.1 | GLiFormer-base 8.6 (our run) |
| Relations end to end, GLiFormer Table 4, mean of 4 (strict F1) ⁵ | 6.9 | 8.0 | 17.4 | GLiFormer-large ⁷ |
| ACL-Verbatim evidence ⁶ (word F1, 35 clean of 100 items) | 41.7 | 46.7 | 30.0 | ACL-Verbatim ModernBERT ⁷ |
| QASPER evidence ⁵ (word F1, in-domain) | 30.2 | 30.2 | 50.8 | – |
| Zero-shot classification, the 13 sets of the GLiFormer paper (macro-F1) | 71.3 | 73.0 | 74.1 | GLiFormer-large ⁷ |
One network with no task adapters: on span and relation tasks it is below the adapter package vela3-fast 3.0. All numbers on clean items unless a footnote says otherwise.
⁴ The RAGTruth rows of the LettuceDetect v2 test (2,694 of 2,700 kept). RAGTruth's train split is in the training data. ⁵ All rows: QASPER's test split was not part of the item scan; of the four Table 4 sets, DocRED has 29% of its gold items overlapping training text and FewRel 5%, which this row does not remove. ⁶ In-domain: ACL-Verbatim's train split is in the evidence data, and 65 of its 100 test items overlap training text; the row uses the other 35, so it is noisy. ⁷ Reported only, on all rows, so not comparable with the clean items: LettuceDetect v2 qwen-2b 92.1 example-F1 and 68.9 span-F1 (2.3B), HDM-2 85.0 on RAGTruth (3B), gliner-multitask-large-v0.5 62.76 on OOD-7, GLiFormer-large 50.91 on the 26 NER sets, 21.3 on Table 4 and 75.03 on the 13 classification sets, ACL-Verbatim ModernBERT 53.4. GLiNER multi v2.1 reads 54.3 on OOD-7 in our run (all rows; no per-item output kept).
Speed
| Call, on CPU | fp32 (PyTorch) | int8 (onnx/) |
|---|---|---|
| Core router call ⁸ (one pass, 970 tokens) | 2,170 ms | 1,364 ms |
| 1 decision | 560 ms | 250 ms |
| 5 decisions | 2,194 ms | 1,197 ms |
| 10 decisions | 4,743 ms | 2,346 ms |
AMD EPYC 7413 24-Core Processor on an otherwise idle machine, 4 threads, median of 5 after 2 warm-ups; one fixed 200-token request with a context and an answer; PyTorch 2.8.0, onnxruntime 1.31.1; the int8 column runs the encoder in onnx/. Measured 7 Nov 2026 on these weights. A call made only of decisions runs one pass per question.
⁸ Domain, fact-check need, feedback, modality, harmful request and jailbreak, plus PII and unsupported-claim spans, on one request.
Reduced precision. Spans are evaluated in fp32. int8 moves some unsupported-claim span boundaries by a few characters; use fp32 or bf16 as the span reference.
Training data
vela3.1-pico-150m was initialised from a 151M decision tower and is trained as one network on the Vela 3.1 decision mix (see vela3.1-fast) together with the router-signal and span data, about 10% of it multilingual, with several questions per sequence, and with hand-written benign customer requests that carry personal data as safe examples. Questions and options are rendered by one module in training and in the package. Three runs with different seeds were averaged weight by weight. No test split and no Decision Index item is trained on; test items that overlap training text anyway are left out of the evaluation (clean items).
Limits
- Safety signals are below vela3-fast 3.0 and Vela 2.0 0.3B. Harmful request on the suite's test file 0.888 against 0.913 (3.0) and 0.930 (Vela 2.0 0.3B); Nemotron-Safety-Guard v3 0.808 against 0.860 (3.0) and 0.895 (Vela 2.0 0.3B); PolyGuardPrompts 0.827 against 0.870 (3.0) and 0.897 (Vela 2.0 0.3B). Outside English 0.816 against 0.868 and 0.901 (31 cells), lowest in Thai, Arabic and Hindi. On the 14 router-safety sets the mean AUC is 0.845, against 0.877 for 3.0, 0.871 for Vela 2.0 0.3B and 0.921 for Vela 2.0 4B.
- Against vela3-fast 3.0 over the suite's 24 test files: 6 better, 6 level, 12 worse. Its losses: harmful request (the test file, AEGIS 2.0, Nemotron-Safety-Guard v3, PolyGuardPrompts, WildGuardMix); hazard category (AEGIS 2.0); modality (the test file); personal data (MAPA); feedback (the test file, SGD); unsupported answer (the test file, RAGTruth).
- Spans are below the adapter package: against vela3-fast 3.0, unsupported claims 83.2 against 87.2, RAGTruth 70.4 against 78.6, NER (26 sets) 46.6 against 57.3, end-to-end relations 6.9 against 17.4, QASPER 30.2 against 50.8. On unsupported claims it is also below LettuceDetect v2 mmBERT-base (84.7).
- On held-out safety sources vela3-fast 3.0 is ahead on 7 of 7 files: CoCoNot 0.943 against 0.969; CoCoNot contrast 7.9% against 6.3% flagged; JailbreakBench 0.943 against 0.963; OpenAI moderation 0.920 against 0.925; OR-Bench hard 63.2% against 57.7% flagged; ToxicChat 0.965 against 0.971; XSTest 0.933 against 0.948 (AUC, or share of benign requests flagged at 0.5 on the benign-only files).
- On held-out jailbreak sources vela3-fast 3.0 is ahead on 1 of 6 files: NotInject 4.1% against 3.2% flagged. On the other files' attack AUC this model is level or ahead (see the Safety section).
- Hazard categories (which kind of harm): below vela3-fast 3.0 on 4 of the 4 held-out hazard files (AILuminate 0.620 against 0.786; Aya red-teaming 0.532 against 0.549; BeaverTails eval 0.663 against 0.831; MLCommons synthetic 0.682 against 0.880), and far below Decision-1.0-Route-0.6B, which trains on the suite (0.676 against 0.913 macro AUC on the test file).
- Jailbreak outside English is below vela3-fast 3.0 and vela3.1-fast: 0.875 against 0.911 and 0.927 AUC over the 579 clean non-English rows of the suite's seven jailbreak files (136 attacks); Vela 2.0 0.3B reads 0.858.
- Benign requests carrying personal data are read as harmful more often than by Vela 2.0 0.3B: 4.9% of 470 held-out support requests, against 13.6% for vela3-fast 3.0 and 4.0% for Vela 2.0 0.3B (vela3.1-pico-380m 3.4%, vela3.1-pico-150m 4.9%). Set the harmful-request cut on your own rows.
- int8 moves some unsupported-claim span boundaries by a few characters; fp32 or bf16 are the span reference.
- MASSIVE has not been read yet for this checkpoint on the test protocols of the vela3.1-fast card.
- Decisions-only calls run one pass per question.
- Decision Index Full score (private tests): board submission pending.
The family
| Model | Size | Best for |
|---|---|---|
| vela3.1-fast | 0.4B, adapters | the default router: best decisions, spans and safety signals of Vela 3.1 |
| vela3.1-pico-380m | 0.4B, one file | the whole router call in one pass |
| vela3.1-pico-150m (this model) | 0.15B, one file | the smallest one-pass router; CPU |
Citation
@misc{vela31_pico_150m,
title = {vela3.1-pico-150m (Vela 3.1, 0.15B): decisions, router signals and spans in one forward pass},
author = {{KR Labs} and {vLLM Semantic Router}},
year = {2026},
howpublished = {\url{https://huggingface.co/vllm-sr/vela3.1-pico-150m}}
}
Revision history
7 Nov 2026 · weights
c4f08902(branchr31-s-l): first release of vela3.1-pico-150m. Every number on this card is for these weights; the card text was revised the same day (CPU latency from one idle-machine session, quick start with described labels, Decision Index rows from the board data of 6 Nov 2026 with GLiDE 0.4B, size wording).9 Nov 2026 · card text only: MASSIVE two-stage reads of the shipped pico-380m and pico-150m packages added (same protocol and clean items as vela3.1-fast); weights unchanged.