s1d1-router — a one-pass "System 1" router for LLM requests

One small encoder, one forward pass over the first user message alone, three outputs:

Use it in front of a pool of models: send the easy, tool-free requests to a small or local model and keep the expensive one for what needs it.

Try it without installing anything: the in-browser demo (runs in your browser, sends nothing) or the free hosted API at https://s1d1.qoo.gy (what you send is kept, anonymised and used to retrain s1d1; read its terms first). To serve it yourself, for example as the classifier of LiteLLM's Auto Router, use s1d1-tantive (see Use it with LiteLLM).

Variants

The models, trained on six sources, are available in four sizes and three formats. They are grouped in the s1d1-router collection.

Each repository card gives its validation measures, its agreement with the full model and its latency. To try the router without installing anything, the in-browser demo runs the compact ONNX variant locally in your browser: nothing you type leaves your machine.

Correction of 2026-10-25 (ONNX int8). The model_int8.onnx of the four ONNX int8 repositories was re-quantized with reduce_range (7-bit weights). The earlier files gave wrong scores on x86 CPUs without VNNI (AVX2 only): for small, "hi" scored 1.0 instead of 0.04. If you downloaded an ONNX int8 file before that date, download it again; the fixed revisions are small 33c7f34, full 068d0d2, compact 7e87870 and tiny 868fb8f. The agreement and latency figures on the cards of these four repositories were measured on the earlier weights unless a card says otherwise.

Quick start

Python ≥ 3.10, transformers ≥ 5 (checked with 5.19) and torch.

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

repo = "scombat/s1d1-router"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()

TASKS = model.config.task_names  # 19 names, in output order 1..19
batch = tok(["Quelle est la capitale de l'Australie ?"], truncation=True, max_length=1024,
            padding=True, return_tensors="pt")
with torch.no_grad():
    out = model(**batch).logits[0]

complexity = float((0.5 + 0.25 * out[0]).clamp(0.01, 1.0))   # 0.01 .. 1.00
task = TASKS[int(out[1:20].argmax())]
needs_tools = bool(torch.sigmoid(out[20]) >= 0.5)

examples/usage.py is the same thing as a script, and s1d1_router.py wraps it: Router(repo, device="mps") takes a Hub id or a local folder, picks the backend (PyTorch, MLX or ONNX Runtime, from variant.json), applies vocab_map.npy for the pruned-vocabulary variants and returns complexity_score, complexity_tier (trivial / low / medium / high), task, needs_tools, needs_tools_probability and prompt_tokens. On a CPU it loads the weights in float32.

Do not read this model through pipeline("text-classification") with its defaults, nor through Text Embeddings Inference: both apply one softmax over the 21 outputs, but output 0 is a regression and output 20 is a binary logit. config.json declares problem_type: regression, so the pipeline returns the raw outputs when asked for all of them: pipeline("text-classification", model=repo, top_k=None) gives 21 {label, score} pairs named by id2label (complexity, task:<name>, needs_tools; checked with transformers 5.19), to decode as above. The text-embeddings-inference tag is added by the Hub for this architecture; TEI's scores are a softmax over the 21 outputs and are wrong for this model. Hence inference: false.

Expected output on the three prompts of examples/usage.py:

How to use the outputs

A deliberately simple policy (thresholds are yours to set on your own traffic):

if needs_tools or complexity >= 0.75:      route_to = "frontier"
elif complexity >= 0.50:                   route_to = "mid"
else:                                      route_to = "small/local"

Two other uses: choosing the reasoning effort or thinking budget of a single model per request; triaging a queue (cheap batch endpoint for trivial/low, interactive endpoint for the rest).

Why this design

• Encoder, not decoder. At equal size, controlled comparisons favour bidirectional encoders for classification and regression (Ettin), published routers (RouteLLM, Hybrid LLM, NVIDIA's prompt classifier) use encoders of 100–400 M parameters, and an encoder costs about 4× less to train and is faster at batch size 1.
• Why mmBERT-base. It is the only candidate that covers all the languages of our data with a native ForSequenceClassification class, an MIT license and 8k context. In a pilot (30,000 training prompts, 2 epochs) it beat the encoder of laya-multilingual (itself mmBERT-base with a decision head) on every validation metric, by 1–2 points. One seed: read it as "not worse and probably slightly better".
• A standard model, a custom loss. The published model has no custom code: the loss lives in the training script (train.py here) — squared error on the complexity (rows whose complexity is null are masked), cross-entropy on the task, binary cross-entropy on needs_tools, summed with no coefficient (complexity is expressed in tier units so its scale matches the cross-entropies).
• Trained on the prompt alone. The router never sees the answer, a system prompt or the conversation history. It reads the first user message.

Training data and labels

98,033 / 4,914 / 10,000 prompts (train / validation / test) from six public sources (WildChat, lmsys-chat-1m, UltraChat, Aya, Magpie, OASST2), drawn by source × language × length cell, with a second wave oversampling high complexity and rare tasks. The dataset is not published.

Labels come from an LLM judge, not from humans: google/gemma-4-31B-it with a 512-token reasoning budget and a fixed rubric (v7), temperature 0. Rubric agreement with a Claude Opus reference on an 85-item bench: 76 % (complexity, QWK 0.87), 88 % (task), 96 % (needs_tools). A human read of 200 labelled prompts corrected 11.5 % of them (mostly low that should be trivial); about 3.4 % of labels change between two passes. Every number below measures agreement with this judge. The rubric (v7, byte for byte) and the exact request sent to the judge (messages, JSON schema, reasoning budget, model revision) are in judge/, so you can label your own traffic the same way.

Test hygiene: training and validation prompts at estimated Jaccard ≥ 0.8 of a test prompt (character 5-shingles, MinHash) were removed, near-duplicate groups are never split across train and validation, and groups of more than 20 rows go to train only. Template families that still cross the sets (300 test rows) are listed in test_template_family_keys.txt; test results are reported on the full test set and without those rows.

Results

Validation, 4,914 prompts (complexity on the 4,850 non-null rows), after 3 epochs. "Weighted" restores the natural distribution of the pool with the rebalance_weight column (factors computed on the training set; the model is trained without them). Chance level for task is 5.3 %, majority class (information_seeking) 31.1 %.

Test, 10,000 prompts, evaluated once (more multilingual than train: English 39.7 % vs 61.1 %):

Complexity confusion matrix, full test, on the 9,894 rows with a judge complexity (106 test rows are null and are excluded from every complexity metric; rows: judge tier, columns: predicted tier):

Recall per tier: trivial 55 %, low 90 %, medium 67 %, high 65 %, expert 0 %. Almost every error is one tier off (30 rows are two tiers or more away).

Task F1 per class, full test (support in parentheses):

Per language, full test (same weights and metrics as above, computed once). Pooled, the 3,973 English prompts give MAE 0.073 / QWK 0.71 / task accuracy 0.87 / needs_tools F1 0.70, the 6,027 non-English prompts MAE 0.068 / QWK 0.70 / task accuracy 0.85 / needs_tools F1 0.60 (AUROC 0.94). The table lists the languages with at least 100 test rows (51 of 78, sorted by test rows) and pools the 27 others. Read it with care: one seed, and outside English every line rests on 100 to 355 prompts, so about ±0.07 to ±0.09 on an accuracy at n = 100 and more on a QWK or on an F1 with a handful of positives. Read the QWK with the share of low next to it: where the judge put about 80 % or more of a language's prompts in low (Japanese, Dutch, Irish, Bengali, Malagasy, Turkish, Finnish, Thai) the kappa hangs on a handful of rows and is low although the MAE is the best of the table. Read the task accuracy with the majority-class share next to it: for most Aya languages a constant predictor already gets 0.5–0.85. needs_tools has 0 to 18 positives per language outside English; for several languages the F1 at 0.5 is 0 while the AUROC is above 0.9 (the few positives rank above the negatives but under the threshold). 247 of the 300 template-family rows are English, 28 Chinese, 18 Punjabi; without them English QWK is 0.69, Chinese 0.75, Punjabi 0.59. The lat and tgl rows (WildChat) are mostly language-detector errors (English, Chinese or Uzbek prompts).

Pilot (30,000 training prompts, 2 epochs, validation, for orientation only): QWK 0.72, task accuracy 0.82, macro-F1 0.72, needs_tools F1 0.62 / AUROC 0.95. Rebalancing weights in the loss changed nothing measurable in the pilot, so the released model is trained without them.

Latency (Apple M4 Max, batch 1, 200 prompts after warm-up): PyTorch fp16 on MPS (Router(repo, device="mps")) p50 10.1 ms (p95 39 ms); MLX 8-bit p50 4.5 ms; ONNX int8 on CPU p50 10.9 ms. Smaller variants: see Variants.

Independent, replayable bench against local LLM judges

s1d1-bench (public, v0.1.0) measures how well a classifier agrees with the median of three local judges (Gemma 4 31B, Qwen3.8 27B, gpt-oss-20b, thinking capped at 512 tokens) on base-v1: 90 base prompts drawn from Dolly, OASST2, m-ArenaHard and hand-written ones, each in English, French, Russian and Chinese (360 prompts). Quadratic weighted kappa on the five-tier scale, 95 % intervals by bootstrap over the 90 base prompts: full int8 0.807 [0.734 ; 0.852], small int8 0.772, compact int8 0.759, tiny int8 0.614; the inter-judge ceiling (each judge against the other two) is 0.774 [0.718 ; 0.815]; the prompt-length witness 0.543; Laya 0.405 (english) and 0.119 (multilingual); LiteLLM's built-in heuristic 0.089. Per language, full int8 holds 0.84 / 0.80 / 0.79 / 0.79 (en / fr / ru / zh). The judges are the instrument, not competitors; the bench writes no "better than" sentence unless the paired difference passes a 2 σ rule, and it does not separate full from small or compact. Caveat the bench states first: Gemma 4 is both the labeler of this model's training data and one of the three judges; against the two other judges alone, full int8 is at 0.785. Everything is replayable from the repository: judge files, classifier files, bench run, bench report.

Agreement with the judges against median latency per prompt (batch 1, Apple M4 Max; s1d1 int8 on CPU, Laya on MPS, judges on MLX with thinking capped at 512 tokens). The judges' y is each judge against the other two. Figure and data: s1d1-bench, report/base-v1.

The same agreement for every candidate and witness, with the judges' ceiling (mean and per judge) and the floors (length, constant, random).

Agreement with the judges' median (truth_all) on the 360 prompts, 95 % intervals by bootstrap over the 90 base prompts; "exact tier" is the share of prompts whose tier is the median's (s1d1-bench, report/base-v1/tables):

The bench does not separate small from full (paired difference +0.035 [−0.004; 0.076]); small weighs 56 MB against 310 and answers in 5.8 ms against 9.1 per prompt (M4 Max, batch 1, CPU). Laya and the heuristic are the two options LiteLLM documents for a self-hosted classifier, read zero-shot on this rubric with LiteLLM's four tiers mapped name to name. RouteLLM's router predicts whether GPT-4 beats Mixtral on an English question, not a tier; the bench reads its probability with five equal bounds fixed in advance. The 4B model is of the Gemma 4 family, one of the three judges and the labeller of this model's training data: against the two other judges it is at 0.713 and full int8 at 0.785; it answers in 1.1 s per prompt on a GPU and weighs 8.9 GB, against 9 ms and 310 MB. NVIDIA's classifier reads English only, so it is scored on the 90 English prompts (full int8 on the same 90: 0.840 [0.767; 0.890]; paired gap 0.529 [0.404; 0.634]); its complexity score is a weighted sum of six dimensions, read with the same five equal bounds.

Limitations (read before relying on it)

• Labels are model judgments. The router learns the judge's idea of complexity, with its noise (11.5 % of the 200 prompts we re-read were corrected). It is not a measure of how hard a prompt is for your model.
• trivial is hard to separate from low (recall 50 % in validation, 55 % in test) and expert is not learnable: 81 examples in train + validation, 3 in test, never predicted on test. s1d1_router.py therefore reports four tiers and labels every score above 0.625 high; the score itself still runs up to 1.00.
• Rare tasks are unreliable: data_analysis (531 train rows), action_execution (357), other (1,367), planning, role_playing. Their F1 moves a lot between runs.
• needs_tools has 4.9 % positives in train (5.1 % in validation, 4.0 % in test); the 0.5 threshold is not tuned. Calibrate it on your own data.
• Test is more multilingual than training. The per-language table above rests on about 100 test prompts per language outside English, one seed, no confidence interval; the language labels of WildChat, lmsys and Magpie come from automatic detectors and are partly wrong on short prompts.
• Complexity is not cost. We did not measure the quality or price of any model behind the router. The routing policy above is an example, not a validated one.
• One seed per configuration, no confidence intervals on the tables above, except the small tier, trained with three seeds (its cards give the spread). Differences of about 0.01 are within what we would expect from noise. The independent bench above gives bootstrap intervals, on 360 prompts.
• Single turn only. No system prompt, no history. Prompts longer than 1,024 tokens are truncated (3.8 % of train, 4.0 % of test).
• Template-heavy data. Some of the training prompts are web-scraped templates (e.g. a Chinese slide-deck template) with very homogeneous labels.
• Raw prompts. The data was not moderated; sexual, hateful and jailbreak prompts are labelled by intent.

Use it with LiteLLM's Auto Router

LiteLLM's auto router sends each request to the model of a tier and asks a classifier for that tier. s1d1-tantive serves this router as that classifier, in a few milliseconds and without an LLM call, as a container image per variant (multi-arch, signed):

docker run --rm -p 8788:8788 ghcr.io/scombat/s1d1-tantive:0.8.1-small-onnx-int8

Then, in LiteLLM 1.106 or later, in config.yaml (tested on 2026-10-31 with LiteLLM 1.106.6 against s1d1-tantive):

model_list:
  # ... your four models: cheap, medium, strong, reasoning
  - model_name: s1d1-classifier
    litellm_params:
      model: hosted_vllm/s1d1-small       # the name after the slash is sent and ignored by the service
      api_base: http://localhost:8788     # the service root; use https://s1d1.qoo.gy/api for the free hosted API
    model_info: {mode: decisions}
  - model_name: s1d1-auto
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_default_model: medium
      complexity_router_config:
        classifier_type: oss_classifier
        opensource_classifier_config: {deployment_name: s1d1-classifier, timeout_ms: 1000}
        classifier_fallback: default_model
        tiers: {SIMPLE: cheap, MEDIUM: medium, COMPLEX: strong, REASONING: reasoning}

If the service is slow or unreachable, LiteLLM uses the default model (a timeout opens a 30-second circuit breaker; a stopped service falls back on each request). The response header x-litellm-complexity-router-cause reads jev_classifier for any classifier, so check tantive_decisions_total on the service's /metrics instead. The provider: laya / model: multilingual fallback (same tiers on the same /v1/systemone; needed only where the admin GUI offers no deployment), the curl route and LiteLLM before 1.106 (chat adapter) are in s1d1-tantive docs/litellm.md and tutorials/02-litellm.md.

Use it in your stack

s1d1_router.py loads any repository of the Variants table (Hub id or local folder; PyTorch, MLX or ONNX backend) and decodes the outputs; integrations/ turns them into a model and a reasoning effort with a YAML policy (policy.example.yaml, with profiles for Claude Code, Codex and generic harnesses), a small HTTP server, a launcher for Claude Code / Codex. integrations/serve_variant.sh MODEL starts a server for any variant without installing anything (it needs uv). For LiteLLM, use the s1d1-tantive image (see above). Tutorials: tutorials/ (HTTP, LiteLLM, Claude Code, Codex, n8n, OpenClaw). Each page says what was checked against the tool's documentation and what is untested.

Fine-tune it for your own routes

Three levels, from cheapest:

1. Re-use as-is and set your own thresholds on complexity_score / needs_tools, using a few hundred of your own labelled requests.
2. Continue training with the same 21 outputs on your labels: train.py in this repository reads a parquet file (prompt, complexity_score nullable, task, needs_tools, optional weight), starts from these weights (or from mmBERT with --base jhu-clsp/mmBERT-base) and applies the loss above with the recipe of the published model. To label your own traffic with the same judge, the rubric and the exact request are in judge/.

   python train.py --train my_prompts.parquet --validation my_validation.parquet --out my-router --device mps
   

3. Your own classes: keep the encoder and replace the head.

   model = AutoModelForSequenceClassification.from_pretrained("scombat/s1d1-router", num_labels=N, ignore_mismatched_sizes=True,
                                                              problem_type="single_label_classification")  # the config says regression
   

   then train with any standard classification loop (Trainer works).

A full fine-tune of this size ran at about 27 prompts/s on a laptop-class Apple GPU (3 epochs on 98k prompts ≈ 3.0 h).

Estimated upside (a best-case estimate, not a measurement)

Question: what is the most you could save by not sending every request to a frontier model at maximum reasoning effort? Upper bound on the share of requests that are even candidates for a smaller model:

• In the naturally-sampled part of our data (wave 1, 27,871 train prompts), 69% of prompts are trivial or low and 96% need no tools. Those are the candidates.
• If a request costs C_big on the frontier model with extended reasoning and C_small on a local/small model, the best-case saving is share_routable × (1 − C_small / C_big). With a local model at a marginal cost close to zero this is bounded by the routable share itself (about 69% of the spend if costs per request were equal, more if extended reasoning is expensive on easy prompts, because easy prompts then cost as much as hard ones).
• What this ignores: that the small model must actually answer those prompts well (not measured), requests that the router misplaces (trivial recall is 50 % in validation), your own traffic mix (ours is a public-data mix), the cost of running the local model, and latency. Treat it as the ceiling, not as the expected value.

Intended use

Routing, triage and effort selection for LLM requests; analysis of prompt mixes. Not for: moderation or safety decisions, grading users, or any decision that must be right on a single input.

License

Apache-2.0. The base model, mmBERT-base, is MIT-licensed; its notice is kept in NOTICE. The license requires you to keep LICENSE and NOTICE when you redistribute the weights or a derivative; it does not require attribution when you merely use the model in a product or behind an API. If you use it in published work or a product, we would appreciate a mention and the citation below (a request, not a condition).

The three s1d1 projects, side by side:

Citation and related work

@misc{s1d1_router_2026,
  title  = {s1d1-router: a one-pass multilingual router for LLM requests},
  author = {scombat},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/scombat/s1d1-router}}
}

Related: laya-multilingual and THX-01 (also built on mmBERT-base), NVIDIA prompt-task-and-complexity-classifier.

References:

• mmBERT: A Modern Multilingual Encoder with Annealed Language Learning (arXiv:2509.06888)
• Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference (ModernBERT, arXiv:2412.13663)
• Seq vs Seq: An Open Suite of Paired Encoders and Decoders (Ettin, arXiv:2507.11412)
• EuroBERT: Scaling Multilingual Encoders for European Languages (arXiv:2503.05500)
• Unsupervised Cross-lingual Representation Learning at Scale (XLM-R, arXiv:1911.02116)
• RouteLLM: Learning to Route LLMs with Preference Data (arXiv:2406.18665)
• Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing (arXiv:2404.14618)
• Arch-Router: Aligning LLM Routing with Human Preferences (arXiv:2506.16655)
• FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance (arXiv:2305.05176)
• ThinkSwitcher: When to Think Hard, When to Think Fast (arXiv:2505.14183)
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (arXiv:2306.05685)
• Model Cards for Model Reporting (arXiv:1810.03993)