s1d1-router, six sources, small, MLX 8-bit (Apple Silicon)

A "System 1" router: it reads a user prompt and returns its complexity score (0.01 to 1.00), its task type (19 classes) and whether it needs tools. It never picks a model; the deployer maps these outputs to models with rules. The method, the training data, the full results and the limitations are in the card of the main repository, scombat/s1d1-router. An open, replayable bench of the variants against three local LLM judges on 360 multilingual prompts, with confidence intervals and witnesses, is s1d1-bench.

Variant: v7_small_vocab32k-mlx-8bit, mmBERT-small (22 layers x 384), 32k vocabulary, distilled from the full model.
Format: MLX 8-bit (Apple Silicon), 95 MB.
Outputs: 21 logits. Output 0 is the complexity in tier units (score = 0.5 + 0.25 x output, clipped to 0.01..1.00), outputs 1 to 19 are the task logits (order in task_names of config.json), output 20 is the needs_tools logit.
Pruned vocabulary variants ship vocab_map.npy (full tokenizer id -> pruned id); the loader below applies it.
Do not read it through pipeline("text-classification") with its defaults, nor through Text Embeddings Inference: both apply one softmax over the 21 outputs. config.json declares problem_type: regression, so pipeline(..., top_k=None) returns the raw outputs; decode them as the main card shows.

Measures (validation, 4,914 rows, six sources, unweighted)

complexity QWK	complexity MAE	task accuracy	task macro-F1	needs_tools F1	needs_tools AUROC
0.732	0.079	0.834	0.749	0.608	0.949

Agreement with the full fp32 model on validation: same complexity tier 86.3%, same task 87.6%, same needs_tools 97.4%. Latency, batch 1, mlx backend on an Apple M4 Max: p50 3.0 ms, p95 14.2 ms. Full results and method: docs/15-catalog-v4-v7.md in the project; eval.json in this repo.

Seed-to-seed spread

The student behind this variant was trained 3 times with the same recipe and seeds 42, 43, 44 (fp32, before conversion); the published weights are seed 42. Mean ± sample standard deviation, validation as above and the test set (10,000 prompts, read once per seed). A difference between two variants of this size smaller than about twice this standard deviation is within seed noise.

	complexity QWK	complexity MAE	task accuracy	task macro-F1	needs_tools F1	needs_tools AUROC
validation	0.726 ± 0.006	0.079 ± 0.000	0.833 ± 0.004	0.741 ± 0.006	0.610 ± 0.009	0.950 ± 0.004
test	0.649 ± 0.002	0.076 ± 0.000	0.823 ± 0.001	0.711 ± 0.005	0.554 ± 0.009	0.933 ± 0.002

Use

s1d1_router.py from scombat/s1d1-router loads this repository directly (Python >= 3.10, tokenizers, huggingface_hub and mlx-embeddings):

from s1d1_router import Router
router = Router("scombat/s1d1-router-6src-small-mlx-8bit")
router.route(["Quelle est la capitale de l'Australie ?"])

License

Apache-2.0. The base model, mmBERT-small, is MIT-licensed; its notice is kept in NOTICE. The license requires you to keep LICENSE and NOTICE when you redistribute the weights or a derivative; it does not require attribution when you merely use the model in a product or behind an API. If you use it in published work or a product, a mention of the author and the citation below are appreciated (a request, not a condition).

Citation

@misc{s1d1_router_2026,
  title  = {s1d1-router: a one-pass multilingual router for LLM requests},
  author = {scombat},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/scombat/s1d1-router}}
}