TinyRouter1-22M

TinyRouter1-22M is a lightweight 22.7M parameter prompt router. It classifies an English prompt into one of 4 broad domains, so you can route it to the right specialist model:

It is designed for edge, mobile, and low-cost inference. 85MB, runs on CPU in <50ms, no GPU needed for inference.

Benchmarks

Test set: 1255 held-out domain-routing examples, filtered from MaziyarPanahi/Jev-Intent-Domain-Routing-130K (domain task only).

Accuracy: 89.48%

Training progression

Measured on the same 1255 test examples after each epoch:

Training plateaus after epoch 3. More epochs do not help - the dataset has noisy labels and the ceiling for MiniLM on this task is around 90-91%.

Real prompt tests

Tested with the final saved model:

The last example is intentionally hard: half English, half Turkish. The model still picks multilingual correctly, but with lower confidence. Mixed-language prompts are the hardest case.

Training Details

• Base model: sentence-transformers/all-MiniLM-L6-v2 (22.7M)
• Dataset: MaziyarPanahi/Jev-Intent-Domain-Routing-130K, filtered to question_key == "domain" only. Train + validation + test domain splits combined, then re-split 95% / 5%.
  • Train: 23842 examples
  • Test: 1255 examples
• Input format: state_preview + " [SEP] " + question_text, truncated to 256 tokens
• Hyperparameters:
  • Epochs: 6 (best epoch 3)
  • Batch: 256
  • Max len: 256
  • LR: 2e-5, cosine schedule, warmup 10%
  • Weight decay: 0.01
  • Label smoothing: 0.1
  • Loss: CrossEntropyLoss
  • Optimizer: AdamW, mixed precision (AMP)
• Hardware: 1x T4 (14.5GB), ~25 min
• Framework: PyTorch + Transformers

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch, json

tok = AutoTokenizer.from_pretrained("Kuantum-ai1/TinyRouter1-22M")
model = AutoModelForSequenceClassification.from_pretrained("Kuantum-ai1/TinyRouter1-22M")
model.eval()

labels = ["code", "general", "multilingual", "stem"]

def predict(prompt: str):
    state = json.dumps({"conversation": [{"content": prompt, "role": "user"}]})
    text = state + " [SEP] Which broad domain best describes this conversation?"
    enc = tok(text, truncation=True, max_length=256, return_tensors="pt")
    with torch.no_grad():
        probs = torch.softmax(model(**enc).logits, dim=-1)[0]
    return {labels[i]: round(float(p)*100, 1) for i, p in enumerate(probs)}

print(predict("Write a Python script to calculate average with pandas"))
# {'code': 91.8, 'general': 2.0, 'multilingual': 3.0, 'stem': 3.1}

print(predict("How do I solve integrals and derivatives?"))
# {'code': 6.7, 'general': 9.0, 'multilingual': 5.8, 'stem': 78.6}

Routing example

# Route to specialist models based on prediction
result = predict(user_prompt)
domain = max(result, key=result.get)

if domain == "code":
    answer = code_model.generate(user_prompt)
elif domain == "stem":
    answer = math_model.generate(user_prompt)
elif domain == "multilingual":
    answer = translation_model.generate(user_prompt)
else:
    answer = general_model.generate(user_prompt)

Limitations

• English only: The model is trained on English prompts. Non-English prompts are classified as multilingual by design, not routed by topic. Turkish, Chinese, etc. will all go to multilingual.
• Short prompts: Very short prompts like "hi" or "help" are ambiguous. The model will default to general.
• Mixed-language: Prompts that mix English and another language (e.g. "Translate this... Bugun cok yorgunum") have lower confidence, though argmax is usually still correct.
• Noisy labels: The source dataset has some label noise, especially between general and stem (e.g. weather can be both). This caps accuracy around 90-91% for this architecture.
• No conversation history: The model only sees the current prompt wrapped as a single-turn conversation. It does not use multi-turn history.

License

Apache 2.0. Same as base model all-MiniLM-L6-v2. Free for commercial use.

Citation

@misc{tinyrouter1-22m,
  title = {TinyRouter1-22M: Lightweight 22M Prompt Router},
  author = {Kuantum-ai1},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Kuantum-ai1/TinyRouter1-22M}}
}

Acknowledgements

• Base model: sentence-transformers/all-MiniLM-L6-v2
• Dataset: MaziyarPanahi/Jev-Intent-Domain-Routing-130K