AlphaRoute-D (0.8B Decision-Native SLM)

AlphaRoute-D is a decision-native, non-autoregressive semantic routing and agent dispatch small language model built on top of Qwen3.5-0.8B.

Unlike traditional autoregressive routing models that generate structured JSON tokens step-by-step (query -> {"route": "..."} -> parse), AlphaRoute-D evaluates dynamic candidate options in parallel using dedicated <DECISION> markers in a single forward pass, producing an exact softmax probability distribution over candidate routes at 5–13 ms latencies.

AlphaRoute-D learns the meta-task of routing: it does not rely on hardcoded static labels. At runtime, developers supply an arbitrary task instruction, a dynamic ontology (with descriptions), and an incoming query.

---

Benchmark Evaluation (Deterministic Fixed Splits)

All evaluations are conducted on standardized fixed test splits against AlphaRoute v1.5 (Generative SLM), Kev-0.8B (jaredpalmer/kev-0.8b, Pointer Head), Laya (convaiinnovations/laya, ModernBERT-421M Bi-Encoder), and mini-Jev (samatv256/mini-Jev, Qwen3-0.6B + LoRA + SetHead):

---

Latency Profile Across Hardware Platforms

---

Architecture: How It Works

Instead of decoding output tokens sequentially, AlphaRoute-D formats the prompt with universal <DECISION> candidate markers:

<STATE>
Find all users whose subscription expired before August 2026 and count their total invoice debt.

<QUESTION>
Route developer assistant command to appropriate autonomous tool.

<OPTIONS>
<OPTION_0> sql_database_query: Execute read-only SQL SELECT queries against database
<OPTION_1> bash_terminal_exec: Run local shell commands, git operations, or build pipelines
<OPTION_2> browser_subagent: Navigate web pages, fill DOM forms, and take screenshots
<OPTION_3> vector_semantic_search: Perform cosine similarity retrieval across embedded knowledge
<OPTION_4> out_of_scope: Direct conversational query not requiring tool invocation

<DECISIONS>
<DECISION> <DECISION> <DECISION> <DECISION> <DECISION>

1. Full Causal Attention Receptive Field: The model processes the input state, question, and candidate options.
2. Hidden State Extraction: Contextual hidden representations $h_i$ at the <DECISION> marker positions are gathered.
3. Shared DecisionHead MLP: A parameter-shared 2-layer MLP (LayerNorm -> Linear(d->d) -> GELU -> Linear(d->1)) scores candidate vectors into scalar logits.
4. Dynamic Masking & Softmax: Padded candidate slots are masked out, yielding a normalized probability distribution $P(\text{choice}_i \mid \text{state}, \text{question}, \text{options})$.

---

Quickstart Inference

AlphaRoute-D is a fully native Hugging Face model. You can load and run it directly with standard transformers using trust_remote_code=True:

import json
import torch
from transformers import AutoModel, AutoTokenizer

# 1. Load model & tokenizer directly from Hugging Face Hub
model_id = "NamanAgnih0tri/AlphaRoute-D-0.8B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16
)

device = "cuda" if torch.cuda.is_available() else ("mps" if torch.backends.mps.is_available() else "cpu")
model.to(device)
model.eval()
model.set_tokenizer(tokenizer)

# 2. Define runtime routing prompt & dynamic candidate categories
query = "My debit card was stolen while traveling in London, please freeze it immediately."
task = "Identify customer banking intent."
categories = {
    "lost_or_stolen_card": "Misplaced, stolen, or compromised physical card.",
    "card_arrival": "Inquiring about dispatch or delivery of new card.",
    "dispute_charge": "Fraudulent or unauthorized transaction on account.",
    "pin_blocked": "ATM withdrawal failed after multiple incorrect PIN entries.",
    "out_of_scope": "General conversation or inquiry unrelated to banking card services."
}

# 3. Route in a single forward pass
decision = model.route(query=query, task=task, categories=categories)
print(json.dumps(decision, indent=2))

Output:

{
  "choice": "lost_or_stolen_card",
  "confidence": 1.0,
  "probabilities": {
    "lost_or_stolen_card": 1.0,
    "card_arrival": 0.0,
    "dispute_charge": 0.0003,
    "pin_blocked": 0.0003,
    "out_of_scope": 0.0004
  },
  "latency_ms": 7.85
}

---

Transparent Capabilities and Current Limitations

AlphaRoute-D is engineered specifically for fast, accurate decision classification. For full technical transparency, users should understand its validated strengths alongside current empirical limitations:

1. Primary Validated Capability: Choice Routing

• Validated Scope: Dynamic multi-candidate routing, intent classification, agent tool dispatch, and out-of-scope rejection.
• Empirical Validation: Rigorously benchmarked on frozen splits across Banking77 (97.00%), CLINC150 (95.40%), HWU64 (93.87%), and Golden 300 (97.67%).
• Calibrated Probabilities: Competing candidate logits are normalized via softmax over the candidate slots, providing stable, interpretable probability distributions.

2. Current Limitation: Continuous Score Regression (score)

• Status: Experimental.
• Limitation: While multi-task pre-training included scalar rating targets (e.g., continuous priority or sentiment from 0 to 1), raw unconstrained scalar predictions lack universal calibration across out-of-distribution prompts. Crucially, no RLCD (Reinforcement Learning for Calibrated Decisions) post-training was performed on AlphaRoute-D, unlike the multi-stage alignment pipelines utilized in architectures like Jev and Laya.
• Recommendation: For numerical scoring or tiering (e.g., priority classification), developers should discretize the output into explicit categories (such as {"low": "...", "medium": "...", "high": "..."}) rather than relying on raw continuous regression logits.

3. Current Limitation: Standalone Boolean Decisions (boolean / noul)

• Status: Limited stability.
• Limitation: Standalone binary assertions (evaluating a single marker without an explicit competing alternative) exhibit higher prediction variance and sensitivity to prompt formulation than comparative choice selection. As with continuous score regression, no RLCD tuning was conducted to calibrate single-marker threshold boundaries against neutral/null states.
• Recommendation: Formulate binary checks as paired, explicit choice options (for example: {"is_eligible": "Customer meets criteria", "not_eligible": "Customer does not meet criteria"}). Pitted against each other through the shared decision head, accuracy and stability improve significantly.

4. Non-Generative Architecture

• AlphaRoute-D is non-autoregressive and cannot generate conversational text, markdown responses, or complex nested JSON objects with slot extraction. If multi-field JSON schema generation or slot extraction is required, use a generative routing SLM such as AlphaRoute-0.8B-v1.5.

---

Model Positioning & Use Cases

AlphaRoute-D is suited for low-latency decision dispatch:

• Agent Tool Calling & Skill Routing: Directing user instructions to the correct tool or sub-agent in multi-agent frameworks.
• Support & Ticket Triage: Real-time classification of incoming customer inquiries into domain departments.
• Workflow Branching & Guardrails: Routing requests through security filters or compliance queues.
• Out-of-Scope Rejection: Detecting and intercepting queries that fall outside supported system scopes.

---

Citation

@misc{alpharoute_d_2026,
  author = {Naman Agnihotri},
  title = {AlphaRoute-D: Decision-Native Semantic Routing and Agent Dispatch SLM},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/NamanAgnih0tri/AlphaRoute-D-0.8B}}
}