RouteFM: Pretrain Once, Route Anywhere

Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing
Guannan Lai and Han-Jia Ye · Nanjing University

Project page · Live demo · Run in Colab · Routing collection · GitHub · Paper · PDF

RouteFM is a pretrained in-context model router. Given an anonymous candidate pool, behavioral observations of candidate quality and relative cost, and a new target query, it predicts the target-specific quality and relative cost of every candidate. A frozen RouteFM adapts to new routing environments through context alone, without target-domain parameter updates.

On MMR-Bench, which is excluded from pretraining, RouteFM outperforms the strongest non-RouteFM baseline by 2.23 quality points with only eight observations per candidate.

Video overview

90-second introductions in English and Chinese, with matching narration, on-screen text, and captions.

English

Download English MP4 · English subtitles

中文

下载中文版 MP4 · 中文字幕

Watch on the project page · All video assets and checksums

Concept footage and narration are AI-generated; method diagrams and results use original paper materials. The final composition is 1080p; the concept footage is native 720p.

Interactive demo

The RouteFM demo provides a free interactive explorer for switching among environments, target queries, and context budgets. Its predictions were precomputed with the released RouteFM-BGE checkpoint; the built-in behavioral contexts are synthetic illustrations, not benchmark records. For real inference with arbitrary target queries and custom context, use the one-click Colab or run the full Gradio app in the GitHub repository.

Model description

RouteFM turns LLM routing from repeated local fitting into global routing pretraining. Candidate identities, providers, parameter counts, and other explicit identity features are never exposed to the router. Instead, it:

1. builds a compact capability profile for every anonymous candidate from behavioral context;
2. retrieves complementary evidence from both the profile and the original observations for each target query;
3. compares the current pool jointly with a permutation-equivariant Transformer; and
4. predicts target-specific quality and relative cost.

The released architecture uses 24 capability tokens, hidden dimension 256, and episodic pretraining with candidate-slot permutation. The training objective combines point prediction, pairwise ranking, and routing regret. See the paper for the complete method.

Released variants

Each variant contains a canonical model.safetensors and config.json. The legacy/ directory contains the exact PyTorch checkpoint bytes from the GitHub v1.0.0 release. manifest.json records sizes, configurations, immutable revisions, and SHA-256 digests. The root config.json indexes both variants and is the standard Hugging Face query file used for repository download statistics.

The query encoders are frozen external feature extractors. They are not included in this repository, and RouteFM is not a fine-tune of either encoder. Users must obtain the encoders or compatible embedding services separately and comply with their licenses and access terms.

Usage

Install the official package from PyPI:

python -m pip install "routefm-router[bge]"

The package downloads the correct safetensors checkpoint from the immutable Hub artifact revision, validates its SHA-256 digest, and stores it in the standard Hugging Face cache.

routefm-predict --encoder qwen --input my_episode.npz \
  --output predictions.json --device cpu

To prefetch both released variants for offline jobs:

routefm-download --encoder all

The input schema, embedding instructions, observation masks, local checkpoint overrides, and output format are documented in docs/CUSTOM_DATA.md.

Training

Both variants were trained from random initialization in one continuous 10,000-update run. All router modules were trained jointly, while the external query encoder remained frozen. The configured pretraining mixture is LLMRouterBench 45%, RouterBench 25%, RouterEval 25%, and MixInstruct 5%. MMR-Bench is not a pretraining source.

The complete data contract, source proportions, episode construction, curriculum, and optimization commands are available in docs/PRETRAINING.md. Underlying third-party datasets and embedding models are not redistributed.

Evaluation

The paper evaluates the frozen Qwen-based RouteFM on held-out RouterEval queries and on cross-modal transfer to MMR-Bench.

These are the paper's matched-protocol results. MMR-Bench uses dataset-wise five-fold context/target splits in the limited-observation setting and a 40%/60% split in the large-observation setting. RouteFM remains frozen and uses no evaluation labels for parameter updates.

The GitHub release additionally publishes a fixed seed-31010 split for artifact auditing. That split is retrospective and post-selected, differs from the paper's five-fold protocol, and should not be compared directly with the table above. Its exact IDs, hashes, aggregation rules, and reference outputs are documented in docs/MMRBENCH_V1.md.

Intended use and limitations

RouteFM is intended for research on routing among a user-supplied candidate pool with observed context outcomes. It does not call candidate models, generate query embeddings, or establish that a candidate is safe or suitable.

• Candidate order and embedding family must remain consistent within an episode.
• Each candidate needs at least one valid context observation.
• Predicted cost is relative, not a calibrated monetary or latency estimate.
• The default decision rule ignores predicted cost and selects maximum predicted quality.
• Performance may change across encoder revisions, languages, domains, candidate pools, and context sizes outside the training distribution.

License

RouteFM source code and released routing weights are licensed under Apache-2.0. Third-party assets retain their own licenses and terms; see THIRD_PARTY_NOTICES.md.

Citation

If you use RouteFM in your research, please cite:

@misc{lai2026pretrainoncerouteanywhere,
  title         = {Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing},
  author        = {Guannan Lai and Han-Jia Ye},
  year          = {2026},
  eprint        = {2609.37362},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  url           = {https://arxiv.org/abs/2609.37362}
}