ymoslem/ModernBERT-base-TeleMath-router-qe-classifier-binary-10ep-lr2e-05-q30-think-5runs

Quality estimator for the Stage 2 cascade of CRE-Router on TeleMath, for Qwen3-30B-A3B-Thinking-2507 (FP8). Gates clusters 0 and 1 of the two-card TPOT routing (B = 20 ms), whose Stage 1 sends them to Qwen3-30B-Thinking; rejected answers escalate to Gemma4-26B with reasoning.

Given an efficient model's answer, it predicts whether to accept it (label 1) or escalate the query to a stronger model (label 0). An answer is escalated when the probability of acceptance is below 0.5.

Input format

question [SEP] full_output [SEP] num_tokens

full_output is truncated to its last 1,000 words before tokenisation, and the tokeniser runs at max_length=4096.

Training

ModernBERT-base, up to 10 epochs, lr 2e-5, batch size 64, max_length 4096, class-weighted cross-entropy. The training set is the model's own answers on the 299 TeleMath training questions, five sampled runs each (1,495 rows: 1074 accept, 421 escalate), labelled by task correctness. Training stops early, with patience 4, and keeps the checkpoint with the best macro-F1 on one run of the 201 TeleMath test questions (132 accept, 69 escalate); the kept checkpoint is epoch 5 of 10, macro-F1 0.734.

Intended use

Deployed only where Stage 1 routes to Qwen3-30B-A3B-Thinking-2507. It is trained on that model's own outputs and does not transfer to another model's outputs.