KnowLine-26B-A4B-Gen2.5

KnowLine-26B-A4B-Gen2 with a new way of serving it: the same weights, and a front end that reads short requests twice before it answers.

Inference guide · 中文说明 · weights Apache-2.0 · previous version: KnowLine-26B-A4B-Gen2 · sibling: KnowLine-35B-A3B-Gen1.5

We raise the minor version when only inference changes and the major version (Gen) when the weights change. Every weight, config and tokenizer file of Gen2.5 is byte-identical to Gen2.

Highlights:

• Same weights, read twice. For a text-only request of up to 3,000 tokens, the front end writes the whole input a second time before the model answers. Nothing was trained; see What changed in Gen2.5.
• Decision Index 0.3 public suite: 68.21 (self-run, fresh full run), up from 66.94 for Gen2 with the same weights. The gains are concentrated in reasoning and inference over long inputs (knowledge and reasoning 51.9 → 55.1, language 69.7 → 71.2); tools and agents are slightly lower (84.4 → 83.3). See Comparison.
• Decision tasks from new domains: on our larger set of decision tasks from domains not in training (34 tasks) it scores 66.4, against 60.6 for Gen2 with the same weights on the same harness. On our older 10-task set it scores 56.6 (Gen2: 55.8).
• Private scores: the board's full score comes from the maintainers' private run. Gen2 is listed at 61.34. Gen2.5 has the same weights as Gen2; its private scores will be produced by the maintainers' run.
• Our other evaluations: Open-Jev 1.1 test / OOD 92.7 / 92.7 (Gen2 on the same files: 88.5 / 87.9), C-Eval 77.9 (74.8), JevBench / JevBench-hard 90.5 / 81.1 (88.3 / 76.6), and slightly higher scores on our Chinese in-house held-out sets (English flat). Planted instructions changed 0.1% of answers (Gen2: 0.1%), and 1.7% with injection templates not seen in training (Gen2: 1.0%; the difference is not significant).
• Calibration: the temperature folded into the weights is unchanged (T = 1.25). Computed with the board's method, ECE is about 0.013-0.018 (Gen2: 0.008-0.012), slightly higher, and the Brier score about 0.24-0.27 (Gen2: 0.26-0.28); see Calibration.
• Cost: about 1.54 times the prompt tokens of Gen2 on the Decision Index suite. One request at a time on one H20: 66.3 ms median and 151 ms mean, against 43.1 ms and 82.8 ms for Gen2 (our measurement, not the board's reference hardware); see INFERENCE.md.
• Images: unchanged. Requests with images are sent exactly as in Gen2, so Gen2's image results apply; see Images.
• KOF '98 harness: 18-0-0 against Jev, 16-2-0 against StartLux-Decision-35B-A3B and 5-11-2 against KnowLine-35B-A3B-Gen1 in single-bout mirror matches; see Game harness.

KnowLine is an independent model. It is not affiliated with or endorsed by TypeSafe, Jev or Google, and it is not derived from TypeSafe or Jev.

What it does

You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model answers every question with a probability distribution over its options:

• one forward pass per question, with no text generation;
• the output is the probability of each option's label token;
• existing Jev clients only need to change the base URL, and users of other KnowLine models only need to change the model name, because the interface is the same.

Quickstart

pip install "sglang==0.5.21" "transformers==5.12.1" requests
bash serve_knowline.sh PelaAI/KnowLine-26B-A4B-Gen2.5 0 8080   # SGLang (bf16) on :9080 + /v1/systemone on :8080
curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "m",
  "state": "Customer: my order arrived broken, I want my money back.",
  "questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},
                "tone": {"type": "choice", "instructions": "Customer tone?",
                         "criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'

• Front end: knowline_server.py with knowline_layout.py next to it; together they depend only on transformers and requests.
• Full settings: the exact settings used for our Decision Index run are in INFERENCE.md.

What changed in Gen2.5

1. Read twice. Gen2 saw each request once: the state, then the instruction, the question and its options. Gen2.5 writes all of that a second time, introduced by the line "Read the same input again before answering:", and then reads the answer from the label token as before. The rule is mechanical: it applies to every text-only request of at most 3,000 tokens (94.1% of the Decision Index requests); longer requests and requests with images are sent as in Gen2.
2. A likely reason. With causal attention, the first copy of a long state is read before the model knows the question; in the second copy every token can also attend to the first copy, question included. This fits where the gains are: reasoning and inference over long inputs.
3. No training. The weights, the temperature and the label-token readout are those of Gen2. We checked the change on our own evaluation sets and on a fresh full run of the Decision Index public suite before release. It is the same change as in KnowLine-35B-A3B-Gen1.5.
4. Credit. The read-twice layout follows FlexDM (github.com/1000luca/FlexDM, MIT); repeating the input goes back to RE2 (Xu et al., 2023, arXiv:2309.06275). knowline_layout.py is our own implementation.

Comparison

Scores on the Decision Index. Since Decision Index 0.4, the board's full score is 60% private tests of the same skills and 40% private tasks from new domains; the public suite is shown but no longer counts. Gen2.5 has no board entry yet.

Board rows from the board's data of 2026-10-21 (22:34 UTC, Decision Index 0.4.1).

Public-suite skill by area (DI 0.3, same weights):

Releases

The model is trained in a self-evolving loop, and new versions will follow.

Evaluation

All results are self-run and have not been verified by a third party. Every Gen2.5 number below was measured through the Gen2.5 front end (read twice on). Unless noted, the Gen2 numbers were measured again on the same files.

Held-out and Chinese evaluations

• In a paired test against Gen2, p < 0.001 for Open-Jev 1.1 test and OOD and for C-Eval, and p = 0.04 for the Simplified Chinese in-house set; the other differences have p > 0.05 (English in-house set p = 0.89, Traditional Chinese 0.16, JevBench 0.23, JevBench-hard 0.18). With unseen injection templates, 10 of 600 answers changed against 6 of 600 for Gen2 (p = 0.45).
• The Gen2 column matches the Gen2 card.
• ¹ Our sets of decision tasks from domains not in training, re-run for both models under the same serving (FP8 at load, read twice through our evaluation front end, which uses the same layout); the Gen2 card's values (60.8 and 55.8) came from the serving of the time.

Web operation (Mind2Web official test splits, evaluation only)

• Element selection is slightly higher than with Gen2 on all three splits (paired test p = 0.04 on test_domain, p ≥ 0.17 on the other two); operation accuracy is within noise of Gen2 (p ≥ 0.18). No request failed.

• The task is to pick the target element from a candidate set: the target plus up to 5 other elements sampled from the page. This is easier than the original Mind2Web protocol, so do not compare these results with the Mind2Web leaderboard.

Images (zero-shot)

• Unchanged from Gen2: read twice applies only to text-only requests. A request with images is sent exactly as in Gen2, byte for byte, to the same weights, so Gen2's image results hold for Gen2.5 and we did not rerun them.
• Gen2's results: 64.7 on our reconstruction of the Decision Index 0.3.1 vision public suite and 63.3 on the official vision public suite (self-run); see the Gen2 card for the per-benchmark table.

Game harness (KOF '98)

• Setup: single-bout character-mirror matches, 18 games per pair, argmax actions; the same settings as for Gen2.
• Result: 18-0-0 against Jev (TypeSafe API, jev-latest on 2026-10-22), 16-2-0 against StartLux-Decision-35B-A3B and 5-11-2 against KnowLine-35B-A3B-Gen1; 39-13-2 overall, a score of 0.741 [0.61, 0.84] (Gen2: 16-2-0, 17-1-0 and 6-6-6; 39-9-6 overall).
• How the games ended: against Jev, 17 of the 18 wins were by K.O.; against StartLux-Decision-35B-A3B, 10 of 16 by K.O., and both losses came at time-out; against KnowLine-35B-A3B-Gen1, all 5 wins by K.O., 5 of the 11 losses by K.O., and both draws at time-out.
• Play style: special_2 39%, heavy kick 29%, special_3 11%, heavy punch 8%, wait 8%.
• Caveats: 18 games per pair give wide intervals; against KnowLine-35B-A3B-Gen1 the result is below Gen2's 6-6-6 but within those intervals. This is a measured result in this harness, not general fighting-game skill.

Calibration

Computed on the 0.3 public suite using the method described by the Decision Index board: each field is scored right or wrong, confidence is the probability of the chosen option, benchmarks are weighted equally, and 10 equal-width bins are used.

• Temperature in the weights: unchanged from Gen2: the final norm weight and the final logit soft-cap include T = 1.25, so every final logit is divided by 1.25 before the softmax over the option labels.
• Why ranges: the board does not publish which benchmarks it uses for calibration, so we report ranges over the plausible benchmark sets, as for Gen2.
• Recommendation: if your use depends on probability thresholds, check calibration on your own data.

Disclosures

• Training data: the weights are those of Gen2, so Gen2's disclosures on training data, Decision Index format data and decontamination apply unchanged; see the Gen2 card.
• Selection on evaluations: we adopted read twice after checking it on our own evaluation sets and on a fresh full run of the Decision Index 0.3 public suite (the 68.21 above).
• No model outputs as labels: no output of Jev or any other decision model was used as a training label.

Limitations

• Slower than Gen2: about 1.54 times the prompt tokens on the Decision Index suite; one request at a time on one H20, 66.3 ms median and 151 ms mean (Gen2: 43.1 ms and 82.8 ms), and 629 ms at the 95th percentile (Gen2: 238 ms). Use --no-read-twice if latency matters more than these gains.
• Only short requests change: text-only requests over 3,000 tokens, and requests with images, are answered exactly as by Gen2.
• Not better everywhere: a few benchmarks, including some tool and agent tasks, are slightly lower than with Gen2, and unseen injection templates changed 1.7% of answers against 1.0% (not significant).
• Everything else as Gen2: Chinese is still clearly below the 35B-A3B (C-Eval); web operation and images are below the 35B-A3B; other prompt-injection attacks may still work; part of the public-suite score comes from benchmarks whose tasks are in our training data, so the private tests may show a lower score.

Citation

@misc{knowline26ba4bgen25,
  title  = {KnowLine-26B-A4B-Gen2.5: a 26B-A4B decision model served with read twice},
  author = {PelaAI},
  year   = {2026},
  url    = {https://huggingface.co/PelaAI/KnowLine-26B-A4B-Gen2.5}
}

License

• Weights: Apache-2.0, the same as the base model.
• Code: knowline_server.py, knowline_layout.py and knowline_engine.py are MIT (see the file headers).