N-Q-26M-JEV

A 26M-class decision model: a choice, a score or a yes/no with calibrated probabilities, in one forward pass

[!IMPORTANT] This is a research release, not a product. The model is good at the kinds of decisions it was trained on and well calibrated there; on new kinds of questions it is close to chance (see Honest results). Treat it as a fast, cheap classifier for the listed task families, not as a general reasoner.

What this is

A decision model in the style of TypeSafe AI's Jev: instead of generating text, it reads a state (any text), a question and a list of options, and returns a distribution over the options in a single forward pass. Three primitives:

It is N-Q-26M-Chat (Gated DeltaNet + gated attention, Russian and English) with a linear decision head. The language-model head is kept, so the checkpoint can still generate text.

Usage

from transformers import AutoModel, AutoTokenizer
tok = AutoTokenizer.from_pretrained("q-project/N-Q-26M-JEV", trust_remote_code=True)
jev = AutoModel.from_pretrained("q-project/N-Q-26M-JEV", trust_remote_code=True).eval()

state = "Write a Python function that returns the n-th Fibonacci number."
print(jev.decide(tok, state, {
    "topic": {"kind": "choice", "question": "What is this request about? Pick the topic.",
              "options": ["coding", "math", "writing", "cooking", "travel", "health"]},
    "level": {"kind": "score", "question": "How difficult is this request?", "tiers": ["easy", "medium", "hard"]},
}))
# {'topic': {'choice': 'coding', 'probs': {'coding': 0.673, 'math': 0.326, ...}, 'confidence': 0.673},
#  'level': {'level': 'medium', 'probs': {'easy': 0.258, 'medium': 0.666, 'hard': 0.077}, ...}}

jev.noul(tok, "Как сварить борщ?\n---\nВозьмите свёклу, капусту, картофель и бульон; варите 1.5 часа.",
         "Отвечает ли ответ на вопрос пользователя?", labels=("нет", "да"))     # 0.988
jev.noul(tok, "Как сварить борщ?\n---\nПитон — это язык программирования с динамической типизацией.",
         "Отвечает ли ответ на вопрос пользователя?", labels=("нет", "да"))     # 0.014

• All questions about one state run as one batch; each question sees only the state and itself.
• symmetric=True (default) also scores the options in reverse order and averages: the backbone is causal, so a single pass lets later options see earlier ones. symmetric=False is a strict single forward pass.
• Input layout, if you want to call the head directly: <bos><USER>state<EOT><R12>question<R11>option_1<R11>…option_K<R11>; the head reads the hidden state at the <R11> that closes each option (NQ4ForDecision.option_logits).

Honest results

Held-out test splits, symmetric mode. ECE = expected calibration error (0 = confidence matches accuracy).

Task types never seen in training (zero-shot):

At 26M parameters the model learns every task family it is trained on, with honest probabilities, but it does not become a general decision model: new kinds of questions are answered near chance. Questions that need world knowledge or reasoning (OpenBookQA, PARus) are out of reach at this size.

A failure you will meet: "Доставили не тот размер, хочу вернуть деньги, а поддержка не отвечает уже третий день!" is classified as neutral (0.59) rather than negative (0.36) — the Russian sentiment training data is noisy.

Training

Full fine-tune of N-Q-26M-Chat plus the new head, log-loss over the options (a proper scoring rule, which is what keeps the probabilities calibrated). Option order is shuffled for choice and randomly reversed for score/noul.

Adding the new families did not hurt the old ones (all within ±1–2 points) and lifted the new ones from their zero-shot level (NLI 38% → 65%, sentiment 44% → 78%, toxicity 36% → 80%, news 67% → 89%).

Packed single-file format (model.qpack)

Same QPACK1 container as the other N-Q releases (89 MiB): a folder with model.qpack and no *.safetensors loads through AutoModel.from_pretrained(..., trust_remote_code=True); python qpack.py --selftest model.qpack prints SELFTEST 5/5.

Limitations

• Near chance on task types it was not trained on; no world knowledge or multi-step reasoning.
• Not a calculator: the exact <CALC> circuits of the chat model work only when generating text.
• Russian sentiment and answer-quality labels are noisy, and so are the model's judgements there.
• Causal backbone: use symmetric=True (default) to remove the option-order bias.
• No safety tuning. Research and education only.

License and data terms

Code and weights: Apache-2.0 (see LICENSE). Training data keeps its own terms — check each dataset card before commercial use: saiga_scored, saiga_preferences, UltraFeedback, SmolTalk, GrandMaster-PRO-MAX, XNLI, the RU sentiment collection, SST-2, ru_paradetox_toxicity, AG News; plus the pretraining data of N-Q-26M.