linne-fern-1.0
linne-fern-1.0 is a 27B decision model. Give it a state (text, a document, JSON, a conversation or an agent trajectory) and a question with a set of options, and it returns one option together with a probability for every option, in a single forward pass.
It is built for the decisions that sit inside real products and agent loops: classification, routing, fact checking, choosing the right tool response, and approving what an agent is about to do.
Highlights
- Decision Index public score 58.91, measured with the official kit (edition 0.3, all 37 public benchmarks, 140,620 requests).
- Broad: knowledge, language understanding, retrieval and classification, tools and automation, in English, Vietnamese and Arabic.
- Agent-safe: strong step-level guard for agents (97.7% caught at 0.3% false blocks on our held-out agent steps) and a low false-block rate on benign prompts.
- Fast: one forward pass per decision, about 46 ms p50 on a single H200.
- Drop-in: standard Qwen3.8 architecture and chat template; runs on transformers, vLLM and SGLang.
Decision Index (public part, edition 0.3)
Chance-corrected skill, higher is better. All rows measured by us with the official kit and the same serving path, except the last row, which is the public-part score published on the leaderboard.
| Model | Overall | Knowledge | Language | Retrieval | Tools | Arts |
|---|---|---|---|---|---|---|
| linne-fern-1.0 | 58.91 | 45.0 | 62.8 | 62.2 | 79.0 | 41.6 |
| pplx-decider-v1-27b | 55.55 | 39.1 | 61.7 | 57.2 | 75.9 | 41.8 |
| qjev+ (27B) | 50.48 | 32.5 | 53.3 | 55.3 | 73.2 | 38.6 |
| Quyet-1.0-Large (31B), leaderboard public part | 60.79 |
Only the public part (20% of the full leaderboard score) can be measured outside the leaderboard; the full score also includes private tests run by the maintainers.
Safety and agent benchmarks
| Benchmark | linne-fern-1.0 | qjev+ | Quyet-1.0-Large |
|---|---|---|---|
| ATBench, accuracy (1,000 agent trajectories) | 72.4 | 73.6 | 58.2 |
| ATBench, false-block rate (lower is better) | 14.3 | 15.7 | 28.2 |
| NotInject, false-block rate (339 benign, lower is better) | 1.8 | 2.1 | 6.8 |
| Prompt protection, accuracy (385) | 97.9 | 97.9 | 97.9 |
| ViCulturaBench, accuracy (4,000) | 91.5 | 91.5 | 95.2 |
| Latency p50, one H200 (ms) | 46 | 47 | 33 |
Usage
The model answers by putting probability on the option letter that follows a fixed prompt: the state, a blank line, the question,
one line per option (A: name - description) and Answer with the letter of one option only. Use the chat template with thinking
disabled and read the next-token probabilities of the option letters.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "beyoru/linne-fern-1-0"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, dtype=torch.bfloat16, device_map="auto")
state = "My Stripe integration keeps failing. Please help ASAP."
question = "Which team should handle this request?"
options = {"billing": "Charges and refunds", "technical_support": "Integration errors", "sales": "Questions about buying a product"}
letters = [chr(65 + i) for i in range(len(options))]
lines = [f"{l}: {k} - {v}" for l, (k, v) in zip(letters, options.items())]
user = "\n".join([state, "", f"Question: {question}", *lines, "Answer with the letter of one option only."])
prompt = tok.apply_chat_template([{"role": "user", "content": user}], tokenize=False,
add_generation_prompt=True, enable_thinking=False)
ids = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
with torch.no_grad():
logits = model(**ids).logits[0, -1]
label_ids = [tok(prompt + l, add_special_tokens=False)["input_ids"][-1] for l in letters]
probs = torch.softmax(logits[label_ids].float(), dim=0)
print({k: round(float(p), 4) for k, p in zip(options, probs)})
It also works as a regular chat model, and with structured output (JSON with an enum field, thinking disabled) for typed decisions.
License
Apache 2.0, same as the base model.