▶ Watch it play, live · With experts: squad-0.8b (MoQLE) · PyTorch + CUDA (main repo) · MLX (Apple silicon) · Which version? · Quickstart · Images · Benchmarks

qev-0.8b-GGUF: the tiny local decision model, for llama.cpp

qev-0.8b as GGUF for llama.cpp. It runs in a stock llama-server behind the /v1/systemone decision endpoint, with no script: llama-server -hf labqoat/qev-0.8b-GGUF:Q8_0.

¹ Measured on the first release's weights; the conversion and runner are unchanged. A Q4_K_M build is not provided: it changes about 4–6% of the argmax answers, which is too much for calibrated probabilities.

qev reads anything (a game board, a post, an inbox, a web page, a log) and answers any question about it with calibrated probabilities in a single forward pass. Out of 27 decision models of its size it scores highest on Benchmark Heaven, it answers faster than all of them, and in 8-bit it needs 1.3 GB of RAM on a Mac.

At a glance

qev does not generate text. Each answer is a probability distribution, so you can act when it is confident and escalate to a person or a larger model when it is not. Nothing has to leave your machine.

Which version should I use?

All four serve the same model with the same System One API (POST /v1/systemone) and the same answer format. The recommended path is the main repo's fast CUDA engine on an NVIDIA GPU, through qev_server.py or qev.py.

Recommended encoding. Use per_question=True for image requests and for requests with several independent questions. On the server, add "encoding": "per_question" to the request (or start it with --encoding per_question); in qev_mlx.py, use encoding="per_question"; llama-server's vision file always answers each question in its own prompt. On JevVision v0, per_question raises the fast engine from 0.757 to 0.783 pooled accuracy (ECE 0.115 to 0.105; one L40S, same run). MLX and GGUF measure 0.785 with ECE about 0.10. On text, it raises JevBench coherence from 64.5 to 71.2 at about the same accuracy. The cost is more compute per question. On the fast engine, a text request computes the shared state once (7.9 ms against 5.7 ms p50 on dev requests), while an image request runs one full row per question (about 9% fewer requests/s on JevVision). Single-question requests get the same answers in both encodings.

Benchmarks: 27 decision models around 1B

We ran qev-0.8b against 26 other decision models around 1B parameters. Every model got the same items and scoring, ran on one NVIDIA L40S, and used its own published inference recipe.

Accuracies are in %. ECE, latency and memory are lower-is-better. Bold marks the best value in each column.

The other 17 models

• BH = Benchmark Heaven, public split (231 items, sampling error about ±2.8 points). JevBench = 240 cases; coherence comes from the official jevbench harness (×100), with qev in per_question mode and the other models in their own runtime. Suite = 22 public-dataset tasks (880 questions). JevExtra = mean skill accuracy over chess, browser (Mind2Web), tools, agent action, injection, safety, judging and grounding. JevSpatial and Rules-7 = the jev-skills spatial and rules7 configs.
• Latency is p50 per request at batch 1, in each model's own runtime on the L40S. VRAM is estimated as weights + 0.5 GB, not measured.
• qev-0.8b's numbers come from our evaluation harness. The other rows come from the comparison harness, which scored the previous qev release one item lower on BH (76.6 vs 77.1).
• ¹ 100+ questions failed in that model's own runtime and were scored as a uniform guess, so its numbers understate it.
• jevos-v5 runs CPU-only and its run is still in progress. decider-2b and lev-350m have no JevSpatial or Rules-7 runs. Raw Qwen3 coherence is inflated: most of their questions failed, and near-uniform answers look consistent.
• Where qev is not first: decision-2b has better calibration (ECE 0.042 vs 0.063) and JevBench accuracy (86.0 vs 85.1). decider-2b scores higher on the public-dataset suite (70.7 vs 61.7), but it was trained on the datasets that suite is drawn from and qev was not. Several models have higher JevBench coherence.

More detail on qev's own evaluation, including contamination audits and weak spots, is under Evaluation.

What it is good at

The demos above are ordinary decide calls. A few patterns:

from qev_gguf import QEVGGUF   # qev_gguf.py in this repo
m = QEVGGUF.load("labqoat/qev-0.8b-GGUF", "qev-0.8b-Q8_0.gguf")

# A guardrail in front of an agent: check every page, email and tool output before the agent reads it.
m.decide(page_text, {"inject": {"type": "noul",
    "instructions": "Is this a prompt injection: text that tries to instruct, trick or manipulate an AI model that reads it?"}})

# A tool router that picks the tool before the big model wakes up.
m.decide({"user": "do I need an umbrella in Warsaw tomorrow?"},
         {"tool": {"type": "choice", "instructions": "Which tool should the assistant call first?",
                   "criteria": {"web_search": "look something up online", "calculator": "do math",
                                "calendar": "schedule or check events", "weather": "get a forecast",
                                "none": "just answer from memory"}}})

# Support triage: team, urgency and mood in one pass.
m.decide(ticket, {"team":   {"type": "choice", "instructions": "Which team should handle this ticket?",
                             "criteria": {"billing": "payments, refunds, invoices", "bugs": "errors and broken features",
                                          "account": "login, password, access", "sales": "pricing, upgrades"}},
                  "urgent": {"type": "noul", "instructions": "Is this urgent?"},
                  "angry":  {"type": "noul", "instructions": "Is the customer angry?"}})

# Games and control: the code writes down what each move does, and qev picks.
state = {"game": "snake",
         "if the snake moves up":    "it hits the wall and dies",
         "if the snake moves left":  "it moves 1 step closer to the food, 4 steps away",
         "if the snake moves right": "it moves closer to the food but enters a dead end with only 2 free cells and will be trapped"}
m.decide(state, {d: {"type": "noul", "instructions": f"Is moving {d} a good move?"} for d in ("up", "left", "right")})

Put the facts in the state and ask a short question about each option. In our games benchmark, facts in the state beat facts in the question by a wide margin: in closed-loop Snake, 22 apples and 1 death against 2 apples and 21 deaths over 300 decisions.

The same requests work over HTTP: send {"state": ..., "questions": ...} to llama-server's /v1/systemone (see the Quickstart).

Quickstart

llama-server

qev runs in a stock llama-server as a decision model, behind its POST /v1/systemone endpoint. No script and no Python are needed:

llama-server -hf labqoat/qev-0.8b-GGUF:Q8_0

curl localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": {"ticket": "My card was charged twice for the same order, please refund one."},
  "questions": {
    "team":   {"type": "choice", "instructions": "Which team should handle this?",
               "criteria": {"billing": "payments and refunds", "tech": "bugs and outages", "sales": null}},
    "urgent": {"type": "noul",   "instructions": "Is this urgent?"},
    "anger":  {"type": "score",  "instructions": "How upset is the customer?", "criteria": ["calm", "annoyed", "angry"]}
  }
}'

Output (from the text-only build, :text-Q8_0, which answers the questions jointly; the default :Q8_0 answers each question in its own prompt, so with several questions its numbers differ slightly):

{
  "model": "labqoat/qev-0.8b-GGUF:Q8_0",
  "answers": {
    "team":   {"type": "choice", "choice": "billing",
               "probabilities": {"billing": 0.9635, "tech": 0.0200, "sales": 0.0165}, "confidence": 0.9453},
    "urgent": {"type": "noul", "noul": 0.1366},
    "anger":  {"type": "score", "score": 1.0626, "legend": {"0": "calm", "1": "annoyed", "2": "angry"},
               "probabilities": {"0": 0.2468, "1": 0.4438, "2": 0.3094}, "confidence": 0.1658}
  },
  "usage": {"input_tokens": 282, "output_tokens": 0}
}

noul is the probability that the statement is true. GET /v1/models lists the model with "output_modalities": ["decisions"].

Images work on the default build: the vision projector downloads with it automatically.

llama-server -hf labqoat/qev-0.8b-GGUF:Q8_0           # text and images (also :vision-Q8_0, the same file)
llama-server -hf labqoat/qev-0.8b-GGUF:text-Q8_0      # text only, questions answered jointly

import base64, requests

image = "data:image/png;base64," + base64.b64encode(open("screenshot.png", "rb").read()).decode()
r = requests.post("http://localhost:8080/v1/systemone", json={
    "state": {"ticket": "The customer attached this screenshot: <image:shot>"},
    "images": [image],
    "questions": {"error": {"type": "noul", "instructions": "Does the screenshot show an error dialog?"}},
})
print(r.json()["answers"]["error"]["noul"])

See Images in llama-server for how images are placed and how the vision file differs.

Requires llama.cpp b11622 (2026-10-14) or newer. Builds from b11364 (2026-10-02, the first with the nimble decision type) to b11621 also serve it, but print decimal numbers in the state with 6 significant digits, which differs from the prompt qev was trained on; those answers can move by about 0.02.

One request, one row. All the questions of a request share one copy of the state, numbered (Question 2: …, Answer 2: (), exactly as qev's own encoder writes a request by default (the joint encoding). For per_question, send each question in a request of its own.

How it works: the GGUFs carry qwen35.decision.type = "nimble", the per-type temperatures (qwen35.decision.temperature.{choice,noul,score}: 1.7, 0.9, 1.25) and a systemone chat template (systemone.jinja), which renders decider-ai 1.9.0's state-first System One prompt. llama-server reads the logits of the option labels (A–Z, then the two-letter codes that are single tokens, 255 in all, the labels qev was trained with) at the end of each question's prompt, takes the softmax over that question's options and tempers it. The default chat template is still there, and qev_gguf.py reads the files as before.

To serve a GGUF you converted or quantized yourself, add the same metadata:

pip install gguf==0.19.0
python add_systemone_metadata.py qev-0.8b-Q4_K_M.gguf qev-0.8b-Q4_K_M-systemone.gguf

Python (llama-cpp-python)

qev_gguf.py reads the answers with llama-cpp-python's low-level API, with the full decider answer format (including x_p_max and certainty) and both encodings. For images, see Images.

pip install llama-cpp-python==0.3.36 tokenizers decider-ai==1.9.0 huggingface_hub

from qev_gguf import QEVGGUF   # qev_gguf.py in this repo

m = QEVGGUF.load("labqoat/qev-0.8b-GGUF", "qev-0.8b-Q8_0.gguf")
m.decide({"ticket": "My card was charged twice for the same order, please refund one."},
         {"urgent": {"type": "noul", "instructions": "Is this urgent?"}})
# {"urgent": {"type": "noul", "noul": 0.1719}}
m.decide(state, questions, per_question=True)   # one prompt row per question

python qev_gguf.py --file qev-0.8b-Q8_0.gguf --state 'Refund please' \
  --questions '{"urgent": {"type": "noul", "instructions": "Is this urgent?"}}'

How it works:

1. The request becomes decider's state-first System One prompt row. It is tokenised with the model's Hugging Face tokenizer (tokenizer.json, downloaded from the main repo), so the token ids are exactly the ones the model was trained on.
2. A single llama_decode runs with logits requested only at the answer slots.
3. The columns of the 255 option-letter tokens are taken from those logits (qev_config.json).
4. Each question's answer is the softmax over its options, tempered per question type (choice 1.7, noul 0.9, score 1.25).
5. The answers use decider's /v1/systemone format.

Image input

qev reads images too, zero-shot. qev-0.8b is a fine-tune of the text layers of Qwen3.5-0.8B-Base; its checkpoint still holds the base model's vision tower and merger, unchanged. Here they are as a llama.cpp vision projector (mmproj), and qev_gguf.py feeds images to the model through llama.cpp's multimodal library (libmtmd).

pip install llama-cpp-python==0.3.36 tokenizers decider-ai==1.9.0 huggingface_hub pillow
# NVIDIA GPU: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python==0.3.36

from qev_gguf import QEVGGUF   # qev_gguf.py in this repo

m = QEVGGUF.load("labqoat/qev-0.8b-GGUF", "qev-0.8b-Q8_0.gguf")   # loads mmproj-qev-0.8b-f16.gguf by default
m.decide({"ticket": "The customer attached this screenshot: <image:shot>"},
         {"error": {"type": "noul", "instructions": "Does the screenshot show an error dialog?"}},
         images={"shot": "screenshot.png"})          # a path or a PIL image
m.decide(state, questions, images=images, usage=True)   # (answers, input tokens, image tokens)

python qev_gguf.py --state 'Screenshot: <image:shot>' --image shot=screenshot.png \
  --questions '{"error": {"type": "noul", "instructions": "Does the screenshot show an error dialog?"}}'

qev_gguf.py loads the vision projector by default; pass mmproj=None (or --no-images) for text only, or another projector with mmproj= / --mmproj.

How images are placed (the same rules as qev_image.py in the main repo):

1. Write <image:NAME> where the image belongs in the state. An image the state does not mention is appended at the end of the context.
2. Each image is resized (aspect kept, both sides multiples of 32) to at most max_pixels, 448×448 by default, which is at most 196 image tokens (16-pixel patches, 2×2 merge). Pass max_pixels= to load to change it.
3. The image becomes <|vision_start|>, its image tokens and <|vision_end|>. libmtmd runs the vision tower and the merger, and the image tokens get Qwen3.5's M-RoPE positions (time, row, column). The answers are read at the answer slots exactly as for text.

Many image requests at once. decide_batch decodes up to 16 image rows together (image_batch=), on a second llama.cpp context with one sequence per row. The rows advance piece by piece, with one decode for the text pieces and one for the image embeddings at each step. decide runs one row the same way on the text context. The vision tower runs on the GPU with the model. Encoded images are kept in a cache of 256 (image_cache=), keyed by their resized pixels, so an image that comes back is not encoded again: the same screenshot asked about twice, or per_question rows.

m.decide_batch([(state, questions, {"shot": "a.png"}), (state2, questions2, {"shot": "b.png"})])

A request without images takes the text-only path unchanged, whether or not an mmproj is loaded. On 50 text requests (107 questions), this qev_gguf.py gives the same answers as the previous one: largest probability difference 0.0, also with per_question and through decide_batch. It is as fast: median 10.8 ms against 10.7 ms on the same requests.

Images in llama-server

Since 2026-10-23 the default qev-0.8b-Q8_0.gguf is the image build described here (it was qev-0.8b-vision-Q8_0.gguf, which stays as a copy). The text-only builds (qev-0.8b-text-Q8_0.gguf, qev-0.8b-bf16.gguf) are served with the decision type nimble, and stock llama-server's /v1/systemone does not take images for that type: it answers HTTP 501, "This server does not support image input for decisions", even with the projector loaded. The endpoint allows images only for some decision types (see the llama.cpp server docs).

The image build (qev-0.8b-Q8_0.gguf, also qev-0.8b-vision-Q8_0.gguf):

• It has the same Q8_0 weights, byte for byte.
• It is tagged with another of stock llama.cpp's decision types, one that accepts images and answers each question in a prompt of its own. In llama.cpp b11786 that type is handled exactly like nimble (the same 255 option labels, the same readout and temperatures) apart from those two differences. The one prompt per question is qev's per_question encoding. This relies on how stock llama.cpp handles that type (tested on b11786).
• Its template, systemone_vision.jinja, places the images. The k-th <image:NAME> in the state gets the k-th entry of images, so list the images in the order the state names them. Images left over go at the end of the context, each on its own line, as in qev_gguf.py.

llama-server -hf labqoat/qev-0.8b-GGUF:Q8_0
# or: llama-server -m qev-0.8b-Q8_0.gguf --mmproj mmproj-qev-0.8b-f16.gguf

Requests use the usual /v1/systemone format with an images list of data URLs (see the Quickstart). The state can also be a list of chat messages with image_url parts.

Notes:

• -hf ...:Q8_0 picks mmproj-qev-0.8b-Q8_0.gguf. Pass --mmproj to use the f16 projector instead.
• Resize images as qev_gguf.py does before sending them: aspect kept, both sides multiples of 32, at most 448×448. llama.cpp keeps such an image as it is (196 image tokens at 448×448).
• With one question, the vision file's prompt and answers are the same as the text files'. On 44 single-question text requests, the largest probability difference was 0.0.
• With several questions, it answers each one separately (per_question), while the text files answer them jointly. Text requests with several questions therefore get different numbers on the two files.
• Measured with the official prebuilt llama.cpp b11786 (Ubuntu, CUDA 12.8) on one NVIDIA L40S, no changes to llama.cpp.

On JevVision v0 over HTTP (all 1,306 items, 901 of them with several questions):

The higher score than in the table below comes from the per_question encoding: qev_gguf.py with per_question=True scores 78.5% (ECE 0.100) on the same items, against 75.9% with the joint encoding.

Images in Ollama

The same vision file works in Ollama's decision endpoint (tested with Ollama 0.40.4 on Linux). Pull it by its tag: Hugging Face attaches the matching vision projector automatically.

ollama pull hf.co/labqoat/qev-0.8b-GGUF:Q8_0
curl localhost:11434/v1/systemone -d '{
  "model": "hf.co/labqoat/qev-0.8b-GGUF:Q8_0",
  "state": "A photo.",
  "images": ["<raw base64 of a PNG or JPEG>"],
  "questions": {"q": {"type": "choice", "instructions": "What is in the image?",
                      "criteria": {"gray": "A gray square", "other": "Anything else"}}}}'

• :Q8_0 (and :vision-Q8_0) take images. :text-Q8_0 and :bf16 are the text builds. Hugging Face attaches the projector to every tag, so Ollama lists them with a "vision" capability, but their decision type does not take images: an image request gets HTTP 501 "This server does not support image input for decisions" (its --mmproj hint does not apply, the projector is already loaded). Text-only requests give the same answers on all three tags when they have one question.
• In Ollama, images takes raw base64, without a data:image/...;base64, prefix; a data URL gets HTTP 400 "illegal base64 data". To send data URLs, give state as chat messages with image_url parts.
• With several questions, the vision build answers each one in its own prompt (see above).
• Without Hugging Face: download qev-0.8b-Q8_0.gguf and mmproj-qev-0.8b-Q8_0.gguf, write a Modelfile with two lines, FROM ./qev-0.8b-Q8_0.gguf then FROM ./mmproj-qev-0.8b-Q8_0.gguf, and run ollama create.

Checked on Ollama 0.40.4: the gray-square request above answers gray 0.94; red, gray and blue squares are classified at 0.95-0.98; two images named <image:a> and <image:b> in the state are told apart.

Image results

JevVision v0, our internal image decision benchmark (1,306 items, 17 skills in 6 families: charts, scenes, boards, documents, app screens and public evaluation splits), described in labqoat/qev-0.8b. Zero-shot: qev was not trained on images. Pooled = mean accuracy over the 17 skills; ECE is the mean per-skill top-1 calibration error. "Same answer" is the share of the 2,371 questions where the build's most likely answer equals the transformers checkpoint's. Both columns below were measured on the same items in our runs; the transformers numbers are the main repo's re-run from its files.

Batched rows are not bit-identical to rows decoded one at a time, but they agree with them on 99.3% of questions (mean probability difference 0.005). decide on one request gives exactly the answers of the previous qev_gguf.py.

The GGUF builds score the same as the transformers checkpoint within noise (a skill has 70–100 items, so per-skill differences under about 5 points are noise). Their probabilities differ from it by 0.009 on average (largest difference per question; median 0.004), and calibration is slightly worse on this run (ECE 0.117–0.125 vs 0.113).

Speed on one NVIDIA L40S (CUDA build of llama-cpp-python 0.3.36, Q8_0 + mmproj f16). Batch-1 numbers are the median of 15 runs after warm-up, one image per request. "Cached" means the same image was sent again.

Throughput over the whole benchmark (1,306 requests, every image new):

decide_batch on the first 384 requests:

llama-server (vision file, mmproj f16, one client) takes 25 ms for a one-question request with a 448×448 image. With two questions, which become two prompts, it takes 36 / 48 / 89 ms at 224 / 448 / 768 pixels.

Each new image goes through the vision tower on its own, since libmtmd does not batch this projector, so that step bounds the batched speed. Image speed on a Mac was not measured.

Questions and answers

A request has a state (a string, an object or an array) and one or more questions, each of one type:

The fields follow decider's format_answer, the /v1/systemone wire format:

• confidence is TypeSafe's confidence. For a choice, it is the top probability rescaled so that uniform = 0. For a score, it is 1 minus the expected distance from the top level, relative to a uniform distribution.
• x_p_max is the top probability.
• certainty is 1 minus the normalised entropy.

A string state is used as it is. An object or array is rendered as JSON with array indices annotated, as decider's server does. A question can have up to 255 options.

How a request is answered

1. The request becomes a prompt row in decider's state-first System One layout: Context: and the state, then each question with its lettered options, ending in an Answer: ( slot.
2. One forward pass reads the logits of the option letter tokens at each answer slot. Only the letters' rows of the tied embedding are used.
3. Each question's answer is the softmax over its own options, sharpened per question type with temperatures fitted on a held-out calibration split: choice 1.7, noul 0.9, score 1.25 (p -> p^(1/t), renormalised). The temperatures and the letter tokens are in qev_config.json.

Encodings

• joint (default): one row per request, with every question behind one copy of the state. This is the cheapest encoding.
• per_question (per_question=True): one row per question, each with its own copy of the state. Each answer then does not depend on which other questions were asked, or in what order. This raises JevBench coherence from 64.5 to 71.2 at about the same accuracy, and on JevVision v0 it raises image accuracy from 0.757 to 0.783 (ECE 0.115 to 0.105) on the fast engine. Its cost depends on the runner. On qev.py's fast engine, a text request computes the shared state once and then runs each question's part. An image request runs one full row per question. The transformers backend runs one full row per question. The MLX runner in qev-0.8b-MLX computes the shared state once, so there it costs about the same as joint.

The two encodings give the same answers for single-question requests. For multi-question requests where consistency across questions matters, use per_question.

In llama-server, a request is answered in the joint encoding; send one question per request for per_question. Its answers carry choice, probabilities and confidence (score adds score and legend), but not x_p_max or certainty; qev_gguf.py returns the full format.

Evaluation

All numbers come from one harness, with the same items and scoring for every model. The temperatures were fitted on our own calibration split. No benchmark items were used for training or calibration. Benchmark scores were used to choose between checkpoints and weight soups, and part of the training data is written in the same formats as some benchmarks.

• Benchmark Heaven (BH): the public split, 231 items. The sampling error of BH accuracy is about ±2.8 points.
• JevBench: 240 cases, 1,938 gold questions. Coherence is scored by the official jevbench harness, ×100.
• Jev suite: 22 public-dataset tasks, 880 questions.
• Pooled: the mean of BH, JevBench and suite accuracy.
• JevExtra: our own skills benchmark: chess, browser, tools, agent action, prompt injection, safety, judging and grounding. The headline is the mean skill accuracy.
  • qev's indirect prompt-injection items share text with training rows (16-gram audit). The table includes them, like every other item.
• JevSpatial: our games and spatial benchmark, 2,040 items: game moves with the facts in the question and in the state, rules in the question, coordinates and small grids. Breakout, Whac-A-Mole, Knife Hit, 2048, Tetris and Pac-Man are never used in training.
• JevosTasks: seven policy and rule tasks built like the ones other System One models report (admission, rental, agent rules, authority, fraud points; ShARC dev; patent phrase similarity test), 3,440 items. The five generated domains are never used in training.

Where two values are given, they are joint / per_question. The two are equal on BH and the suite, which have one question per request.

Contamination audit. None of the training data shares any 16-token sequence with a BH or JevBench item. In the Jev suite, 2 of 22,773 test-pool items reach 50% 16-gram coverage. The training data was also screened by meaning against every test item, using embeddings with a threshold set from near copies.

Where it is weakest:

• JevBench score questions (4-level scales): accuracy 0.39. Most errors are off by one level, and predictions lean toward the middle levels.
• Long and hard BH items: accuracy 0.51 on long items and 0.57 on the hard tier.
• The suite's wide-label and graded tasks: go_emotions, helpsteer2, stsb.
• Abstaining in agent decisions, when no tool fits or information is missing.

Runtime fidelity.

• On the 902-request dev set used for release checks (1,782 questions), qev.py on an NVIDIA L40S agrees with our CUDA reference path on 99.7% of argmax answers, with mean absolute probability difference 0.003.
• On Apple MPS (bf16) and on CPU (fp32), over the first 50 requests (103 questions), the agreement is 99.0%: one near-tie question flips. The mean difference is 0.003.
• The MLX and GGUF builds agree with the reference on 99.0–99.8% of argmax answers (see their repos).
• These agreement figures were measured on the first release's weights; the architecture, code and quantisation are unchanged.

This repo.

• llama-server's prompts are token-for-token the ones qev_gguf.py builds, on 15 questions of every type with unicode, nested and long arrays (≥ 8 items get _index, as in training), numbers, object-valued instructions and options, described noul options and several-question requests. Its probabilities differ from qev_gguf.py's by at most 1.3e-4 (Q8_0) and 8.6e-4 (bf16), with the same answer every time.
• Stock llama-server tokenizes the whole prompt at once, while qev was trained on a prompt tokenized piece by piece (the state, each question, and, above 10 options, each option label on its own). Two kinds of prompt therefore differ by a few tokens: questions with more than 10 options (llama.cpp joins ( with the option letter, (A), and states that end in a newline (merged with the newlines that follow). On 579 prompts from JevBench, the Jev suite and extra non-English and many-option tests (llama.cpp b11786), every other prompt was token-for-token identical: 527 of 527 with up to 10 options and no trailing newline, including Hindi, Tamil, Bengali, Kannada, Telugu and Gujarati text. All 48 prompts with more than 10 options and the 8 states ending in a newline differed. Over the 534 questions, llama-server's top answer differed from qev_gguf.py's on 8 (Q8_0 and bf16), and the largest probability difference was 0.46. For questions with many options, or whenever the exact training prompt matters, use qev_gguf.py: it tokenizes as qev was trained. llama.cpp also does not apply the training tokenizer's NFC normalisation, so send text in NFC form (most text already is).
• qev_gguf.py agrees with the CUDA reference on 99.8% (bf16) and 99.5% (Q8_0) of argmax answers.

Speed

llama-server

A one-question request takes about 17 ms on an M4 Pro (Metal, Q8_0).

qev_gguf.py (M4 Pro, 24 GB, Metal)

Batch-1 numbers over 200 dev requests (tokens mean 746, median 417).

The CUDA engine in the main repo, for comparison

Measured on our dev requests: mean 746 tokens, median 417, about 2 typed questions each. These are longer than the benchmark items behind the 3.5 ms p50 in the 27-model comparison above, hence the higher figure here. CUDA: one NVIDIA L40S with our optimised engine (packed variable-length batches, the flash-linear-attention Gated DeltaNet kernels, CUDA graphs for small batches). Apple: an M4 Pro (24 GB) with MLX, bf16, compiled.

per_question (shared prefix): 7.9 ms p50 on the L40S. MLX prefix-cache hit: 18.5 ms.

The CUDA rows are what qev.py gives by default on an NVIDIA GPU: its fast backend (qev_fast/), after pip install -r requirements.txt -r requirements-cuda.txt. In a fresh environment installed from those two files alone (torch 2.14, flash-linear-attention 0.5.2, transformers 5.20; no vLLM), one L40S measured:

• 5.6 ms p50 and 19.5 ms p90 per request, 9.7 ms mean;
• about 105 requests/s one request at a time, and about 97 requests/s in packed batches (117 with vLLM's FlashAttention-2 build installed);
• 99.8% argmax agreement with our evaluation path on 1,782 dev questions.

Start-up: about 25 s until ready when the kernel cache is warm, about 2 min on the very first start (kernel compilation). The common CUDA graphs are captured at start; the rest are captured in the background after the model is ready.

The model is 0.8B parameters: 1.7 GB in bf16, 0.8 GB in 8-bit. For Apple silicon, use qev-0.8b-MLX.

Training

qev-0.8b is a full fine-tune of Qwen3.5-0.8B-Base (all text layers and the final norm; the tied embeddings are the base's). It was trained on our own decision data, in stages:

1. Base fine-tune: two epochs, about 1.8B tokens, over a decision mix of 1.28M requests (learning rate 5e-6; DiLoCo over 16 workers).
2. Anneal on a newer data release (about 142M tokens).
3. Coherence anneal: about 52M tokens, with a consistency loss on twin sets built from our own probability laws: invariance to rephrasing and reordering, complements, partitions, bounds, monotonicity and conditioning. No benchmark code or items were used.
4. A further round on newer data with coherence twins (about 104M tokens).
5. Weight soups of the trunk tensors to keep calibration while adding consistency.
6. A gap pass (150M tokens) over data for weak skills, BH-style long hard decisions, hard yes / no and grounding negatives, order-invariance twins, and replay.
7. A 50/50 weight soup of the pre-gap and post-gap models (the first release).
8. A games, rules and spatial pass (109M tokens with replay; 27M new): code-generated game-move decisions with the facts in the question or in the state, rules written in the question, and coordinate / grid relations. The labels are computed by code; the benchmark-only games and domains above are excluded, and every row was screened against the benchmark items.
9. The release: a 50/50 weight soup of the model from step 8 and the first release. The per-type temperatures were then refit on the calibration split.

Training data and licenses

The decision data comes from public datasets and data generated for the project. Each training row records its source, its license and how its label was made. Labels are of these kinds:

By license, for the decision dataset release the training mix was drawn from (2.2M requests):

• Permissive licenses, about two thirds of the rows: CC-BY-4.0 (about 35% in total, including NVIDIA's Nemotron agentic data, reasoning-core's procedural pile, FLD and ProofWriter), CC0-1.0 / public domain (about 17%, including the CFPB complaint database and NVD), Apache-2.0 (about 8%), MIT (about 5%, including QA sets under mixed MIT / CC-BY / Apache terms) and BSD-3-Clause (2.7%). A further 1.8% is under per-source upstream terms or ODC-BY.
• Share-alike sources, about 10.5%:
  • CC-BY-SA-4.0 from MTOP (1.6%), the Schema-Guided Dialogue dataset (1.1%), SQuAD-derived QNLI (1.1%) and HotpotQA (0.2%), plus other CC-BY-SA-4.0 sources (4.3%);
  • DBpedia (CC-BY-SA-3.0, 1.1%);
  • Bitext customer-support data (CDLA-Sharing-1.0, 1.0%).
• Generated for the project, about 17%:
  • computed by programs (6.3%);
  • written by an LLM and checked by two blind solver models (8.7%);
  • other generated rows released under CC0 (2.0%).
• Model-made labels: a further 1.2% carries ratings made by a language model, under MIT.
• Research-use releases: about 0.7% comes from the LeWiDi research release, whose component datasets carry their own terms, including research-use-only terms for one of them.

Later training stages added program-generated and writer/verifier-generated rows of the same kinds. The model is released under Apache-2.0. Whether share-alike or research-use terms on a part of the training data affect a given use of the weights is for the user to assess.

Limitations

• Ordinal scores are the weakest type. Treat score outputs as soft signals.
• Long and hard inputs. Accuracy drops on long states (several thousand tokens) and on deliberately hard items. The model reads the request in one pass and does no reasoning.
• Coherence is improved, not guaranteed. Complements and marginalisation across related questions, and unit restatements such as "90 min" vs "1.5 h", are still often violated. Use per_question for multi-question requests, and do not assume that answers to related questions are mutually consistent.
• Abstention. The model rarely chooses "cannot answer" or "no tool call" options, even when it should. Give it an explicit option and check its probability rather than relying on the argmax.
• Calibration is fitted, not universal. On data far from the calibration split, probabilities can be over- or under-confident. Recalibrate on your own labelled data when probabilities drive decisions.
• Language and domain. The data is mostly English business, operations and agent-log decisions. Other languages and domains have not been evaluated.
• Options. A question can have at most 255 options. Option names matter, because the model reads them.
• Images are zero-shot. Image input uses the base model's vision tower and merger, unchanged; qev was never trained on images. It does well on charts, simple scenes, forms and UI screenshots, and poorly on board games (connect-four and chess near chance) and on counting real objects; see labqoat/qev-0.8b. Image requests need a vision projector (mmproj), with qev_gguf.py or, in llama-server, with the default qev-0.8b-Q8_0.gguf, which answers each question separately (see Images in llama-server). Image speed was measured on CUDA only.
• Inherited behaviour. The model inherits biases of the base model and of its data, part of which was labelled by larger models. Do not use it as the only decision maker for consequential decisions about people (medical, legal, credit, employment).

Files

Provenance

The base is Qwen/Qwen3.5-0.8B-Base at revision dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68. qev-0.8b is the release called SG50 in our internal reports.

The GGUFs were converted from qev-0.8b with llama.cpp's convert_hf_to_gguf.py (commit 7fe450e1, --no-mtp) and llama-quantize, then given llama-server's decision metadata with add_systemone_metadata.py (gguf 0.19.0). The weights are unchanged by the metadata.

The image build (qev-0.8b-Q8_0.gguf, qev-0.8b-vision-Q8_0.gguf) is the text build qev-0.8b-text-Q8_0.gguf with two metadata values changed (gguf 0.19.0): qwen35.decision.type (a stock llama.cpp decision type that accepts images and answers each question in its own pass) and the systemone template from systemone_vision.jinja. Its tensors are identical.

All GGUFs with a vocabulary (qev-0.8b-bf16.gguf, qev-0.8b-Q8_0.gguf, qev-0.8b-text-Q8_0.gguf, qev-0.8b-vision-Q8_0.gguf) have tokenizer.ggml.pre = "qwen2", the pre-tokenizer qev was trained with (the conversion had written qwen35, which also joins combining marks into letter runs). Only that metadata value changed; the tensors are identical.

The vision projectors were converted from the same checkpoint with convert_hf_to_gguf.py --mmproj at the same commit (--outtype f16 and q8_0). They hold the base model's vision tower and merger, which qev's training did not change.

License

Apache-2.0, as is the base model, Qwen3.5-0.8B-Base. See "Training data and licenses" for the data.