▶ Live demo · qev-0.8b

squad-0.8b

A team of models. On your device. In milliseconds.

squad: the qev model and its team of experts. Not related to the SQuAD question-answering dataset.

A 0.8B typed-decision model (System One API: a state plus typed questions in, a probability per allowed answer out, one forward pass, no text generated) built as a MoQLE, a Mixture of Quantized LoRA Experts: one trunk, a 1.1 MB router after layer 5, and 2-bit LoRA experts it calls in when a question needs them.

This revision (October 2026) has a new trunk and four experts: games, e2e (decisions of a test agent driving a web app), multilingual (10 languages) and documents.

• Team: each expert is 2.7 MB; the four add about 11 MB to the trunk.
• Fast: 3.7 ms median per decision on one L40S (single requests), about 110 decisions/s with 16 clients. The router and experts cost nothing measurable.
• Better than the previous squad-0.8b on rule following (+10 points on Rules-7), consistency (+5.8 on JevBench coherence), game boards (+3.9 on JevSpatial) and web-app tests on apps it never saw (84% vs 21% of tests passed); slightly lower on Benchmark Heaven (−0.5) and games_bench (−0.9), both within run-to-run noise.
• New e2e expert (2026-10-27): trained in a JAX simulation of the tester-army executor (93 generated app families, exact prompts, exact gold at every state; on-policy imitation of an oracle), selected only by real closed-loop runs on apps it never saw. It passes 52 of the 8 real eval apps' 61 tests (shipped c13: 12) and 69 of 82 on held-out test apps (previous expert: 29).

Results

All numbers measured with the same benchmark client; each model served by its own runtime (this repo's qev_fast engine, per-question requests). basal-omni-pico (Remek/basal-omni-pico) was served by basal 2.4.0, fast mode.

Speed on one NVIDIA L40S (Benchmark Heaven requests, one question each, after warm-up):

Read these honestly:

• Benchmark Heaven moves by about ±1 point between runs (one item is 0.43 points); its hard tier (111 items) is around 57% for every model here.
• Rules-7: part of the gain comes from procedurally generated rule-application data whose layout (the policy inside the question) resembles Rules-7's. No benchmark text was used; labels are computed by code. Sub-scores vary across training seeds.
• e2e: runs of one test are near-copies (decisions are argmax), so count tests, not runs. Held-out apps: 14 validation apps (67 / 82; used to pick the checkpoint among ~120) and 14 test apps (69 / 82, 69 on a repeat). On the 8 eval-only real apps 52 of 61 tests pass in every one of 3 runs (expenses 2/2, tracker 8/9, shop 8/9, billing 8/9, helpdesk 8/9, booking 8/9, TodoMVC react 5/7, vue 5/7). The last training step focused on Enter-only list inputs after a failure analysis of the eval apps' TodoMVC failures, so the 8-app number is not fully independent of the selection. Repeat noise: about ±3 tests. Real-app skill peaks after few updates and decays with more training on generated apps, so checkpoints are chosen only by real runs. The e2e expert is used only when the router gives it at least 35%.
• games_bench is our own closed-loop benchmark of games never trained on; all models are close to random when a game is shown as a raw grid, coordinates or prose.
• Image input was not re-evaluated for this revision (the trunk changed; the vision tower is unchanged).
• Benchmark scores were used for reporting; checkpoints were chosen on our own validation data, except the serving threshold above.

How it works

• Trunk: a full fine-tune of Qwen3.5-0.8B-Base (24 layers, Gated DeltaNet + gated attention). It continues the previous qev-0.8b trunk on procedurally generated, exactly labelled data: consistency twins (the same decision asked in rephrased, reordered, merged or complementary forms, trained with a consistency loss), rule application with natural "up to / below" semantics, graded similarity, plus replay of the old data. Final weights are a weight average of three runs. No LLM-written text is in the trunk.
• Experts: rank-64 LoRA on gate_proj, up_proj, down_proj of layers 12–23, y = W x + 2.0 · B A x, stored as 2-bit rows (specialists/slots_2bit.safetensors). Slot 1 games, 2 e2e, 3 multilingual, 4 documents. Trained by reinforcement learning only (closed-loop games, simulated web apps, decision tasks with a proper-scoring reward), trunk frozen. The multilingual and documents experts learned from data written and cross-checked by two LLMs.
• Router: LayerNorm → Linear(1024, 256) → GELU → Linear(256, 9) on the mean RMS-normalised residual after layer 5 (specialists/router.safetensors), top-1 of {trunk, games, e2e, multilingual, documents}. Trained on measured outcomes (log-probability of the right answer) on validation rows no expert saw, so it only calls an expert where it helps. router.min_p_slots in the manifest holds per-expert confidence thresholds (e2e: 0.35).

Quickstart (NVIDIA GPU)

pip install -r requirements.txt -r requirements-cuda.txt

from qev import QEV

m = QEV.from_pretrained("labqoat/squad-0.8b", specialists=True)     # router + experts (fast CUDA engine)
m.decide(state, questions, per_question=True)                        # System One answers
m.last_routes                                                        # e.g. ["games"]

python qev_server.py --model labqoat/squad-0.8b --specialists     # POST http://127.0.0.1:8017/v1/systemone

Without specialists=True you get the trunk alone (not the old qev-0.8b: the trunk changed in this revision; pin the previous revision 2043fda for it). Experts need the fast CUDA engine (requirements-cuda.txt).

On a Mac or CPU

• MLX (labqoat/squad-0.8b-MLX): router + 4 experts, about 50 ms per decision on an M4 Pro; same top answer as this runtime on 100% of 387 test questions (bf16).
• GGUF (labqoat/squad-0.8b-GGUF): llama.cpp b11786 or newer (/v1/systemone), one LoRA file per expert, qev_gguf.py for routing; same route on 333/333 requests and the same top answer on 99.5% of questions (Q8_0, Metal).

Cache

All three runtimes keep a prefix cache across requests (on by default): a request that starts like an earlier one (same page, same goal and history) only computes the new tokens, and an exact repeat is answered from memory. Answers are the same with the cache on or off (same top answer and route; probabilities within bf16 noise). On a recorded tester-army session: CUDA model time 900 → 721 ms (exact repeats 0.08 ms), GGUF on an M4 Pro 25-30% less total time, MLX 22.0 → 12.9 s and memory capped (about 3 GB).

Files

model.safetensors (new trunk; embeddings, head and vision tower as before) and tokenizer / config files · specialists/ (manifest, router, 2-bit experts) · qev.py, qev_server.py, qev_fast/ (runner, server, CUDA engine with experts) · qev_image.py, qev_image_server.py (transformers image path, trunk only) · SHA256SUMS.

Limitations

Text decisions only were evaluated for this revision. The e2e expert still fails some real-app patterns: identical row buttons where the goal names the row by another column, submitting before a form is filled, and repeating an action that changed nothing; it also cannot work around harness gaps (controls the executor doesn't expose, loading screens). Polish sentiment and QA (KLEJ) are behind basal-omni-pico. Probabilities are calibrated with the previous version's temperatures; check them on your own data before using them as error rates.

License: Apache-2.0.