Decision 4B v1.5 (FlyMyJev-4B, preview)

A new training run (qwen35_4b_letter_h14_v67b) from the same base, on a larger mixture than v1.4, with the per-request-type readout of v1.3: yes/no (noul) requests use their own, lower temperature so that an answer commits when the model leans one way.

An open model for typed decisions: give it a state (a ticket, a policy and a case, a log, an answer to judge) and a question with a closed set of answers (noul yes/no, choice, score), and it returns a probability for every declared option in one forward pass. Nothing is generated. Independent project: not a release of TypeSafe, not affiliated with it, not a reconstruction of Jev's closed implementation.

Run qwen35_4b_letter_h14_v67b. Every number below is our own measurement; we make no claim about a JevBench rank.

What it is

Requirements

• A C compiler at run time (for example build-essential): the flash-linear-attention Triton kernels are compiled on the first request, and CUDA-graph capture at load needs them too.
• model.load() verifies every file against manifest.json and refuses symlinks, so a Hugging Face cache snapshot (whose files are links into blobs/) must be copied to a real directory first, for example with huggingface-cli download --local-dir.

Use

import model                       # this repository's model.py
m = model.load()                   # verifies every file against manifest.json, downloads the pinned base on first use
m.decide("Refunds need a receipt and a purchase within 30 days. The customer bought 12 days ago and has no receipt.",
         {"type": "noul", "instructions": "Is a refund permitted under the policy?"})
# {'probabilities': {'true': ..., 'false': ...}, 'input_tokens': ..., 'seconds': ...}

JevBench: server.py serves TypeSafe's /v1/systemone wire format for the typesafe adapter; jevbench_adapter.py is an in-process adapter; run_jevbench.py runs the official runner without editing its registry. Set FLYMYJEV_DEVICE=cpu to run on CPU (slow, no CUDA graphs).

Results (our measurements)

The frozen base and this adapter were measured on the same GPU in the same session with the same prompt; changes are paired (fixed / broken items). JevBench's judge tier and sealed set are not public and are not included.

• Hard dev set by family (frozen -> this model): date and number decisions 10 -> 60, paraphrase robustness 55 -> 95, trade-offs 25 -> 65, abstention 60 -> 95, multi-hop lookups 30 -> 65, adversarial traps 50 -> 80, long policies 55 -> 55, answer judging 55 -> 50 (20 items per family).
• Calibration on the public hard tier at the served temperature: ECE 0.112, fidelity to exact gold distributions 89.2 (1 - mean total variation distance).
• Real-case set v3 puts the gold at option A in 46 % of its choice items; the frozen base's first-option habit scores there, and training removes it. Read v3 with that in mind.

Package check (29 Sep 2026)

This package, as published, was run on one RTX 4090 against the JevBench v1.4.2 harness on the 231 public items: the official runner with jevbench_adapter.py and server.py with the harness's typesafe adapter both read 204/231 (easy 48/48, standard 72/72, hard 84/111). Details: package_check.json.

Training

One run from the base, no merging of runs and no ensembles: 2 epochs, 986 steps of at most 12,288 padded tokens (11.1M tokens), lr 3e-05 with warm-up and cosine decay, cross-entropy on the option-letter logits (the exact distribution where an item declares one), choice options shuffled on every pass, seed 99. The recipe shape follows JevK5 v0.2 (allebee/jevk5, Apache-2.0).

Data: our own decision data and licence-reviewed public datasets. No JevBench item and no output of Jev was used for training, tuning or model selection.

Benchmark hygiene

• The temperature was fitted on our own calibration split and hard dev set, never on benchmark items.
• The gates that made this run a release candidate were declared before the run.
• The JevBench public items were used to measure, and once to diagnose: comparing our frozen outcomes with SemIf's published per-item outcomes showed that our first yes/no rendering lost about 5 points on yes/no items; we adopted SemIf's rendering everywhere. No training data was chosen or built from public items.

Limitations

• One pass of a 4B model: no step-by-step reasoning. Chained arithmetic and long multi-hop documents stay weak.
• Messages where two options are both true under a one-answer schema are untested.
• English only; the calibration was fitted on our own data.

Licence and credits

Adapter, code and configuration: Apache-2.0 (LICENSE). Base: Qwen3.5-4B by the Qwen team (Apache-2.0). Prompt format: SemIf by TheoLeeCJ (MIT). CUDA-graph runtime design and recipe shape: JevK5 by allebee (Apache-2.0). Evaluated with JevBench by fstandhartinger (MIT). Not affiliated with TypeSafe AI or Jev. See NOTICE.md and EVALUATION-PERMISSION.md.