rsi-jev-v7.0-vl-4b

A typed-decision model built by a self-improving loop of AI agents. It is fine-tuned from Qwen/Qwen3.5-4B, reads the options in a new format (the option list repeated once, and the prompt and the options attending to each other in both directions), and has decision heads at layers 16, 20 and 32: each question answers at the first depth that is confident enough. It answers choice / noul / score questions about a document and up to four images, and speaks the Jev HTTP API.

v7.0-VL is a System One model: one forward pass, no generated reasoning, a probability for every option. effort sets how many layers that one pass uses (16, 20 or 32), not how many tokens it writes; it writes none.

pip install "rsi-jev[fast,vision] @ git+https://github.com/Shanghua-Gao/RSI-Jev"
rsi-jev serve v7.0-vl-4b

Serve it with this repository's code: the package turns on the option echo and the prefix-bidirectional attention it was trained with, and the server fixes the kernel configurations so the same request always gets the same answer.

Everything below is the version record from the project repository.

---

RSI-Jev v7.0-VL 4B

Describes release v7.0-VL · updated 2026-11-05

A new way to read the options. v7.0-VL repeats the option blocks once and reads the second copy (the option echo), and lets the prompt and the options attend to each other in both directions on the full-attention layers (prefix-bidirectional attention); only the answer cue stays causal. It starts from the instruction-tuned Qwen/Qwen3.5-4B instead of the base model, and it was trained once, end to end, on a 2.76M-example corpus with long reasoning traces kept as a training-time signal. The effort levels, the exits at layers 16, 20 and 32, images and the API are v6.1-VL's.

Against v6.1-VL on the same evaluation it is more accurate on the suite (+0.005), the held-out set (+0.083), MMLU-Pro (+0.040) and images (+0.005), and better calibrated (final ECE 0.039 against 0.048). On the public Decision Index 0.3 it scores 60.63 (v6.1-VL: 50.98).

v7.0-VL is a System One model: one forward pass, no generated reasoning, a probability for every option. effort sets how many layers that one pass uses (16, 20 or 32), not how many tokens it writes; it writes none.

It is a 4B model, named as Qwen names the 4.66B Qwen3.5-4B: 4.69B parameters — the 32 decoder layers (3.57B), the token embeddings (0.64B), the vision tower (0.33B), the main decision head (0.05B) and the two early-exit heads (0.10B). The checkpoint is self-contained.

Checkpoint: shgao/rsi-jev-v7.0-vl-4b · this record: versions/v7.0-vl.md

1. The task

Unchanged from v6.1-VL: a state (text, optionally up to four images), one or more typed questions, a probability for every option in one forward pass, and usage.depth, the layer each question was answered at.

2. How the model is built and trained

Full fine-tuning (no LoRA) from Qwen/Qwen3.5-4B at revision 851bf6e, own-token option readout. Code: train/v7.0/; specs and commands: data/v7.0-vl_recipe/.

1. Retention traces. A seed-0 sample of 398,101 of the corpus's own training questions is answered by Qwen/Qwen3.5-4B with greedy chain-of-thought (at most 1,536 new tokens); the longest clean traces that end in a boxed answer form the retention pool (21,087 rows used).
2. One training run, one epoch: 36,149 steps of 64 questions (2.76M examples from 977 sources; 115,957 image questions seen) on 4x H200, 147,845 s. Each step trains the tower and the main head at layer 32; the early-exit heads at layers 12, 16 and 20 train on detached states, so they do not change the tower. The losses beside the soft cross-entropy:
   • a retention KL (0.5, top-64 over the last 1,024 tokens, 4 trace rows per step) that keeps the tower's language-model distribution on the retention traces close to the base model's — reasoning is a training signal only, never generated at inference;
   • a confidence-ranking loss (1.0): a pairwise ranking term over correct and wrong answers in a micro-batch, on the main exit's top-1 confidence;
   • an option-order consistency KL (2.0) between two orderings of the same text question. The lower 8 layers learn at a tenth of the tower's learning rate (5e-6); the heads at 1e-4.
3. Calibration and thresholds, fitted on the development sets only, with the rules of v6.1-VL: one temperature per exit (log T 0.28, 0.22, 0.26 at 16, 20, 32) on half A; auto's per-exit thresholds 0.80 at 16 and 0.65 at 20; the default single threshold by v6.1-VL's rule, whose winner (0.85) was not confirmed on half B, so it ships the fallback 0.95, as v6.1-VL did.
4. The package: the trained tower with the base embeddings, the base vision tower (never trained), the heads at 16, 20 and 32 (the layer-12 head is dropped) and the calibration.

New in this release: the option echo and the prefix-bidirectional attention (both in training and in serving), the instruction-tuned start, the retention traces, the order-consistency loss on every text row, and one end-to-end run instead of stages.

3. Checkpoints

rsi-jev serve v7.0-vl-4b          # or: Decider("v7.0-vl-4b")

¹ Without open_jev_ood; the earlier rows include it. ² The suite's calibration is not compared across releases; final-set ECE is in §4.1. v7.0-VL's figures are layer 32 in fp32 (the default reads 0.797 and 0.810); v6.1-VL's are the bf16 package as served by default; the 27B's are layer 64 from its record.

The tower ships in bf16 (9.7 GB in all, as v6.1-VL), the precision the server runs it in; it was trained in fp32 and cast once, so served outputs are the same as from the fp32 weights. The decision heads stay fp32.

The package switches the option echo and prefix-bidirectional attention on itself (meta.json serve_env); nothing needs to be set. Older releases are served exactly as before.

4. Results

4.1 Against v6.1-VL

Both models read with the same evaluation sets and readers, text in fp32, ECE at each model's served temperatures; v6.1-VL's columns are from its record. The v7.0-VL reads run the prefix-bidirectional forward and were checked against the model's own forward pass (identical main-exit logits). low, medium and high stop at layers 16, 20 and 32; auto uses the per-exit thresholds above. Images always run all 32 layers. Suite is the fifteen-benchmark mean without open_jev_ood.

At layer 32 the gains are +0.005 on the suite, +0.083 on the held-out set and +0.040 on MMLU-Pro. The held-out set includes MuSR and BBH, and this release's corpus added training data for reasoning of those kinds (narrative multi-step, causal and code-execution questions). No held-out item appears in the corpus (exact match on the normalised text), and no 13-word run of a long held-out text does. On the newer held-out set eval_final_v3 (BFCL, ConditionalQA, LegalBench; the set the training run checked for overlap), v7.0-VL scores 0.817 at layer 32 and 0.786 with auto.

auto matches high on the suite and MMLU-Pro at 22.1 layers on average: 46% of questions stop at layer 16, 22% at 20 and 33% go to 32. It costs 0.013 on the held-out set. The default (single questions at layer 32, the 0.95 cascade on multi-question requests) scores as high; run on every question it takes 27.5 layers on the suite. KoBBQ is lower than v6.1-VL's (0.903 against 0.937), and the model answers "unknown" less often when the question is ambiguous (0.868 against 0.953). CLINC150's option-after-the-right-one rate is 0.7%, inside the 5% bar.

Images by benchmark (layer 32): HallusionBench 0.763 (v6.1-VL 0.762), InfoVQA 0.893 (0.889), MMBench 0.891 (0.883), POPE 0.900 (0.888), RealWorldQA 0.746 (0.748).

4.2 External: Decision Index

Decision Index 0.3, public index: 60.63 (raw 70.23). Full run of the release package served by this repository's code at the default effort, 140,178 requests, coverage 1.0, all answered, none truncated: knowledge and reasoning 42.4, language 62.6, retrieval 67.9, tools 82.4, arts 48.5; 60.63 with the rows that also appear in our training data left out. v6.1-VL scores 50.98 on the same index. With ACOS, RAGTruth and PhishNChips held at v6.1-VL's values the index is 59.80.

Decision Index 0.2.1, full run: 59.63 (all 150,759 requests answered, nothing cut; knowledge 39.1, language 63.5, retrieval 67.8, tools 82.4, arts 44.4), the same with the overlapping rows left out. v6.1-VL: 50.74.

The largest gains over v6.1-VL are in knowledge and reasoning: MuSR, CLadder, CRUXEval and HellaSwag, the families the corpus added training data for. The largest remaining gaps to the strongest 4B models on the board are ACOS, select-all-that-apply (SATA-Bench), PhishNChips and the arts benchmarks (Habermas, BPoMP, cfcolor, Humicroedit).

4.3 Calibration

Each exit has its own scalar temperature. Final ECE after calibration is 0.039 at layer 32 and 0.047 with auto, against v6.1-VL's 0.048 and 0.055; layer 20 is the best calibrated (0.028), layer 16 the least (0.053). On the half of the development sets the fit did not see, the fitted temperatures lower ECE at layer 16 (0.038 to 0.018) and leave layers 20 and 32 slightly worse than raw (0.022 to 0.030, 0.021 to 0.025): the fit minimises log-loss, not ECE.

5. Speed and serving

Per effort level, on the Decision Index 0.2.1 kit's 16k-question stratified sample (the same rows for every level, one NVIDIA H200, bf16, overlapping rows removed; a sample read, not a full run), with v6.1-VL's figures from its record on the same setup:

v7.0-VL gains 7.0 to 8.4 points at every level. auto reads 0.5 below high at 72% of its mean latency; on this sample 34% of requests stop at layer 16, 23% at 20 and 29% at 32 (the rest are multi-question requests). The medians of auto and high are close because the requests that stop early are mostly the short ones. Over the full Decision Index run (one RTX PRO 6000 Blackwell, default effort) the median is 29.4 ms (p95 236 ms).

The same request gets the same answer. The linear-attention kernels used to pick their block sizes by a timing race when a server started, and the pick changed how sums round: two server processes could answer about 1% of questions differently with an early exit. The server now fixes those kernels to one configuration each, so nothing is autotuned. Fresh processes in different jobs gave bitwise identical outputs on the RTX PRO 6000 and on the H200, at auto and at high, and the same outputs as the evaluation reads in §4. RSIJEV_TUNE_PIN=0 turns it off.

Serving limits are v6.1-VL's: inputs up to 32,768 tokens, nothing cut (longer gets a 422); up to 5,120 options; own-token option pooling. What changed: with the prefix-bidirectional attention the prompt depends on every option, so nothing is cached causally — no prefix or document cache, no padded CUDA-graph path; every question is one full forward pass. Full attention runs through PyTorch's sdpa with the prefix-bidirectional mask.

The API returns a probability and a confidence, and they are different numbers: confidence is the peak statistic Jev documents, (K * p_max - 1) / (K - 1). Route on the probability of the option you intend to act on. The response also carries usage.depth, the layer each question was answered at.

6. Limitations

• KoBBQ accuracy and the "unknown" rate on ambiguous questions are lower than v6.1-VL's (§4.1).
• Our own held-out families show losses against v6.1-VL in medicine and customer support / travel, and on a Brazilian scam-screenshot set.
• The default single threshold, 0.95, is the fallback of the selection rule, not a confirmed choice.
• Deterministic answers are checked on the RTX PRO 6000 and the H200 only (§5); on other GPUs the fixed kernel configurations are used too, but nobody has compared two processes there.
• Serve it as packaged. The option echo and the prefix-bidirectional attention are part of the model; read without them (a causal forward), its outputs change.
• One seed. The training corpus, the retention traces and the development sets are not public.
• The early exits read text only; questions with images always run all 32 layers.

7. How it got here

305 arms ran on the line to this release: 84 on the data loop and 221 on the method line, one per distinct experiment id with a run record, a rerun once.

• The data loop. Each round benchmarked the current model, found its weakest families and added training data for them; this release's corpus is the twelfth round, which added knowledge and reasoning data at volume. No distillation from a larger model.
• The method line. The option echo (+3.5 on our 4B pick set), the prefix-bidirectional attention (+2.6) and the retention traces came out of the method experiments and were combined into one 4B recipe, +3.8 over v6.1-VL on the same set. Most method ideas did not survive the move from the 2B proxy to 4B.
• Instruct start. Every data round since the tenth starts from the instruction-tuned model with the current best method recipe.

The full list of arms on the released line is in EXPLORE.md.

Rerunning it

The code is in train/v7.0/ and the specs and commands in data/v7.0-vl_recipe/. The training corpus, the retention traces and the development sets are not public, so the training run cannot be repeated from this repository alone. Serving the released model needs only the public code.

8. Corrections to v6.1-VL's record

None. The v6.1-VL record is left as published.