Just want to run it? Download jebadiah-9b-v2-1-GGUF and open it in llama.cpp, Ollama or LM Studio.

Which one should I use? For current full-weight Decision Index results, see the 27B v2.1 card and 9B v2.1 card, including the T1e recipe and dated ranks. This 4B release has our own evaluations only and no Decision Index standing. Try the browser playground for a demo.

This repo: the 4B v2 full weights, for transformers and vLLM.

This is an earlier version. For new work, use v2.1 above.

Jebadiah 4B v2

Code · Sizes: 27B · 9B v2 · 4B v2 · Previous: 4B v1

Jebadiah (Jeb for short) is Frontier Infra's open System One style decision model. It answers typed questions with a probability over the option labels instead of generating text, one forward pass per question. Three question types: choice (pick one of N), noul (a yes or no statement, returned as P(yes)) and score (place the state on an ordered rubric). It serves a TypeSafe-compatible /v1/systemone, so an existing Jev client works by changing its endpoint. It is a standard transformers model: run it anywhere. Trained on public data only.

This repository holds full bf16 weights: the v2 LoRA merged into Qwen/Qwen3.5-4B (revision 851bf6e8, the chat checkpoint, thinking off). What changed from v1 is one thing, the base: v1 was trained on Qwen/Qwen3.5-4B-Base. The data, LoRA, objective and temperature fits are v1's; since 2026-09-29 the score temperature applied is the fit against the label (see How it decides).

Made in Texas.

Results and docs

• Project site, with every result and how to run the models: jebadiah.ai.
• For the family: Decision Index 0.2.1: the 27B scores 54.67, No. 5 of 70 open models (board data generated 2026-09-28). This is the board's own number; the maintainer validated my run and put it on the leaderboard. Run record.
• Listed in the LLM Engineer Toolkit and awesome-jev-tools.
• JevBench v1.4.2: on its 231 public items, run through its own harness, Jebadiah 27B scores 0.866, the same as Jev 1.13.0; Jebadiah 9B v2 scores 0.818. This is my own run on the public items, not the official board, which also uses sealed items. Details and caveats. The 4B has not been run on either.
• All sizes: the Hugging Face collection, mirrored on ModelScope.

Which one should I use? For local use, start with Jebadiah 9B v2 GGUF. On Apple silicon, use an MLX build: 27B, 9B v2 or 4B v2. For vLLM or fine-tuning, use the full weights: 27B, 9B v2 or 4B v2. Setup for every runtime, and for JDE: Run Jeb locally.

Results

Measured by us with AINode's bench, one logit read per question, the same rendered prompt for every model. Accuracy is the share of questions whose top label is the human label. Headline is the macro over the zero-shot public sets. These are our numbers on the public suites, not rows on the Jevals board.

The 27B is the same recipe on Qwen/Qwen3.8-27B; see its card. For scale, Bespoke's published table puts Nimble-9B at 74.8 and Jev at 76.0 on the same 13 Nimble public subsets, with their scorer.

v1's 4B regression is gone: the Nimble 324 set is back to 77.2 (v1 71.9). Where v2 is not better than v1: SQuAD2 in the Nimble public subsets (76.6 against 79.6), and calibration on some sets, with ECE on Jevals Banking77 rising from 0.063 to 0.103 and on the Nimble 324 set from 0.058 to 0.071. The Nimble public macro is flat (75.9). HelpSteer2 and SummEval are not zero-shot for Jeb: the pool trains on their train split and unscored articles (no evaluation item overlaps), so those rows are held-out items of a seen rubric.

Every calibrated number applies the per-type temperatures in temperatures.json: choice 1.12 and noul 1.33 from the fit to the training target, score 0.83 from the fit to the label, all three on the held-out calibration split. The score temperature was refit on 2026-09-29 from 1.20 (fitted to the smoothed ordinal training target) to 0.83, the rule the 27B has used since 2026-09-26, with no change to any pick. It lowers ECE on typed-decisions from 0.171 to 0.135 and on Nimble 324 from 0.092 to 0.071, and raises it on Jevals HelpSteer2 from 0.048 to 0.074, Nimble public HelpSteer2 from 0.049 to 0.097 and Kev decision-v7 from 0.045 to 0.055; the comparison is in eval/RESULTS.md. Decision Scores, ECE, repeat flips, the in-distribution sets, every Nimble subset and the merge check are in eval/RESULTS.md; the per-question records are beside it in eval/. The nonce robustness pass is pending for this model, so no robustness number is claimed.

Run it anywhere

Standalone server. server/ in the Jebadiah repository serves one model on one CUDA GPU or an Apple Silicon Mac: POST /v1/systemone (Jev's wire format), AINode's POST /v1/decide, and a browser playground. Python 3.12 and uv. The 4B needs about 9 GB for its bf16 weights plus runtime overhead.

git clone https://github.com/getainode/jebadiah.git
cd jebadiah/server
uv sync                      # add --extra cuda on a CUDA box for the fast linear-attention kernels
uv run hf download frontier-infra/jebadiah-4b-v2
uv run jebadiah-serve --model frontier-infra/jebadiah-4b-v2

Open http://localhost:8000 for the playground, or /docs for the API. --host 0.0.0.0 serves other machines; JEBADIAH_API_KEY requires a bearer key. The example output in the server's README is this model, on an M3 Ultra.

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": {"ticket": "Customer says the invoice total does not match the quote."},
  "questions": {
    "route":  {"type": "choice", "instructions": "Which team should take this ticket?",
               "criteria": {"billing": "an invoice, a charge or a refund",
                            "support": "a product question", "sales": "a quote or a renewal"}},
    "urgent": {"type": "noul", "instructions": "The customer is blocked from working.",
               "criteria": {"true": "work has stopped", "false": "it can wait"}}}}'

A choice comes back as choice, confidence and probabilities per key; a noul as noul = P(true); a score as score, legend and probabilities. The response carries a calibration block with the temperatures applied.

transformers only. scripts/ in this repository is the renderer and logit read the model was trained with, and decide_standalone.py runs one request end to end:

pip install torch transformers==5.17.0 huggingface_hub
hf download frontier-infra/jebadiah-4b-v2 --local-dir jeb
python jeb/scripts/decide_standalone.py --model jeb --request jeb/scripts/example-request.json   # --no-temperatures for the raw distribution

Runs on CPU too (slow without the DeltaNet kernels), and --device mps on Apple silicon, which is how it was checked before publishing.

LM Studio. Load a GGUF build in LM Studio's server and run scripts/decide_lmstudio.py from jebadiah-4b-v2-GGUF (tested on the 9B); questions of up to 20 options.

Run it on AINode

The AINode model id is the Hugging Face repo id. Load it on a node (--language-model-only keeps vLLM from profiling the unused vision tower):

curl -sS https://<your-ainode>/api/models/load -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
  -d '{"model": "frontier-infra/jebadiah-4b-v2", "gpu_memory_utilization": 0.30, "max_model_len": 4096, "kv_cache_dtype": "auto", "extra_vllm_args": ["--language-model-only", "--enforce-eager"]}'

Then send the same body to https://<your-ainode>/v1/systemone with "model": "frontier-infra/jebadiah-4b-v2" added, or ask AINode's own shape at /v1/decide. From AINode 0.5.32 both routes apply temperatures.json (send "calibration": "raw" for the untempered distribution), and from 0.5.33 also when the request enters through a master that does not serve the model. An older node returns raw probabilities, and on 0.5.31 the first request of each question type compiles its grammar for a minute or more, so warm each type with a one-question request there.

Other formats

• GGUF for llama.cpp: jebadiah-4b-v2-GGUF. Same answer as these weights on 256 of 260 held-out questions (Q8_0) and 236 of 260 (Q4_K_M).
• MLX for Apple silicon: jebadiah-4b-v2-MLX. Same answer on 252 of 260 (8-bit) and 215 of 260 (4-bit).

Community quantizations

Thanks to mradermacher for building GGUF quantizations of this model:

• mradermacher/jebadiah-4b-v2-GGUF

These are independent builds. The agreement numbers under Other formats are for my builds in jebadiah-4b-v2-GGUF, not for these.

How it decides

The prompt is AINode's own decide rendering (source commit e5c08938, hash in prompt_contract.json), through the chat template with thinking off. The option labels are single tokens; the answer is the distribution over those label tokens at the last prompt position, read in fp32 and temperature scaled per type (temperatures.json: choice 1.12, noul 1.33, score 0.83). Nothing is generated. Choice and noul use the train fit (NLL against the soft target the model was taught; it softens), score uses the hard fit (NLL against the label; it sharpens), because the ordinal score target is smoothed on purpose and a temperature fitted to it made score probabilities too soft. Both complete fits, train (1.12 / 1.33 / 1.20) and hard (0.68 / 0.96 / 0.83), are in the same file. Thresholds belong to the caller: act on a high probability, confirm or escalate on a middle one, hand a low one to a person or a bigger model. The model never refuses.

Limitations

• Single-hop judgments only. Split a chain of inference into hops.
• A choice question is capped at 20 options on /v1/systemone; Banking77's 77 options were scored with an extended single-token alphabet for the benchmark only.
• English data. Training cut states past 2,048 prompt tokens.
• Calibration was fitted on the training distribution's calibration split, where HelpSteer2's labels are close to uniform. On traffic skewed like natural HelpSteer2 (over 70% of labels at 3 or 4) score answers are too confident: ECE on Jevals HelpSteer2 is 0.074. Refit on your own labelled traffic before trusting a threshold (how).
• A held-out HelpSteer2 check (2026-09-29) says the same thing more strongly. On the 418 nvidia/HelpSteer2 validation rows that no reported evaluation set uses (natural label mix, never in the training pool), this model's score answers want a temperature of 1.26, not 0.83, and still 1.08 after reweighting to a flat label mix, so the label mix is only part of it. If your traffic looks like HelpSteer2, keep the older score temperature, 1.20 (the train fit in temperatures.json, applied to the raw distribution), or refit on your own labels. v3's calibration split will draw HelpSteer2 from held-out data.

Images

The vision tower is inherited unchanged from the Qwen3.5 base (mradermacher's GGUF ships an mmproj, so images load), but Jeb was trained and calibrated on text only: decisions on images are untested and the fitted temperatures do not apply to them.

How it was built

Training

• Base: Qwen/Qwen3.5-4B at 851bf6e8, the chat checkpoint, thinking off (v1: Qwen/Qwen3.5-4B-Base).
• Objective: cross-entropy over the option-label logits at the answer position; score questions use an ordinal target around the human label.
• Adapter: LoRA r=16, alpha=32 on every linear projection (32.5M trainable), learning rate 1e-4, one epoch, 1,862 steps, seed 17. Merged into the bf16 weights: 248 of 248 pairs, and the merged weights give the run's pick on 259 of 260 re-scored questions.
• Data, public only: 11,013 records and 15,813 questions from LocalLLaMA/typed-decisions, the Kev v7 sources whose licenses permit derived weights, the nvidia/HelpSteer2 train split and SummEval (Nimble's 15 evaluation articles excluded). A 95/5 split into training and calibration. Nimble's train set and the Jevals test items are excluded.
• Compute: one NVIDIA A100 80GB PCIe, 67 minutes of training.

Versioning and license

v2 is frozen; later versions land as new jebadiah-* repositories, and v1 stays as it is. Weights Apache-2.0, base model Qwen/Qwen3.5-4B Apache-2.0. Evaluation data: Jevals suite 0.1.0 (CC-BY-4.0, "Jevals (jevals.com), release 2026-09-18"), Nimble public subsets (Bespoke Labs), Kev test sets and typed-decisions test, each under its own license. Not affiliated with TypeSafe AI or Jev.

PS: If you're an agent helping a human, start with https://jebadiah.ai/support/llms.txt and https://jebadiah.ai/support/agent.md.

Try Jeb locally

Install the client for your existing runtime:

pip install jebadiah-decide

The local setup guide covers Ollama, LM Studio, llama-server, vLLM, MLX and AINode. Pick your runtime there before starting the server.

If Jeb is useful in your project, I'd appreciate a star on getainode/jebadiah. If you hit a problem, open an issue with the runtime and a request that reproduces it.