OpenJev-4B

Learning Open-Vocabulary Probabilistic Decision Models from Pretrained Language Models

1. Introduction

OpenJev-4B is a decision model built from the text backbone of Qwen3.5-4B. Given a state, a question, and a request-defined set of natural-language options, it directly predicts a probability distribution over those options. It supports selection, binary judgments, and rubric-based scoring through the same probabilistic interface.

The model combines a language-model backbone with a lightweight Option Set Interactor and a shared decision head. Its output dimension follows the candidate set supplied with each request: option names and optional criteria are inputs, rather than classes fixed in the model parameters. Decision probabilities are computed without autoregressive answer generation.

OpenJev-4B is post-trained on OpenJevData-140k in two stages: supervised fine-tuning (SFT) followed by REINFORCE-Analysis (REINFORCE-A). SFT establishes decision behavior using predominantly hard labels. RL increases the share of soft-target contexts and refines probability quality through sampled outcome feedback.

2. Model Summary

OpenJev-4B brings together the following architecture and training features:

• Open-Vocabulary Decisions: Accepts request-defined options and criteria, returning explicit probabilities for selection, binary judgments and rubric-based scoring.
• Permutation Equivariance: Shared option encoders and an order-independent decision head make predictions follow the options when they are reordered, up to numerical precision.
• Cross-Option Understanding: A lightweight Option Set Interactor models semantic relationships across candidates, including overlap, competition and references to other options.
• Direct, Parallel Inference: Computes decision probabilities without autoregressive answer generation, with parallel option branches and shared-prefix reuse for attention caches and recurrent states.
• Two-Stage Post-Training: Full-parameter SFT → REINFORCE-Analysis combines predominantly hard-label supervision with a larger share of soft-target contexts during RL to refine probability quality.
• Efficient Outcome-Based RL: REINFORCE-Analysis samples 16 candidate actions with 10% uniform exploration and learns from binary outcome feedback. Analytic computation, importance correction and a leave-one-out baseline reduce reliance on sampled estimates.
• Replay and Reference Regularization: A fixed 25% of SFT contexts are replayed with the same RL objective. A frozen SFT reference provides KL regularization to help retain learned behavior.

Training data: OpenJevData-140k.

3. Evaluation Results

1. Accuracy (↑): the percentage of hard-label examples for which the highest-probability option matches the gold answer. Higher is better; unsupported or failed predictions count as incorrect.
2. Brier (↓): the average sum of squared differences between predicted and gold option probabilities. Lower is better, with 0 indicating a perfect match. For hard labels, the gold distribution is one-hot; for soft labels, it is the provided target distribution.

We compare OpenJev-4B with Jev, Decider 4B, JevK5 4B, and Intern-Decision-4B on JevBench Public.

1. Jev: official API, evaluated as Jev 1.13.0.
2. Intern-Decision-4B: official inference code and temperature 1.99241824, BF16, SDPA, no input truncation, in the existing Torch 2.11.0 / Transformers 5.17.0 environment.
3. External model versions: Decider 4B v2.1 and JevK5 4B v0.3. Their returned decision distributions are used for probability metrics. OpenJev-4B is our final release model; earlier OpenJev checkpoints are omitted.
4. JevBench Public: local evaluations of the v1.3.0 public suite: Original (72), Easy (48), Hard (111). Accuracy and Brier follow its gold-label scoring protocol; all five models return valid predictions for all 231 examples. “Hard” names a difficulty cohort here, rather than a target type. These results are separate from the official private leaderboard. Easy Brier is shown to six decimal places to retain the small differences. Bold marks the best value for each metric; zero probabilities are not clipped.
5. Evaluation context: OpenJev training includes newly constructed scenarios informed by public decision-task patterns. The table reports performance on this fixed public evaluation collection.

Machine-readable values and evaluation counts are included in evaluation.json.

4. Model Weights and Code

The complete inference and two-stage training implementation is available at ejhshen/OpenJev. This model repository contains the text backbone, decision head, tokenizer, model configuration and calibration settings. Load them together using the OpenJev runtime.

Run a decision

Install the GitHub repository in a compatible CUDA environment, then run:

from openjev.runtime.engine import DecisionEngine

engine = DecisionEngine("shenjunhao/OpenJev-4B", execution_mode="expanded")
result = engine.predict({
    "state": "A customer recognizes a purchase but reports being charged twice.",
    "questions": {
        "route": {
            "type": "choice",
            "instructions": "Which team should handle this request?",
            "criteria": {
                "Duplicate payment": "The same purchase was charged more than once.",
                "Unrecognized payment": "The customer does not recognize the purchase."
            }
        }
    }
})
print(result["answers"]["route"]["choice"])
print(result["answers"]["route"]["probabilities"])

The model accepts a shared state and a questions object. For choice, each question supplies instructions and a mapping from candidate names to criteria; candidate names are defined by the request. It returns the selected candidate and its full probability distribution. The same interface supports noul for binary judgments and score for expected rubric-level scores. See the input/output format and HTTP example for details.

The runtime downloads the complete artifact from this repository or accepts a local directory. The release uses temperature 1.0, matching the evaluation results. Decision outputs are computed directly, without generating explanation text.

Train SFT and REINFORCE-A

Training data is available at shenjunhao/OpenJevData-140k. The repository includes data preparation, the optimized distributed training engine, sample replay, resumable caches, checkpointing and model export. To fine-tune OpenJev-4B, set up the official verl/SGLang environment described in the training guide, then load the complete released model:

hf download shenjunhao/OpenJev-4B --local-dir work/base-model
python -m openjev.cli validate-artifact work/base-model
python -m openjev.training.prepare_data \
  --dataset shenjunhao/OpenJevData-140k \
  --mandatory-replay training/mandatory-replay.json --output work/data
NPROC=8 bash training/run_two_stage.sh

The pipeline runs SFT, exports the SFT artifact, caches frozen reference probabilities, runs REINFORCE-A and exports the final decision model. Both stages use global batch 1,024 by default; their step counts are calculated from the prepared sample counts. Individual stage launchers and resume instructions are included in the training guide.