VEJI-V2

VEJI-V2 is a compact, non-autoregressive typed decision architecture designed to compile a large reusable state once and answer many structured questions against it. It combines a frozen multilingual semantic encoder with learned evidence routing, recurrent reasoning, exact/code-aware features, joint option scoring, confidence estimation, and optional abstention.

Reported test accuracy: 82.95% on 522 evaluated questions
Evidence chunk recall: 100%
Maximum context: 250,000 characters
Trainable VEJI parameters: 3,282,499 (excluding the frozen encoder)
Warm decision throughput: approximately 87–89 questions/s in the supplied benchmark

What VEJI-V2 is

VEJI-V2 is not an autoregressive text generator. It receives:

1. a state/context,
2. a typed question/instruction, and
3. a list of candidate options,

then scores the options jointly and returns the most likely answer, probabilities, confidence status, and the spans of the routed evidence chunks.

The architecture is intended for structured decision workloads such as routing, classification, extraction, policy checks, code reasoning, workflow decisions, semantic matching, and multi-hop relation questions.

Architecture

                         ┌─────────────────────────────┐
State / context ───────► │ Chunking (768 chars, 96 ov.)│
                         └──────────────┬──────────────┘
                                        │
                                        ▼
                         ┌─────────────────────────────┐
                         │ Frozen multilingual encoder │
                         │ MiniLM-L12-v2, 384 dims     │
                         └──────────────┬──────────────┘
                                        │
                              projected chunk states
                                        │
Question ─► frozen encoder ─► query projection + type embedding
                                        │
                              top-k evidence routing
                              (k = 8 by default)
                                        │
                                        ▼
                         weighted routed memory
                                        │
                     query/memory fusion + GRU refinement
                                        │
Options ───► frozen encoder ─► option projection
                                        │
                       semantic option compatibility
                                        │
Routed text + question + option ─► exact/code features
                                        │
                                        ▼
                         semantic score + exact score
                                        │
                          learned temperature scaling
                                        │
                                        ▼
                    probabilities + confidence + abstention

Main components

• Frozen semantic compiler: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2.
• Hard autograd boundary: encoder output crosses a NumPy copy boundary before entering trainable VEJI layers.
• Reusable compiled state: long contexts are encoded once and reused across many questions.
• Evidence router: query-to-chunk routing selects the top-k chunks before decision scoring.
• Typed decisions: semantic type embeddings condition the decision process.
• Reasoning/refinement: fused query and routed memory are refined with a GRUCell for configurable hops.
• Exact/code features: exact substring, token overlap, code-token overlap, numeric matching, negation agreement, length ratio, and option prior augment semantic scores.
• Confidence behavior: low-confidence answers can be returned as uncertain instead of forcing a decision.

Default configuration

Dataset and reference training run

The saved notebook uses VEJI-Synth-v2-200K with the following indexed splits:

Dataset manifest values saved in the run:

• Core records: 200,000
• Coding records: 70,822
• Coding share: 35.411%

The reference run in the supplied notebook sampled 30,000 training records for 1 epoch, using AdamW with learning rate 3e-4, weight decay 0.01, gradient accumulation 8, and seed 92. The saved run finished with a rolling training loss of 0.4701 and approximately 14.58 records/s near the end of training. The saved runtime reports CPU execution.

Evaluation results

The reported evaluation below covers 522 questions on validation and 522 questions on test. These are the results supplied with the model run; they are not a full 5,000-record held-out evaluation.

Family-level accuracy

Expand the complete validation/test family breakdown

Result profile

The model is strongest on many routing, classification, semantic matching, policy, and code-analysis families, with numerous reported families at 100% test accuracy. The main weak areas in this evaluation are ranking (27.27%), scheduling (18.18%), temporal order (18.18%), numeric policy (45.45%), log anomaly (45.45%), and exact code-output prediction compared with the code-understanding families.

These family scores come from a small evaluation slice, so a family percentage should not be treated as a stable population estimate without a larger held-out evaluation.

250K context and speed benchmark

The benchmark compiles a 250,000-character state into 372 chunks, then reuses the compiled state for repeated questions.

Important: the ~87–89 q/s figure is warm decision throughput after compilation. It does not include the one-time ~42.4 s state-compilation cost. Hardware was not attached to the supplied benchmark result, so these latency figures should not be generalized across machines.

The notebook's separate full long-context held-out evaluation did not complete in the saved run; that cell ended with KeyboardInterrupt. The 250K architecture smoke test and the compile/reuse speed benchmark did complete.

Example

The standalone notebook exposes VEJIV2, save_model, and load_model. After loading the VEJI implementation, a decision call looks like this:

from huggingface_hub import snapshot_download
import sys

repo_path = snapshot_download(
    repo_id="loaiabdalslam/VEJI-V2"
)

print("Downloaded to:", repo_path)

sys.path.insert(0, repo_path)

from veji import VEJIV2



#from veji import VEJIV2

model = VEJIV2.from_pretrained(
    "loaiabdalslam/VEJI-V2"
)

state = """
Customer Ahmed requested a refund.
Policy: refund requests go to billing.
The purchase was 3 days ago.
"""

questions = [
    {
        "id": "route",
        "type": "choice",
        "semantic_type": "route",
        "instruction": "Which team should handle this?",
        "options": [
            "sales",
            "billing",
            "support",
            "legal"
        ]
    }
]

result = model.decide(
    state,
    questions
)

print(result)

state = (
    "Customer Ahmed requested a refund. "
    "Policy: refund requests go to billing. "
    "The purchase was 3 days ago."
)

questions = [
    {
        "id": "route",
        "type": "choice",
        "semantic_type": "route",
        "instruction": "Which team should handle this?",
        "options": ["sales", "billing", "support", "legal"],
    }
]

results = model.decide(state, questions, abstain_threshold=0.60)
print(results)

For repeated questions against the same state, compile once:

compiled = model.compile_state(state)
results = model.decide_compiled(compiled, questions)

This is the intended performance path for VEJI-V2.

Output format

Each decision includes fields such as:

{
  "id": "route",
  "answer": "billing",
  "best_candidate": "billing",
  "answer_index": 1,
  "probabilities": [0.01, 0.98, 0.005, 0.005],
  "confidence": 0.98,
  "status": "confident",
  "selected_chunk_spans": [[0, 102]]
}

If the maximum option probability is below the threshold, answer becomes null while best_candidate is retained and status is uncertain.

Exported files

The notebook exports:

VEJI_v2_export/
├── config.json
├── metrics.json
├── veji_head.pt
└── README.md

veji_head.pt contains the VEJI trainable state dict. The multilingual sentence encoder remains an external frozen dependency identified by config.json.

Production status

The notebook defines the following engineering gate:

According to that gate, this checkpoint is not yet a production candidate. The result is promising for a compact decision head, but production claims should wait for a full held-out evaluation, out-of-distribution testing, calibration improvement, and targeted work on the weak families.

Known limitations

• It is a candidate-selection / typed-decision model, not a free-form generative LLM.
• Quality depends on the candidate options supplied to the model.
• The reported 522-question validation/test subsets are relatively small.
• Several temporal, scheduling, ranking, numeric-policy, and code-output families remain weak.
• A 250K context can be reused efficiently once compiled, but the initial semantic compilation is comparatively expensive in the reported CPU run.
• The frozen encoder is an external dependency and dominates total dependency size even though the trainable VEJI head is only ~3.28M parameters.
• The original saved notebook did not specify a model license; this Hub repository now declares the MIT license.

Reproducibility notes

• Seed: 92
• PyTorch model with a frozen SentenceTransformer encoder
• Deterministic hash encoder exists only as an offline fallback/testing backend
• Training includes decision NLL, Brier loss, evidence-routing loss, and a margin term
• Evaluation reports accuracy, NLL, Brier score, 10-bin ECE, evidence chunk recall, and per-family accuracy

Citation

If you publish a paper or repository for VEJI-V2, replace the placeholder below with the final bibliographic entry:

@software{veji_v2_2026,
  title  = {VEJI-V2},
  year   = {2026},
  url    = {https://huggingface.co/loaiabdalslam/VEJI-V2}
}

License

MIT