JEV-27B-VL-MLX-4bit

autotrust/JEV-27B-VL for Apple Silicon with MLX-VLM. JEV answers typed questions (yes/no, one of 2–256 options, a 0–5 score) about text and images with a calibrated probability for every option, in one forward pass. All credit for the model goes to AutoTrust AI; this repository only converts it to MLX.

It runs in about 17 GB of unified memory, so it fits a 32 GB Mac.

Status (9 Oct 2026): native JEV support is coming to mlx-vlm in Blaizzy/mlx-vlm#2466, with a different checkpoint format (the LoRA is merged into the weights) and checkpoints under nativ-community, e.g. JEV-27B-VL-MLX-MXFP4 and JEV-27B-VL-MLX-8bit. My earlier proposal, #2463, was closed in its favour.

This checkpoint does not load with mlx-vlm releases, including once #2466 is merged (it stops with KeyError: 'lora_alpha'); it needs the branch below. Use it if you want the adapter kept unmerged, so the same weights also serve as plain Qwen3.8-27B (System 2). Otherwise, prefer the upstream checkpoints once #2466 is released.

Use

# in a separate virtual environment: this replaces mlx-vlm with the branch that reads this format
pip install -U "git+https://github.com/Bayway/mlx-vlm@add-jev"

from mlx_vlm import load, predict

model, processor = load("Bayway/JEV-27B-VL-MLX-4bit")
result = predict(model, processor, "Customer: my card was charged twice for one coffee.", {
    "team": {"type": "choice", "instructions": "Which team should handle this?",
             "criteria": ["billing", "shipping", "tech support"]},
    "refund": {"type": "bool", "instructions": "Is the customer asking for a refund?"},
    "urgency": {"type": "score", "instructions": "Rate the urgency of this ticket on a 0-5 scale.",
                "criteria": ["none", "low", "some", "medium", "high", "critical"]},
})
print(result["answers"]["team"]["probabilities"])   # {'billing': 0.9979, 'shipping': 0.0, 'tech support': 0.0021}

Images go in the state, mixed with text:

state = ["Listing photo: ", {"image": "item.jpg"}, "\nSeller title: wireless earbuds, barely used"]
predict(model, processor, state, {"category": {"type": "choice", "instructions": "Which category fits this listing?",
                                               "criteria": ["electronics", "clothing", "home and kitchen", "toys"]}})

From the command line:

python -m mlx_vlm decide --model Bayway/JEV-27B-VL-MLX-4bit \
  --state "Customer: my card was charged twice for one coffee." \
  --questions '{"team":{"type":"choice","instructions":"Which team should handle this?","criteria":["billing","shipping","tech support"]}}'

System 2 (ordinary chat, optionally with thinking) is the base model:

python -m mlx_vlm generate --model Bayway/JEV-27B-VL-MLX-4bit --prompt "What does this chart show?" --image chart.png --max-tokens 300

The question types follow JEV's protocol: bool (JEV's noul; phrase it so that true is the outcome you want the probability of), choice (2–256 options, optionally "label": "one-line use-when description"; the first 16 use the trained slots), and score (JEV's fixed 0–5 scale; the six criteria are returned as a legend, so state the scale in the instructions). See the JEV-27B-VL card for prompt advice, benchmarks and limitations.

Parity

Measured on a 32 GB M2 Pro (MLX 0.32.3). A bf16 reference for a 27B model does not fit in 32 GB, so the method was validated on JEV-9B, which shares the protocol, head and adapter format, against the official PyTorch math (transformers + peft, bf16). Details: PARITY.md in this repository.

JEV-27B-VL 4-bit against the outputs published on the JEV model cards:

On 36 held-out rows of jev-distill-corpus-v3 (test_set_30k), this checkpoint's mean KL to the teacher's distributions is 0.027 with 34/36 top-1 agreement, in line with the published JEV-27B test KL (≈ 0.017 on 25k rows).

JEV-9B (same pipeline), 39 text decisions against the PyTorch reference and 8 image decisions against the vLLM-equivalent reference:

For comparison, the two official PyTorch paths (merged LoRA + float32 head, and the vLLM-style lm_head LoRA) differ from each other by up to 0.013 (mean 0.0018). At 9B, 4-bit flips decisions only where the top two options are nearly tied, but it moves probabilities by up to 0.12. The 27B checkpoint matched the published examples more closely than that, but the evidence for 27B is limited to the comparisons above. If you need tighter calibration, threshold with margin or use a 6-bit or 8-bit build where memory allows.

The same 39 text decisions on the upstream JEV-9B checkpoints from #2466 (code at 24b24bb): nativ-community/JEV-9B-MLX-8bit 39/39, max / mean 0.014 / 0.0017; JEV-9B-MLX-NVFP4 35/39, 0.147 / 0.014, again flipping only near-ties (details).

Speed on the same Mac: about 1.5 s per text decision (~150 tokens) and 2.1 s per image decision; peak memory 16.9 GB.

License

Apache-2.0, as the original: the base weights are Qwen/Qwen3.8-27B (Apache-2.0, see LICENSE, Copyright Alibaba Cloud), and the JEV adapter and decision head are autotrust's (Apache-2.0). Changes made here: the base was quantized to 4-bit by mlx-community; the adapter's zero rank padding was removed and its tensors renamed for MLX; the decision head was assembled from the base lm_head rows and the adapter's lm_head LoRA. No weights were trained.

JEV is an independent student model by AutoTrust AI, distilled from TypeSafe Jev 1.13's published output distributions; it is not affiliated with or endorsed by TypeSafe AI, and neither is this conversion.