Solar-Decision-JEV-27B-V3-GGUF

The V3 solar run: combined corpus + r32 geometry + lm_head readout. huihui Qwen3.8-27B-abliterated + our solar LoRA (r32/α64) trained on the union of jev-distill-corpus-v3 + solar-decisions-corpus-v4 (764,626 rows, 90 domains) with lm_head added to the target modules, merged at step 2184.

This is the third generation of SargeDev/Solar-Decision-JEV-27B. Two changes vs V2 (r32+step-1894-solar):

1. Training data: V2 trained the solar half on solar-decisions-v4 alone. V3 trains on the union — the r32-style verbalizer/decision-shape rows are in the mix, attacking the readout gap at the data level.
2. Target modules: V2 targeted attention+MLP only. V3 adds lm_head, so the decision head's output vocabulary geometry is trained alongside.

The decision readout is trained INTO the model — a typed decision is a single prefill with max_tokens=1, read back as log-probabilities over the option tokens. No separate head to run. (vLLM only — llama.cpp does not return the per-token logprobs this path needs; use the GGUF for chat, not for decisions.)

Gate results, honestly (27,695 rows, same harness/head/calibration as every number below)

(below bar) on choice/score top-1: V3 closes ~⅔ of the gap V2 left to r32-alone, but does not reach the bar. The sharp rise in choice ECE (0.021 → 0.167) means the model is much more confident on hard rows — better ranking, worse calibration. If you need calibrated probabilities, prefer r32-alone (SargeDev/JEV-27B-Uncensored); if you need top-1 accuracy from a self-contained model, V3 is our best.

What's in the repo

• bf16 merged weights (19 shards, 52 GB) — decisions (vLLM) + everyday chat
• decision_head.json + calibration.json + serve_decide.py — the typed-decision machinery (credit AutoTrust AI)
• MTP/nextn drafter tensors preserved (speculative decoding works in llama.cpp)

Quantizations

• NVFP4-FP8: SargeDev/Solar-Decision-JEV-27B-V3-NVFP4-FP8 (29 GB)
• GGUF ladder (11 quants, f16 + Q2_K … Q8_0): SargeDev/Solar-Decision-JEV-27B-V3-GGUF

llama-bench (GB10, CUDA build, -ngl 99, pp512 / tg128)

Lineage & credits

• huihui-ai — abliterated Qwen3.8-27B backbone
• AutoTrust AI — r32 decision LoRA, 24-slot decision head, calibration, serve_decide.py
• SargeDev — jev-distill-corpus-v3, solar-decisions-corpus-v4 (the training data)
• Qwen — base model
• TypeSafe AI — Jev 1.13 teacher (referenced, not redistributed)