Solar-Decision-JEV-27B-V2-GGUF

GGUF ladder for the V2 r32 + solar-1894 merge (bf16 repo: SargeDev/Solar-Decision-JEV-27B-V2). MTP/nextn drafter tensors are preserved in the f16 export (blk.64.nextn.*, 866 tensors per quant) so speculative decoding works in llama.cpp.

Sizes

(bench: one GB10, CUDA build, -ngl 99, pp512/tg128, 2 reps, on Q4_K_M and Q8_0 only — the rest of the ladder ships unbenchmarked; decode scales roughly inversely with quant size)

Q4_K_M is the daily-driver pick; Q6_K for maximum fidelity; Q5_K_M in between.

The honest headline

The decision-gate numbers live on the bf16 card. This ladder is the same weights in GGUF form, and the V2 merge measured below its r32 parent on the decision cells (choice top-1 0.828 vs 0.869 — full ablation table on the bf16 card). Two practical notes:

• llama.cpp cannot serve typed decisions with this build — it returns no token logprobs on the completion endpoints, and the decision path needs the 24-verbalizer logprobs. Use these GGUFs for fast everyday chat; use the bf16 repos in vLLM for calibrated typed decisions.
• Pass enable_thinking: false (via --chat-template-kwargs) or the template will emit its reasoning trace into the answer.

Quick start

llama-server -m Solar-Decision-JEV-27B-V2-Q4_K_M.gguf -ngl 99 --jinja \
  --chat-template-kwargs '{"enable_thinking": false}' --alias solar-jev-27b-v2

Lineage & credits

• AutoTrust AI — r16/r32 decision LoRA recipe + 24-slot head + machinery (autotrust/JEV-27B).
• huihui-ai — abliterated Qwen3.8-27B backbone.
• SargeDev — training corpora: jev-distill-corpus-v3 (questions/states) and solar-decisions-corpus-v4 (the calibrated typed-decision labels the solar half trained on, labeled by upstage/solar-decide-flash).
• Qwen — base architecture.

Built on 2× NVIDIA GB10 (DGX Spark). Apache-2.0.