Solar-JEV-27B-R32-GGUF

GGUF ladder for the r32-only build — the one with the decision readout trained INTO the model (lm_head LoRA), and the best-measured build of the Solar JEV family.

These are the same weights as SargeDev/Solar-JEV-27B-R32 (bf16) and SargeDev/JEV-27B-Uncensored; this repo is the quantized form.

Sizes

quant	size	decode tok/s (prompt/tokens)
IQ4_XS	14.35 GiB	963.27 / 13.93
Q2_K	10.11 GiB	753.44 / 18.29
Q3_K_L	13.55 GiB	798.24 / 14.01
Q3_K_M	12.56 GiB	821.83 / 14.99
Q3_K_S	11.40 GiB	805.38 / 16.28
Q4_K_M	15.65 GiB	859.45 / 12.52
Q4_K_S	14.73 GiB	907.39 / 13.32
Q5_K_M	18.18 GiB	829.93 / 11.19
Q5_K_S	17.66 GiB	851.30 / 11.54
Q6_K	20.88 GiB	726.09 / 9.79
Q8_0	27.04 GiB	808.14 / 8.16

MTP/nextn drafter tensors are preserved in every quant, so speculative decoding works in llama.cpp.

Which one to pick

you want	repo
the best decision model, quantized	this repo
the best decision model, bf16	`SargeDev/Solar-JEV-27B-R32`
the r32+solar merge (r32 + our solar adapter)	`SargeDev/Solar-Decision-JEV-27B`
the solar-only build (our adapter alone)	`SargeDev/Solar-JEV-27B`
the vision build	`SargeDev/Solar-Decision-JEV-27B-VL`

The decision-gate numbers for all of them are on the bf16 cards, measured on the same 27,695 rows with the same harness and the same head + calibration.

Quick start

llama-server -m Solar-JEV-27B-R32.Q4_K_M.gguf -ngl 99 --jinja \
  --chat-template-kwargs '{"enable_thinking": false}'

Typed decisions: use the bf16 repo for those

The typed-decision path needs the model's own lm_head logprobs over the 24 verbalizer token ids, which works in vLLM (the bf16 repo, verified end-to-end). The llama.cpp server in this build does not return token logprobs - we measured /v1/completions, the native /completion endpoint, and echo: true, and each returns a payload with no logprobs field. So read typed decisions from the bf16 repo, and use this ladder for fast everyday chat.