Jev_Qwen3.8-27B-GGUF
GGUF quantization of SargeDev/Jev_Qwen3.8-27B.
Available quant
- Jev_Qwen3.8-27B-Q4_K_M.gguf — Q4_K_M, about 15.8 GiB, 4.92 BPW
SHA256:
c6260f14c63942d4f468176d4a24838684976b5f1b2789bb3d6c06491d877809
Important conversion note
The source config declares mtp_num_hidden_layers: 1, but the published source weights contain 64 main blocks (blk.0 through blk.63) and no MTP / NextN tensors.
A normal conversion therefore advertises 65 blocks and fails to load in llama.cpp because blk.64.* tensors are absent.
This target GGUF was converted with llama.cpp bd4f514 using --no-mtp, which produces the correct 64-block target model.
Smoke test
Validated with llama.cpp bd4f514 on an RTX 4090.
The source model was trained with thinking disabled. For the tuned behavior, use reasoning off.
Example:
llama-cli -m Jev_Qwen3.8-27B-Q4_K_M.gguf -ngl 999 -c 4096 --reasoning off
Observed behavior matches the source model card's stated goal: brief calibrated judgments with explicit confidence and less verbose hedging.
Calibration A/B
A small held-out A/B test was run against the model's exact upstream base, huihui-ai/Huihui-Qwen3.8-27B-abliterated, using the same Q4_K_M class, llama.cpp bd4f514, reasoning off, temperature 0, and identical prompts.
The primary sample contained 90 held-out rows from SargeDev/jev-distill-corpus-v3/test_set_30k: 30 noul, 30 choice, and 30 score.
| Metric | Huihui base | Jev |
|---|---|---|
| Brier score ↓ | 0.2703 | 0.1471 |
| Argmax accuracy | 53.3% | 68.9% |
| Overconfidence gap ↓ | +0.208 | +0.012 |
| Soft ECE-10 ↓ | 0.208 | 0.085 |
| Top-class Brier ↓ | 0.1138 | 0.0642 |
Paired Brier delta (Jev - base) was -0.1232 with a bootstrap 95% CI of [-0.1905, -0.0678]. Jev had lower Brier error on 60 rows, tied on 9, and was worse on 21.
Argmax disagreements also favored Jev: 19 rows were correct only for Jev versus 5 correct only for the base (McNemar exact p = 0.0066).
By decision type
| Type | Huihui Brier | Jev Brier |
|---|---|---|
| noul | 0.2708 | 0.1280 |
| choice | 0.3480 | 0.1791 |
| score | 0.1921 | 0.1343 |
Important caveats
This does not establish universal real-world calibration superiority.
- Much of the yuri_v3 target distribution was distilled from Jev 1.13, so Jev has a same-family teacher advantage on those labels.
- A separate source-balanced 90-row sanity check still improved Brier overall (0.3216 → 0.2353), but the paired 95% CI was only just below zero.
- openjev_v2 alone showed no clear advantage.
- Jev assigns exact 0.0 probability more often than the base. This can badly hurt KL/log-loss when target mass is nonzero.
- Jev often prepends phrases such as "My calibrated probability:" before otherwise valid JSON, so strict JSON compliance is lower than the base.
An exploratory post-processing sweep found that a small amount of uniform probability smoothing can improve both Brier and log-loss by reducing the exact-zero issue. Because the smoothing strength was inspected on evaluation data, this is a diagnostic result rather than a recommended calibrated setting.
Source
- Model: SargeDev/Jev_Qwen3.8-27B
- Base: huihui-ai/Huihui-Qwen3.8-27B-abliterated
- Dataset: SargeDev/jev-distill-corpus-v3
All model credit belongs to the original authors. This repository provides the GGUF conversion.