Qwen3.8-27B EfficientThink · SimPO · KQV-Q4 + IQ3 Floor · MTP

English ｜ 中文（点击跳转）

Qwen3.8 27B is a great model. Capable, but it eats VRAM: BF16 wants ~54 GB, Q6_K still wants 20.8 GiB, a typical Q3 build sits at 12-13 GB, and on a 16 GB card the context budget is simply too tight. Most GGUFs out there take the uniform-quantization road, straight down to Q3 territory — the size isn't really reduced by much, and the quality is just mediocre. I've gone through a fair number of other people's quantized builds, and none of them satisfied me.

So I hand-picked the base model, and quantized with focus and selectivity, using a mixed-precision strategy I worked out on my own. This is the KQV-Q4 + IQ3 Floor build of the SimPO Q6_K weights: 20.79 GiB down to 10.12 GiB (-51%), without letting anything that matters drop — every K/Q/V anchor block stays at Q4_K or above, every norm stays F32, and the FFN layers ride the IQ3_XXS floor, calibrated with an importance matrix. Built-in MTP. The source is Q6_K, so the quality is guaranteed. On a 16 GB card with a Q4 KV cache, 130k+ context still fits.

What sets it apart: quantized straight from Q6_K — the SimPO post-training distribution is the direct source, with no BF16 round trip; the full 15-tensor MTP head; 130k+ context on 16 GB. All of it in one 10 GB file.

Built on Qwen3.8; thanks to the Qwen team, and to nerkyor for the excellent base. This file is that family's SimPO / Q6_K branch: grafted with MTP, then mixed-precision quantized.

Files

SHA256 of this model: 80e3f758f8bca53b07754d80ceb5310df74d18e9d2b1d7d3925a9f5dda9ae9b9

Mixed precision: KQV-Q4 + IQ3 Floor

866 tensors, 3.18 BPW. The allocation was extracted block-by-block from the accepted production 10 GB build:

Type histogram: Q4_K 49 · IQ3_S 130 · IQ3_XXS 327 · F32 360 — identical to the production pattern, verified per tensor. The IQ3-level tensors were quantized with the unsloth importance matrix (1,251 chunks × 8,192 tokens, 496 weight entries covering blk.0-63); the ten MTP-head and embedding tensors without matrix entries use standard quantization, exactly like the production model.

Why this beats the plain 10 GB tier

1. Half the size, full anchors — 20.79 → 10.12 GiB (-51%) while every K/Q/V anchor stays at Q4_K and every norm at F32.
2. The MTP head is inside — some 10 GB builds ship with MTP, some don't; this one carries the complete 15-tensor head and runs --spec-type draft-mtp at roughly 2-3× on chat-length outputs.
3. 130k+ context on 16 GB — a same-size f16-KV build gives you a fraction of that.

MTP

• block_count = 65, nextn_predict_layers = 1 — the MTP head is a full decoder block at blk.64 (11 attention/FFN tensors + 4 nextn tensors).
• Head precision: attn_v = Q4_K, the remaining weights = IQ3_S (one step above the floor), all norms F32.
• Draft quality inherits the Q6_K distribution — and with it, the acceptance rate.

Usage

Tested on llama.cpp b10728:

llama-server.exe `
  -m Qwen3.8-27B-EfficientThink-SimPO-KQVQ4-IQ3Floor-10G-MTP.gguf `
  -c 131072 -ngl 99 -t 8 --kv-unified `
  --jinja --reasoning on --reasoning-format deepseek `
  --spec-type draft-mtp -np 1 --spec-draft-n-max 2 --spec-draft-p-min 0 `
  --cache-type-k q4_0 --cache-type-v q4_0 -fa on `
  --top-k 20 --top-p 0.95 --repeat-penalty 1.1 --min-p 0

• --spec-type draft-mtp — turns on the built-in MTP head (expect ~2-3× on chat-length outputs).
• --cache-type-k q4_0 --cache-type-v q4_0 — Q4 KV cache; this is what lets 130k fit in 16 GB.
• For vision, pair it with the Qwen3.8-27B vision projector from the base family via --mmproj.
• LM Studio: import the GGUF directly and enable speculative decoding (MTP) in the server settings.

Verification

Disclaimer

This model is provided AS IS, without warranty of any kind, express or implied; the Apache 2.0 license terms apply. It is a personal research build — intended for individual research and evaluation, not for public or production deployment. As an uncensored mixed-precision build, outputs are unpredictable: you are responsible for your own prompts and outputs. Test thoroughly in your own environment before relying on anything it generates; all risk is yours, and the author assumes no liability for any loss.

Base model: nerkyor/Qwen3.8-27B-EfficientThink-Uncensored-K3-Opus5-Grok4.6-GPT5.6Sol-SFT-SimPO-DFlash2.

---

中文说明

Qwen3.8 27B 是一个伟大的模型。能力够，但是显存也吃得凶：BF16 要 ~54 GB，Q6_K 也要 20.8 GiB，通常的 Q3 要 12G 到 13G，一张 16 GB 的卡，上下文太紧缺了。而且市面上的 GGUF，大多走均匀量化、压到 Q3 那一档——体积其实没有压缩多少，质量还一般。我也看过不少别人量化的版本，始终没有一款令我满意的。

所以精选了底模，又用自己探索出的混合量化策略，有重点、有选择地进行量化。这就是 SimPO Q6_K 权重的 KQV-Q4 + IQ3 Floor 混合精度版：20.79 GiB 压到 10.12 GiB（-51%），该保的地方一处不降——每个 K/Q/V 锚点块不低于 Q4_K，每个 norm 保持 F32，FFN 走 IQ3_XXS 地板并用重要性矩阵校准。内置 MTP。源模型是 Q6_K，质量有保证。16 GB 显卡 + Q4 KV 缓存，13 万+ 上下文照样装得下。

它的不同之处：直接从 Q6_K 量化——SimPO 的后训练分布就是量化源，不绕 BF16 中转；带完整 15 张量 MTP 头；16 GB 开 13 万+ 上下文。都装在一个 10 GB 的文件里。

基于 Qwen3.8；感谢 Qwen 团队，感谢底模作者 nerkyor。 本文件即该家族 SimPO / Q6_K 分支：移植 MTP 后做混合精度量化。

文件

本模型 SHA256：80e3f758f8bca53b07754d80ceb5310df74d18e9d2b1d7d3925a9f5dda9ae9b9

混合精度：KQV-Q4 + IQ3 Floor

866 张量，3.18 BPW。分配表从已验收的生产 10 GB 构建逐块提取：

类型直方图 Q4_K 49 · IQ3_S 130 · IQ3_XXS 327 · F32 360 与生产模式一致，逐张量校验通过。IQ3 级张量用 unsloth 重要性矩阵量化（1,251 chunks × 8,192 tokens，496 条覆盖 blk.0-63）；无矩阵条目的 10 个 MTP/嵌入张量走标准量化，与生产模型完全相同。

相对普通 10 GB 档的优势

1. 体积减半、锚点全保 —— 20.79 → 10.12 GiB（-51%），K/Q/V 锚点全部 ≥ Q4_K，norm 全部 F32。
2. MTP 头在里面 —— 同档有的带 MTP、有的不带；这个带完整 15 张量头部，--spec-type draft-mtp 在聊天长度输出上约 2-3×。
3. 16 GB 开 13 万+ 上下文 —— 同体积 f16-KV 构建只有零头。

MTP

• block_count = 65、nextn_predict_layers = 1——MTP 头是 blk.64 完整块（11 个注意力/FFN 张量 + 4 个 nextn 张量）。
• 头部精度：attn_v = Q4_K，其余权重 = IQ3_S（地板上一档），norm 全部 F32。
• 草稿质量继承 Q6_K 分布，接受率也随之而来。

用法

llama.cpp b10728 实测：

llama-server.exe `
  -m Qwen3.8-27B-EfficientThink-SimPO-KQVQ4-IQ3Floor-10G-MTP.gguf `
  -c 131072 -ngl 99 -t 8 --kv-unified `
  --jinja --reasoning on --reasoning-format deepseek `
  --spec-type draft-mtp -np 1 --spec-draft-n-max 2 --spec-draft-p-min 0 `
  --cache-type-k q4_0 --cache-type-v q4_0 -fa on `
  --top-k 20 --top-p 0.95 --repeat-penalty 1.1 --min-p 0

• --spec-type draft-mtp——启用内置 MTP 头（聊天长度输出预期 ~2-3×）。
• --cache-type-k q4_0 --cache-type-v q4_0——Q4 KV 缓存；13 万装进 16 GB 靠的就是它。
• 多模态：用 --mmproj 配底模家族的 Qwen3.8-27B 视觉投影器。
• LM Studio：直接导入 GGUF，服务端设置里启用投机解码（MTP）。

验证

免责声明

本模型按原样（AS IS）提供，不作任何明示或暗示的保证；以 Apache 2.0 许可证条款为准。本构建为个人研究用途——供个人研究与评估，不适合公共或生产部署。作为**无审核（uncensored）**混合精度构建，输出不可预期：使用者对自己的提示与输出负责。请在自己的环境中充分测试后再决定是否依赖其生成内容——风险自负，作者不对任何损失承担责任。

底模：nerkyor/Qwen3.8-27B-EfficientThink-Uncensored-K3-Opus5-Grok4.6-GPT5.6Sol-SFT-SimPO-DFlash2。

---

底模权重与许可证归 nerkyor。MTP 移植与 KQV-Q4 + IQ3 Floor 混合精度量化为本构建新增。 Base weights and license by nerkyor, with thanks. MTP graft and mixed-precision quantization added by this build.