Qwen3.8-27B OBLITERATED — The Pocket-Quant Ladder
A 27B dense hybrid (16 full-attention + 49 gated-delta SSM layers) with its MTP draft head intact, squeezed into the two smallest careful quants we could build. One 24 GB card runs the IQ1_M with room for context. Self-speculating via draft-mtp — no separate draft model needed.
🪜 The Ladder
| File | Size | ~bpw eff | MTP block | Min rig |
|---|---|---|---|---|
Qwen3.8-27B-OBLITERATED-IQ2M.gguf |
12.0 GB | ~3.5 | — | 1× RTX 4090 / 5090 (24 GB) |
Qwen3.8-27B-OBLITERATED-IQ1M.gguf |
10.2 GB | ~2.9 | — | 1× 3090/4090/5090 (24 GB, tons of ctx) |
🧠 The Tensor Budget
Dense is the hardest quant class — no expert redundancy to hide behind. The budget:
| Tensor | Type | Why |
|---|---|---|
token_embd |
Q8_0 | every token |
output |
Q6_K | logits |
attention (attn_q/k/v/output) |
Q5_K / Q6_K | 16 full-attn layers, protected |
ssm* (gated-delta SSM layers) |
Q5_K | 49 layers of dynamics — the model's memory |
blk.64 (MTP block) |
Q8_0 | draft head exact → speculation acceptance stays high |
FFN (ffn_gate/down) |
body ftype | the bulk |
🚀 Serving (llama.cpp ≥ v0.4.1)
llama-server \
-m Qwen3.8-27B-OBLITERATED-IQ2M.gguf \
-ngl 99 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 \
--spec-type draft-mtp --spec-draft-n-max 2 \
--jinja --ctx-size 32768
The MTP block (blk.64) was pinned at Q8_0 through the entire chain — draft quality preserved, acceptance rate stays honest. Requires a build with qwen3_5 hybrid support (llama.cpp ≥ mid-Sept 2026 master / v0.4.1, includes the GDN normalization fix).
📜 Provenance
- Source:
OBLITERATUS/Qwen3.8-27B-OBLITERATED(BF16, 55.6 GB, 1.2M-download abliteration) - Master: converted to Q8_0 with the MTP head kept and verified
- imatrix: fresh, 32 chunks, security-weighted corpus
- Ladder: IQ2_M + IQ1_M dynamic requants, uploaded then cleaned
🙏 Credits
- OBLITERATUS — the source obliteration
- The Qwen3.5 hybrid-attention lineage (gated delta net)
- The llama.cpp imatrix / i-quant ecosystem
⚠️ Responsible Use
Obliterated/uncensored: intended for red-teaming, security research, evaluation, and unfiltered assistant work. You are responsible for what you do with it. Don't do crimes.