✂️ Qwopus3.8-27B-Flash-V2-abliterated — exl3 6.0bpw

[!NOTE] exllamav3 EXL3 quantization of Qwopus3.8-27B-Flash-V2-abliterated — the refusal-reduced 27B model (98/100 → 13/100 on the heretic refusal benchmark, first-token KL 0.0684 vs. the base). The full-precision card carries the method, measurements, and evaluation detail.

[!TIP] ~22 GB on disk. Runs on a single 24 GB GPU at moderate context. The MTP (multi-token prediction) head is intact, so draft speculative decoding works.

📦 Quantization spec

Quantized from the full-precision abliterated weights (the exact model in the linked repo) — nothing else changed: same tokenizer, chat template, generation config, and MTP head.

📄 Files

Plus config.json (with mtp_num_hidden_layers: 1), quantization_config.json, tokenizer files, generation_config.json, and the processor/preprocessor configs. All 39 mtp.* tensors are present (quantized at the MTP rate), so the speculative-decoding head is fully functional.

🚀 Usage

Requires the exllamav3 build of llama.cpp, version 1.5.3 or newer — vanilla llama.cpp does not load EXL3.

# put all six shards + model.safetensors.index.json in one directory,
# then load the first shard:
llama-server -m model-00001-of-00006.safetensors -ngl 99 -c 32768

# with MTP draft speculative decoding (the head is in the file):
llama-server -m model-00001-of-00006.safetensors -ngl 99 -c 32768 --draft-max 2

Chat template and thinking/non-thinking modes behave exactly as in the full-precision model. The base family's recommended settings carry over for coding/agentic work: enable_thinking=True, reasoning_effort="xhigh", temperature=0.3 (supplied by the caller).

⚠️ Caveats

• 6.0 bpw is high-fidelity but not lossless — for work where every point of divergence matters, use the full-precision repo.
• The model keeps the same trade-off as its full-precision source: it is refusal-reduced, not safety-aligned. See the linked card for the details.