Qwen3.8-27B abliterated — groupwise-int for NInfer

A groupwise-int artifact of huihui-ai/Huihui-Qwen3.8-27B-abliterated, packaged for the NInfer engine.

This is the refusal-removed counterpart to the official neroued/Qwen3.8-27B-NInfer artifact, using the same weights profile. One file carries the complete container — Text, Vision, MTP, the optimized proposal head, tokenizer, chat template, generation config and media-processor resources.

qwen3_8_27b_huihui_abliterated.ninfer
  18,210,531,328 bytes
  sha256 8c9f9d67a07ac97506978f6db6695d8074f78dec0fb80c4a85a8fb6fbedd7f03
  identity  qwen3.8-27b / groupwise-int      container v2      1124 objects

There is also an NVFP4 build of this same checkpoint. It is ~35 % faster on speculative decode and 1.6–2× on prefill, for 3.1 GiB more VRAM. Prefer NVFP4 unless VRAM is tight. This profile's advantages are a smaller footprint (15.9 GiB resident vs 19.0) and a dramatically simpler build — one command, no external quantizer, no calibration data, and no patch of any kind.

Why this exists

Qwen3.8-27B is the king of the local models right now, and the great work Neroued has done with the NInfer engine runs it extremely quickly on Blackwell. There has also been a lot of work on obliterating and otherwise liberating models, including Pliny's OBLITERATUS suite.

Some abliterated models have been "ninferized" already — see Related work below. Two things I hadn't seen:

• the walkthrough/recipe to do it yourself
• Pliny's OBLITERATUS Qwen3.8-27B, ninferized

Both are offered here. The recipe lives in the NVFP4 repo — AGENT.md, SUPPORTED.md and scripts/. Under $5 of RunPod time converts a compatible checkpoint; measured runs came in at $1.69 and $3.30.

What it is, precisely

Format conversion only. No fine-tuning, no re-abliteration, no behavioural editing. All refusal characteristics are inherited unchanged from the upstream huihui-ai checkpoint, which removes the refusal direction by weight editing (per remove-refusals-with-transformers, the Arditi et al. 2024 family) on layers 18–51 only — layers 0–17 and 52–63, the vision tower and the MTP head are untouched.

Quantization is round-to-nearest group quantization performed by NInfer's own converter. Nothing external is involved.

Numeric allocation

Layouts: contiguous-le-v1 679, row-split-k128-v1 439. All grouped-integer tensors use MAXABS_F16_RECIP_RNE_V1.

Structurally identical to the official groupwise-int artifact — same 1124 objects, same per-format and per-layout counts, same file size to the byte. Only the values differ.

Requirements

• Blackwell GPU, compute capability 12.0 (sm_120a). NInfer's build rejects any other architecture.
• ~15.9 GiB VRAM for weights; a 32 GB card leaves ~13.6 GiB for KV cache.
• Stock NInfer — no engine patch required.

Quick start

ninfer qwen3_8_27b_huihui_abliterated.ninfer \
  --prompt "Explain group quantization in three sentences." \
  --max-context 16384 --max-new 256 \
  --spec mtp --draft-tokens 5 --lm-head-draft

Serving:

ninfer-serve qwen3_8_27b_huihui_abliterated.ninfer \
  --model-id qwen3.8-27b-huihui-abliterated \
  --max-context 262144 --kv-capacity auto --max-concurrency 1 \
  --spec mtp --draft-tokens 5 --lm-head-draft \
  --preserve-thinking --vision \
  --host 0.0.0.0 --port 8080

--model-id is recommended: the on-disk identity is qwen3.8-27b/groupwise-int, identical to the official artifact, so without it the two are indistinguishable to any client.

Measured

Same GPU, identical flags, --max-context 16384, greedy.

Scope of validation — read this

Verified: the artifact loads on stock NInfer; MTP speculative decoding engages at 86.2 % acceptance with zero fallbacks; thinking mode works; output is coherent and correct on reasoning, code and explanation prompts; the container is structurally byte-for-byte equivalent to the official artifact; the source checkpoint is intact (1199 tensors, 0 non-finite values).

Not run: no capability benchmark, no fidelity comparison against BF16, no refusal-rate measurement. No HumanEval+/GPQA numbers are claimed because none were produced. Abliteration has a documented capability cost and this artifact inherits it, unquantified.

Verify

sha256sum -c SHA256SUMS
python3 -m tools.artifact.inspect qwen3_8_27b_huihui_abliterated.ninfer

Expect identity qwen3.8-27b/groupwise-int, objects 1124 (1118 tensors, 6 resources), file_bytes 18210531328.

Reproducing it

Genuinely one command. No external quantizer, no calibration set, no patch — patches/ for this profile is empty, and upstream NInfer is used unmodified at a140e7ae82a11ed2f370a4d8f2cc16268a3790b8:

python -m tools.convert.qwen3_8_27b.convert \
  --model /path/to/huihui-abliterated \
  --out   out/qwen3_8_27b.ninfer \
  --device cpu

~12 minutes. Expect this first line, which is the whole validation story:

preflight complete: 1124 objects, 1199 source tensors, device=cpu

That single message is the converter accepting an abliterated checkpoint through every gate it has: SHA-256 over six frontend resources, architecture member checks, shape and dtype validation across all 1199 source tensors, and a hardcoded inventory assertion.

Memory. Peak RSS is 18.2 GB. quantize_matrix upcasts to float32 and allocates several full-size temporaries — measured at 6.1× the bf16 tensor size — and both text/token_embedding and text/output_head are [248320, 5120] = 2.54 GB each. With less RAM than that the converter is OOM-killed on object 7 of 1124. --device cuda also works and needs ~18 GB VRAM.

Source choice matters. The converter's only canonical-source gate is a SHA-256 check over six frontend files. Of four abliterated candidates with a complete tensor set, only huihui-ai ships all six byte-identical to canonical (the others match 0–3 of 6), so it converts with no patch and no file substitution. Full candidate comparison and the RunPod cost breakdown are in the NVFP4 repo's REPLICATION.md.

Related work — other NInfer abliterated artifacts

Other people got here first, and their artifacts are also good. Use whichever fits — this list is here so you can find them.

Worth noting three are built on orcarouter/Qwen3.8-27B-Uncensored, which is gated.

As far as I can tell, none of the above ship a conversion recipe. That is what this repo adds.

Licence

Apache-2.0, inherited from the whole chain — Qwen/Qwen3.8-27B (Copyright 2026 Alibaba Cloud), huihui-ai/Huihui-Qwen3.8-27B-abliterated, and Neroued/ninfer are all Apache-2.0. LICENSE is the file distributed with the upstream checkpoint, notice intact (§4(c)); NOTICE carries the attribution chain and statement of changes (§4(b)). No upstream component ships a NOTICE file, so §4(d) does not apply. Unlike the NVFP4 build, this profile involved no calibration data and no modification to NInfer whatsoever.

"Qwen" is a trademark of Alibaba Cloud. Unofficial community derivative, not endorsed by or affiliated with Alibaba Cloud, huihui-ai, or the NInfer project.

A description of what the licences say, not legal advice.

Intended use and limitations

This is an uncensored model: the upstream abliteration removes the refusal direction, so it will attempt requests a stock instruction-tuned model declines. Published for research, evaluation and local deployment where that is understood and wanted. Anyone deploying it is responsible for their own safeguards and for compliance with the Apache-2.0 licence and applicable law. Refusal behaviour and any capability regression originate entirely upstream — this repository changes format, not behaviour.

Credit for the abliteration to @huihui-ai; the base model to Qwen; the engine and artifact contract to @Neroued.