Huihui-Qwen3.8-27B-abliterated, NInfer v3 artifact

A NInfer v3 artifact of huihui-ai/Huihui-Qwen3.8-27B-abliterated, the abliterated Qwen3.8-27B with the lowest reported divergence from the base checkpoint among the widely used releases (the first fifteen layers are untouched; independent KL divergence measurements sit around 0.008) and a working MTP head.

Built for NInfer-all, the master branch of iamwavecut/ninfer-all: ashalliants' NInfer-3090 consolidated with the fixes and features of the other NInfer forks. Since September 2026 it serves the RTX 3090, RTX 4090 and RTX 5090, each with its measured kernel route profile built in (other GPUs calibrate once at first start).

What is inside

Conversion command (from the fork's tree, CUDA device):

python3 -m tools.convert \
  --model Huihui-Qwen3.8-27B-abliterated \
  --recipe qwen3_8_27b \
  --source dflash2=Qwen3.8-27B-DFlash2 \
  --components text,vision,mtp,dflash2 \
  --resource chat_template.jinja=tools/chat_templates/qwen3_8.jinja \
  --proposal \
  --name qwen3.8-27b-abliterated \
  --out Huihui-Qwen3.8-27B-abliterated-ninfer-v3.ninfer

SHA256SUMS lists the checksum of every file.

Running

Any NInfer build that reads v3 artifacts (v0.11.0 or later). A single RTX 3090 serves the full 198,400-token window with vision and MTP:

ninfer-serve Huihui-Qwen3.8-27B-abliterated-ninfer-v3.ninfer --model-id qwen3.8-27b \
  --max-context 198400 --kv-capacity 198400 --kv-dtype rk8v4 --gdn-state-fp16 \
  --spec mtp --draft-tokens 3 --lm-head-draft \
  --vision --vision-residency overlay --vision-max-merged 12288

The artifact also carries the DFlash2 adapter for --spec dflash2; only the MTP path was measured on this artifact.

On the current master

The September 2026 reference measurements of master (2172a598) cover the upstream neroued/Qwen3.8-27B-NInfer artifact, built with the same qwen3_8_27b recipe; this artifact was not measured again, and its drafts may be accepted at other rates than the upstream model's. One request, greedy, thinking off, text only:

Past 262,144 tokens the window runs with --rope-yarn; filled to the largest rk4v4 window, the upstream model returned all three codes planted in the document on every card. Every table is in the reference measurements.

Measured on an RTX 3090 (September 2026, franken-v0.11)

A Release build (sm_86, CUDA 13.1) of the fork's franken/v0.11 branch, now kept as the tag franken-v0.11, with the command line above plus --max-concurrency 4 --host-kv-mib 12288 --host-state-slots 8, greedy decoding, thinking off. The reference is WaveCut/Qwen3.8-27B-MXFP8-CRACK-NInfer-v3 served by the previous NInfer line of the same fork with the same flags and the same client.

Memory with one active request: 16.7 GiB of weights and 5.54 GiB of runtime state at the 198,400-token rk8v4 window, which leaves 1.12 GiB free on a 24 GiB card.

Decode speed is the median per request with MTP (three draft tokens, about 3.3 accepted per round on prose).

Quality over 1,179 items (GSM8K 200, MMLU-Pro 280, HumanEval 164, MGSM-ru 250, Global-MMLU-ru 285), the same items and client for every column:

Against the previous engine's CRACK run this artifact wins 46 items and loses 37 (exact McNemar p = 0.38); no suite differs significantly.

Perplexity on the quick corpus of ninfer-ppl-1m-v1 (261,167 scored tokens, 4,096-token window, 2,048-token stride, rk8v4 KV): 4.3750 for this artifact against 4.4685 for CRACK on the same engine.

Caveats

This is an abliterated model: refusals are removed at the checkpoint level, and the usual hesitation before destructive actions goes with them. Put the guardrails in the tool layer, not in the model. rk8v4 is a lossy KV format (about +0.08 % perplexity against bf16 on the vanilla checkpoint) chosen for the context window; --kv-dtype int8 or bf16 trade window for fidelity.