Qwen3.8-Flash-Next — hibrid48-uncensored: the abliterated body with the 4-bit output head

hibrid47-uncensored with one tensor changed: the 1.18 GiB bf16 output head becomes 0.33 GiB of NVFP4. On the non-abliterated twin (hibrid48) that single change took the two-Spark serve from 16.9 to 22.0 engine steps/s (+30 %): the head is read about 5.4 times per decode step by speculative decoding, and it was 27 % of the step. Everything else — OrcaRouter's abliterated body, the NVFP4 n-gram table on the GPU, the re-quantized drafter — is hibrid47-uncensored, byte for byte.

The head tensor itself is identical to hibrid47's: the abliteration (Arditi et al., one refusal direction removed from 149 residual-writing matrices) never touched lm_head, and the quantizer reproduced hibrid48's global scale and error to the last digit (scale 1.533e-4, mean |Δw| = 9.0 % of |w|).

What changed vs hibrid47-uncensored

Size 98.1 GiB (hibrid47-uncensored: 99.0).

Loading — read this

vLLM 0.29 builds the output head without the checkpoint's quantization config, so stock vLLM fails on this checkpoint with a shape mismatch. The fix is two lines (pass quant_config to ParallelLMHead in the model's model.py and mtp.py), shipped as recipes/qwen38-flash-next-fast/docker/patches/11-lm-head-quant-config.py in the myllmbox repo and applied by that recipe's image. With the config it loads through vLLM's own compressed-tensors NVFP4-A16 linear scheme (Marlin on GB10). This repo is gated — accept the agreement, then hf auth login (or export HF_TOKEN=…) before the download.

git clone https://github.com/bilikaz/myllmbox-runner.git && cd myllmbox-runner
./download.sh myllmbox/Qwen3.8-Flash-Next-hibrid48-uncensored
./build-and-copy.sh qwen38-flash-next-fast           # vLLM 0.29 + the lane's patches, copied to the second Spark
# recipes/qwen38-flash-next-fast/myllmbox.yaml → model: /models/myllmbox/Qwen3.8-Flash-Next-hibrid48-uncensored
./run.sh qwen38-flash-next-fast

Repo layout

15 body shards (model-000NN-of-00017.safetensors, OrcaRouter's; shards 2 — the bf16 table — and 17 — the bf16 head — omitted) + model-lmhead-nvfp4.safetensors (lm_head.weight_packed uint8 [248320, 1280], lm_head.weight_scale fp8 [248320, 160], lm_head.weight_global_scale fp32 — compressed-tensors spelling, global scale stored as the divisor) + model-mtp.safetensors (re-quantized drafter) + 8 table shards + index. Every tensor exists exactly once.

Reproducibility

builds/qwen38-flash-next/quantize-lm-head.py --ckpt <hibrid47-uncensored> --out <this> in the myllmbox repo (the tool detects the compressed-tensors config and writes that spelling); 13 s on a Spark. Standardizer, drafter re-quantization and the table pipeline as documented on hibrid47-uncensored.

Responsible use

No guardrails. Research, red-teaming, interpretability and private use behind your own moderation; anything reachable by other people needs its own safety layer. You are responsible for lawful use and for the outputs.

Attribution & license

• Base model: Qwen/Qwen3.8-Flash-Next (Alibaba) — Qwen Community License 1.0 (included as LICENSE). It governs these weights and every derivative: modification, distribution, hosting and commercial use permitted; products over 100M MAU / $20M monthly revenue must display the model name; a Model-as-a-Service or AI-assistant business on it needs a separate Qwen license.
• Abliteration and the body's NVFP4/FP8 quantization: OrcaRouter, from orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 (their contribution offered under Apache-2.0).
• NVFP4 n-gram table, drafter re-quantization, lm_head quantization, the GPU-resident load path and the serving stack: myllmbox.