Qwen3.8-Flash-Next-Uncensored - EXL3 4.05 bpw, 8 bpw n-gram
EXL3 quantization of orcarouter/Qwen3.8-Flash-Next-Uncensored, an abliterated (refusal-removed) build of Qwen/Qwen3.8-Flash-Next. Built with ExLlamaV3 1.6.6.
Same as the 4.05 bpw build, except the n-gram (PLE) table is at 8 bpw instead of 6. Made on request, following the community discussion about higher-precision n-gram tables.
| Size on disk | 114 GiB |
| of which n-gram table | 51.8 GB (39.0 GB on the 6 bpw-table build) |
| Language model | 4.05 bpw |
lm_head |
6 bpw |
| MTP layers | 8 bpw |
| n-gram (PLE) tables | 8 bpw |
| Vision tower | unquantized (16-bit) |
The language model weights use the same settings as the 4.05 bpw build (which was quantized with ExLlamaV3 1.4.6). With ngram_ram: true the table sits in system RAM, so plan for the extra ~13 GB there.
n-gram table: 6 vs 8 bpw
Reconstruction error of the n-gram table (relative Frobenius norm, rfn,
from the conversion logs, same BF16 source; the 6 bpw table was quantized
with ExLlamaV3 1.4.6, the 8 bpw one with 1.6.6):
| n-gram bits | Table size | rfn |
|---|---|---|
| 6 bpw | 39.0 GB | 0.01664 |
| 8 bpw | 51.8 GB | 0.00433 |
About 3.8x lower error on the table itself. This measures the table only, not the effect on the model's output. A KLD comparison against the BF16 has not been run yet.
Sibling repos
| Variant | Repo |
|---|---|
| 2 bpw | Qwen3.8-Flash-Next-Uncensored-exl3-2bpw |
| 3 bpw | Qwen3.8-Flash-Next-Uncensored-exl3-3bpw |
| 4.05 bpw (6 bpw n-gram) | Qwen3.8-Flash-Next-Uncensored-exl3-4bpw |
| 4.05 bpw (8 bpw n-gram) | this one |
| 5.05 bpw | Qwen3.8-Flash-Next-Uncensored-exl3-5bpw |
Config
model:
model_name: Qwen3.8-Flash-Next-Uncensored-exl3-4bpw-ng8
cache_size: 16384
ngram_ram: true
tensor_parallel: false # true needs ExLlamaV3 1.5.1+
draft_model:
draft_mode: mtp
tensor_parallel needs ExLlamaV3 1.5.1 or later for
Qwen4ExpForConditionalGeneration (QSA layers are not split). On earlier
versions it fails with NotImplementedError and you must use autosplit.
Not tested on my side.
ngram_ram: true loads the n-gram table into system RAM instead of
streaming it from disk per forward.
Conversion
python convert.py \
-i Qwen3.8-Flash-Next-Uncensored \
-o Qwen3.8-Flash-Next-Uncensored-exl3-4bpw-ng8 \
-w /tmp/exl3-work \
-b 4.05 -hb 6 -ngb 8 -mb 8 -vb 16 \
-d 0,1,2,3 -v
Default calibration corpus, 250 rows x 2048 columns.
Measurements
Not run on this build yet. Speed figures for the 6 bpw-table build are on the 4.05 bpw card.
Quality
Not benchmarked. Abliteration figures are orcarouter's, measured on the BF16 source, not on this quant.
Safety
Safety alignment has been substantially removed via abliteration. It will comply with requests the original Qwen3.8-Flash-Next refuses, and has no meaningful built-in guardrails. Add your own moderation layer before any deployment.
License
Qwen Community License 1.0, inherited from Qwen/Qwen3.8-Flash-Next.