Qwen3.8-Flash-Next-Uncensored — EXL3 3.05 bpw

EXL3 quantization of orcarouter/Qwen3.8-Flash-Next-Uncensored, an abliterated (refusal-removed) build of Qwen/Qwen3.8-Flash-Next.


Size on disk	68 GB
Language model	3.05 bpw
`lm_head`	5 bpw
MTP layers	3 bpw
Vision tower	5 bpw
n-gram (PLE) tables	3 bpw

Note on size: turboderp's official 3.05bpw_h5_ng5 is ~101 GB because its n-gram tables are at 5 bpw. This build leaves --ngram_bits at its default (= --bits rounded, so 3), which is where the ~33 GB difference comes from. The n-gram tables are 51B of the model's 176B parameters, so expect some quality difference. Not measured.

Also available: 4.05 bpw (102 GB).

Config

model:
  model_name: Qwen3.8-Flash-Next-Uncensored-exl3-3bpw
  cache_size: 32768
  cache_mode: FP16
  ngram_ram: true
  tensor_parallel: false  # true needs ExLlamaV3 1.5.1+

draft_model:
  draft_mode: mtp

tensor_parallel needs ExLlamaV3 1.5.1 or later for Qwen4ExpForConditionalGeneration (QSA layers are not split). On earlier versions it fails with NotImplementedError: Tensor-parallel is not currently implemented for Qwen4ExpForConditionalGeneration and you must use autosplit. Not tested on my side.

Cache quantization needs ExLlamaV3 ≥ 1.4.8. Below that, cache_mode other than FP16 is rejected with QSA attention currently supports only the fp16 cache layer.

ngram_ram: true loads the n-gram table into system RAM instead of streaming it from disk per forward.

MTP is preserved; TabbyAPI logs Using main model MTP component for drafting on load.

Conversion

python convert.py \
  -i Qwen3.8-Flash-Next-Uncensored \
  -o Qwen3.8-Flash-Next-Uncensored-exl3-3bpw \
  -w /tmp/exl3-work \
  -b 3.05 -hb 5 -mb 3 -vb 5 \
  -d 0,1,2,3 -v

--ngram_bits left at default (3). Pass -ngb 5 to match turboderp's official quant.

Default calibration corpus, 250 rows x 2048 columns.

Measurements

4x RTX 4000 Ada (20 GiB, sm_89), PCIe, no NVLink, 2x Xeon Gold 5318Y. ExLlamaV3 1.5.0 + TabbyAPI, autosplit, MTP on, FP16 cache, 32k context. AIPerf, streaming, ignore_eos, seed 42, with warmup.

ISL 1000 / OSL 500 / concurrency 4 / 20 requests:

Metric	value
Request latency	27,219 ms
Inter-token latency	49.7 ms
Time to first token	2,406 ms
Output throughput	70 tok/s
Active prefill	636 tok/s

Single stream via chat.py, code prompt: 78 tok/s generation, 436 tok/s prefill, 67% MTP acceptance.

20 requests per run, so gaps under ~5% are noise.

Quality

Not benchmarked at 3.05 bpw. Abliteration figures are orcarouter's, measured on the BF16 source, not on this quant.

Safety

Safety alignment has been substantially removed via abliteration. It will comply with requests the original Qwen3.8-Flash-Next refuses, and has no meaningful built-in guardrails. Add your own moderation layer before any deployment.

License

Qwen Community License 1.0, inherited from Qwen/Qwen3.8-Flash-Next.

Lygodactylus/Qwen3.8-Flash-Next-Uncensored-exl3-3bpw