Hugging Face's download widget does not recognize _KT GGUFs. Download links are in the table below.

Qwen3.8-Flash-Next-Uncensored, ik_llama.cpp IQ4_KT and IQ3_KT GGUFs

Text-only GGUFs of orcarouter/Qwen3.8-Flash-Next-Uncensored, an abliterated build of Qwen3.8-Flash-Next, plus an IQ4_KT quant of that model's own MTP draft head.

The v4 IQ4_KT and the v4 IQ3_KT are the main files. The v4 IQ3_KT replaces the IQ3_KT_v2. Against the BF16, the mean KL divergence of the v4 IQ4_KT is 58% lower than that of the v3 IQ4_KT on English text and 55% lower on code. For the v4 IQ3_KT against the IQ3_KT_v2, it is 42% lower on both (see Validation).

The v4 IQ4_KT keeps the routed experts and the PLE table of the v3 IQ4_KT and is 2.0 GiB larger. The v4 IQ3_KT is the same size as the IQ3_KT_v2. The MTP head is unchanged.

The v3 IQ4_KT is the file published on 2026-09-30 as the IQ4_KT_v3. Only the file names changed, with the version now before the quant type.

Both v4 files were quantized with ik_llama.cpp at d9e2868, which includes the CUDA IQ4_KT and IQ3_KT encoder of PR #2572. The v4 IQ3_KT declares IQ3_KT as its file type. The IQ3_KT_v2 declared IQ4_KT.

The source model has its safety alignment removed, and quantization does not restore it. Read the source card's disclaimer before use.

These files require ik_llama.cpp at cb860a9 or later, the merge of PR #2481. Earlier builds read the IQ4_KT tails in these files incorrectly.

Choosing a file

The v4 files store the attention, SSM, hyper-connection, and shared-expert tensors at Q8_0 and the DSA indexer projections at BF16. In the v3 IQ4_KT, most of these tensors use IQ4_KT. The v4 recipe lowers the KL divergence and keeps about 2 GiB more on the GPU. The v4 IQ3_KT needs less RAM for the routed experts than the v4 IQ4_KT, but the same VRAM.

The KV cache and the compute buffer grow with the context length (-c), and the compute buffer also grows with the micro-batch size (-ub). At 256K context and -ub 1024, with the cache types of the example invocation, the two take about 7.5 GiB, more than the difference between the files. Each way to use less VRAM has a cost:

• A lower -c gives less context.
• A lower -ub makes prompt processing slower.
• The v3 IQ4_KT, the faster file, has a higher KL divergence.
• Moving more tensors to the CPU with -ot makes decoding slower.

Files

Download both shards of one quant into the same directory and pass the first shard to llama-server.

The routed expert ffn_down_exps tensors use IQ3_KT. The routed expert gate/up tensors use IQ4_KT in the v4 IQ4_KT. In the v4 IQ3_KT they use IQ4_KT in blocks 0 to 5 and 38 to 47 and IQ3_KT in blocks 6 to 37. The PLE table uses IQ4_KT in the v4 IQ4_KT and IQ3_KT in the v4 IQ3_KT. The DSA indexer projections use BF16 and the four-column PLE convolution F16. Every other two-dimensional weight tensor uses Q8_0, including the token embeddings, the output head, the attention projections, the hyper-connection tensors, and the shared experts.

In the v3 IQ4_KT, the routed expert gate/up tensors, the PLE table, and most other two-dimensional weight tensors use IQ4_KT. ffn_down_exps and ffn_down_shexp use IQ3_KT. attn_k, attn_v, and the token embeddings use Q8_0, the output head Q6_K, and the four-column PLE convolution F16.

MTP draft head

The head is optional. Load it beside the main model with -md and enable it with --spec-type mtp:n_max=1,p_min=0.0. If the head does not fit beside the main model, add -draft "-ub 256 -b 256". It gives the head a separate batch size of 256, which makes its compute buffer smaller. It is the head published with the v3 IQ4_KT. With the v4 IQ4_KT, llama-spec-bench accepted 95.65%, 82.01%, and 72.79% of its drafts on the built-in code, extract, and story tasks (temperature 0, 256 tokens).

Runtime

Example invocation:

GGML_CUDA_NO_PINNED=1 llama-server \
   -m Qwen3.8-Flash-Next-Uncensored-ik_llama-v4-IQ4_KT-00001-of-00002.gguf \
   --defer-ple -ngl 49 -ot 'ffn_.*_exps=CPU' \
   -fa on -ctk q8_0 -ctv q8_0 -ictk q8_0 --jinja \
   -c 196608 -b 2048 -ub 1024 --ctx-checkpoints 16

--defer-ple leaves the PLE table on disk: 25.0 GiB in the IQ4_KT files, 20.3 GiB in the v4 IQ3_KT. The routed experts are read every token and need RAM or VRAM (see Choosing a file). With -ngl 49, all other tensors except the token embeddings are on the GPU. Metal, Vulkan, ARM, and multi-GPU execution are not validated.

For multi-turn chat, use 391a95f (PR #2524) or later, or pass "preserve_thinking": true in chat_template_kwargs. Builds from PR #2382 up to PR #2524 force preserve_thinking off unless the request sets it. With it off, whether forced or set to false by the client, the chat template drops the reasoning of earlier turns, and each new user message makes the server process the previous exchange again.

Validation

KL divergence against the BF16 source

The reference is a BF16 GGUF converted from the same uncensored weights. The test texts, chunking, and flags are the ones used for the base-model release. Each text is ten chunks of 2048 tokens, one of held-out English and one of CPython and PostgreSQL source.

Reference perplexities were 3.3289 for English and 1.7258 for code. The v4 IQ4_KT measured 3.3839 and 1.7325, and the v4 IQ3_KT 3.4706 and 1.7586. The v3 IQ4_KT measured 3.5366 and 1.7909, and the IQ3_KT_v2 3.5802 and 1.8092. Same top-1 is the share of positions where the quantized model's most likely token matches the reference's.

The DSA indexer of this model selects 2048 tokens, the length of one chunk. These runs therefore do not exercise its selection, and the BF16 type of the indexer projections has no measurement behind it.

Calibration

Both importance matrices were computed on the base Qwen3.8-Flash-Next, not on the abliterated weights. Both main-model files used qwen4exp-agentic-v3.imatrix and the head used qwen4exp-mtp-v1-draft.imatrix. Both files and a description of the corpus are in the base-model repository.

Conversion notes

Source: orcarouter/Qwen3.8-Flash-Next-Uncensored at revision 8336e613ea508b13c2159bd0f68965d97a606b95.

Each BF16 conversion used convert_hf_to_gguf.py from llama.cpp at f8dbcd6. The head was converted with the same script and a local patch that exports the mtp.* tensors, which the stock converter omits. The v4 files were quantized with llama-quantize from ik_llama.cpp at d9e2868 with --cuda-quantize. The v3 IQ4_KT and the head were quantized with an ik_llama.cpp branch on 32cddbf that adds the CUDA encoder.

Rules shared by both v4 files, the start of --custom-q:

indexer\.=bf16,hc_=q8_0,ssm_alpha=q8_0,ssm_beta=q8_0,attn_qkv=q8_0,attn_gate=q8_0,ssm_out=q8_0,attn_q\.weight=q8_0,attn_output=q8_0,attn_k\.weight=q8_0,attn_v\.weight=q8_0,_shexp=q8_0,ple_key=q8_0,ple_value=q8_0,eh_proj=q8_0,ple_conv1d=f16

v4 IQ4_KT options:

--imatrix qwen4exp-agentic-v3.imatrix
--token-embedding-type q8_0
--output-tensor-type q8_0
--custom-q "<shared rules>,ffn_down_exps=iq3_kt,ffn_(gate|up)_exps=iq4_kt,per_layer_token_embd=iq4_kt"
iq4_kt

v4 IQ3_KT options:

--imatrix qwen4exp-agentic-v3.imatrix
--token-embedding-type q8_0
--output-tensor-type q8_0
--custom-q "<shared rules>,ffn_down_exps=iq3_kt,blk\.([0-5]|3[89]|4[0-7])\.ffn_(gate|up)_exps=iq4_kt,ffn_(gate|up)_exps=iq3_kt,per_layer_token_embd=iq3_kt"
iq3_kt

v3 IQ4_KT options:

--imatrix qwen4exp-agentic-v3.imatrix
--token-embedding-type q8_0
--custom-q "ffn_down_exps=iq3_kt,ffn_down_shexp=iq3_kt"
iq4_kt

Head options: --imatrix qwen4exp-mtp-v1-draft.imatrix, iq4_kt.

Tensor counts in the v4 IQ4_KT are 666 Q8_0, 97 IQ4_KT, 48 IQ3_KT, 24 BF16, 388 F32, and one F16. Its routed experts and PLE table are identical to those of the v3 IQ4_KT. The v4 IQ3_KT has 666 Q8_0, 113 IQ3_KT, 32 IQ4_KT, 24 BF16, 388 F32, and one F16. The v3 IQ4_KT has 713 IQ4_KT, 96 IQ3_KT, 25 Q8_0, one Q6_K, 388 F32, and one F16.

SHA-256 of the files as uploaded:

d555cebf6ac9fa8e47d18c6af6d7c9d9136f9b96d89bb37b9fa855e740ae2c75  Qwen3.8-Flash-Next-Uncensored-ik_llama-v4-IQ4_KT-00001-of-00002.gguf
dd1c36d9fe023553cd275b101da00f14427220eb23e56252d4ee057845dabe54  Qwen3.8-Flash-Next-Uncensored-ik_llama-v4-IQ4_KT-00002-of-00002.gguf
60a3b982267f316e1d3f88b1194afe66feef2c460a7890473251be763ca25b21  Qwen3.8-Flash-Next-Uncensored-ik_llama-v4-IQ3_KT-00001-of-00002.gguf
eed2a8880920f6b7f2311278198251f77b54e019a305d5ce5050cb05f01b754b  Qwen3.8-Flash-Next-Uncensored-ik_llama-v4-IQ3_KT-00002-of-00002.gguf
61c7292ce699ae360ed115c1caf750a29cd24ba21087fc44ce854de16acffcf8  Qwen3.8-Flash-Next-Uncensored-ik_llama-v3-IQ4_KT-00001-of-00002.gguf
dda38e14dd644d12cb110380ee72d71892c692af1d90291fbbe8e5acf1cedd6b  Qwen3.8-Flash-Next-Uncensored-ik_llama-v3-IQ4_KT-00002-of-00002.gguf
bdfef927fd810d927d3ef1a8e89cd9ed937ef1239907b18dd2b836a0dbacb9a4  Qwen3.8-Flash-Next-Uncensored-MTP-ik_llama-IQ4_KT.gguf

License

Qwen Community License 1.0, the license of Qwen3.8-Flash-Next. The source repository ships the same LICENSE file, and this repository includes a copy.