Qwen3.6-35B-A3B-uncensored-heretic — QSA GGUF

QSA-grafted build of llmfan46/Qwen3.6-35B-A3B-uncensored-heretic, made from the existing Qwen3.6-35B-A3B-uncensored-heretic-No-MTP-custom.gguf BPW4.75 base.

Four files are published:

The original backbone is preserved byte-for-byte: all 733 base tensors keep their exact payload and offset. Only these are added to make the QSA files:

• 50 F32 indexer tensors (5 per full-attention layer): blk.{N}.indexer.q_proj.weight, k_proj.weight, q_norm.weight, k_norm.pre_weight, k_norm.weight
• metadata:
  • qwen35moe.attention.indexer.head_count = 4
  • qwen35moe.attention.indexer.key_length = 128
  • qwen35moe.attention.indexer.top_k = 2048
  • qwen35moe.attention.compress_ratios = 4 on layers [3, 7, 11, 15, 19, 23, 27, 31, 35, 39], else 0

mm_64k graft (base of the MTP file)

mm_64k + MTP graft (recommended file)

This is the same QSA mm_64k model as the No-MTP file, with the donor's complete MTP draft block grafted on as layer 40. The QSA backbone and all 50 indexer tensors are byte-for-byte identical to the No-MTP mm_64k file; only the 20 MTP tensors and the three metadata fields above are added. The MTP layer uses dense attention (compress_ratio = 0) and is only used by speculative decoding, not by the main model's attention path.

In current llama.cpp builds this MTP head works as a self-speculative draft: a small prompt-processing cost for a large token-generation gain. It is optional; the model runs normally, with identical output, when MTP is not enabled.

llama.cpp setup

This model needs the Qwen3.5/3.6-MoE QSA k-pool indexer support in llama.cpp.

Use the qwen35moe-qsa branch in tvall43/llama.cpp:

git clone https://github.com/tvall43/llama.cpp tvall43-llama.cpp
cd tvall43-llama.cpp
git checkout qwen35moe-qsa
cmake -B build -DGGML_VULKAN=ON
cmake --build build --config Release -j

Run the No-MTP file like this:

export LLAMA_QSA_TOP_K=4096
./build/bin/llama-server \
  -m Qwen3.6-35B-A3B-uncensored-heretic-QSA-mm64k-No-MTP-custom.gguf \
  --ctx-size 125536 \
  --host 0.0.0.0 --port 8002

For the MTP file, add the speculative-decoding flags:

export LLAMA_QSA_TOP_K=4096
./build/bin/llama-server \
  -m Qwen3.6-35B-A3B-uncensored-heretic-QSA-mm64k-MTP-custom.gguf \
  --ctx-size 125536 \
  --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0 \
  -fa on \
  --host 0.0.0.0 --port 8002

--spec-type draft-mtp needs a recent llama.cpp build. If your build does not recognise the flag, update the fork or use the No-MTP file.

LLAMA_QSA_TOP_K is the main thing to tune. The GGUF metadata ships the trained budget (2048), which is right for voice/chat-length prompts; for long documents do not just accept the default:

For long prompts, keep the context window large enough that the prompt actually fits in one pass; the QSA path only kicks in above the indexer top_k.

By default, the GGUF already carries the indexer metadata, so there is nothing else to enable on the model side. If you are comfortable with more knobs, the fuller branch at qsa-rebase also has LLAMA_QSA_DENSE_BELOW_K and LLAMA_QSA_PREFILL_ONLY for A/B testing, but the linked branch above is the minimal support needed to run this model.

Runtime status

This is a structurally valid GGUF: header, tensor table, alignment, metadata and payloads were verified after grafting. The reference QSA runtime (kuraneko1/Qwen3.8-27B-QSA-llama.cpp) originally implemented QSA only for the dense qwen35 architecture. A qwen35moe port now adds the indexer tensors, the F32 indexer cache, the hybrid indexer memory and the block-sparse attention branch.

WikiText-2 probe on an H200 (CUDA 12.9, flash attention, Q4 base):

Same probe on the mm_64k checkpoint (dense PPL is bit-identical; the backbone is unchanged):

The mm_64k checkpoint dominates text_64k at every measured K (at 64k/K=2048: 5.5789 vs 5.7790; at 131k/K=2048: 5.9441 vs 6.6043), so it is a drop-in replacement for text workloads too. 2048-token budget stays sub-1% at <= ~48k and closes in around the same K ~= context / 8 curve above that.

The metadata keeps the trained budget of 2048 tokens, which is 12.5% of the 16k training length. At longer contexts the attention mass is spread over more blocks, so a fixed 2048 is too small. The indexer ranking itself is fine: the reference dense-mask path and the block-sparse kernel agree at 64k (5.7859 vs 5.7790). For long contexts, set LLAMA_QSA_TOP_K to about context / 8 in the ported runtime.

For the Vulkan deployment (Mesa RADV / Radeon Pro V340 class, 8 GiB devices): the fork already carries the Vulkan flash_attn_sparse_compact shader and the topk_radix_qsa fusion. Split the 22 GB base across three or four devices with --split-mode layer, keep -fa on, and set LLAMA_QSA_TOP_K=$((CTX / 8)) for 32k-64k sessions. At voice/chat lengths the cache is below top_k, so the indexer branch is not entered and adds no cost.

Stock upstream llama.cpp still loads the model with dense attention and ignores the added indexer tensors.

Validation

• base tensors: sampled byte-for-byte equality against the source file
• new tensors: payload equality against mm1_16k/qsa_indexers.pt (including the +1 zero-centred RMSNorm conversion)
• tensor types: all 50 indexer tensors are F32
• offsets: 32-byte aligned, no overlap, all payloads inside the file
• GGUF re-parsed with the reference gguf-py reader