Qwen3.6 35B-A3B Distill — GGUF

GGUF builds of freakyskittle/qwen36-35b-a3b-distill (LoRA-merged huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated, trained on a code-heavy distillation mix — see the merged repo card for training details and benchmarks).

Every main-model file embeds the MTP (nextn) head — llama.cpp speculative decoding works out of the box. mtp-Merged-F16.gguf is the standalone MTP draft export for custom speculative pipelines.

Qwen36-35B-A3B-Distill-DFlash2-F16.gguf is a separately trained DFlash2 block-diffusion draft model (safetensors + card at freakyskittle/qwen36-35b-a3b-distill-dflash2). It drafts 8-token blocks the target verifies in parallel — held-out top-16 containment 53.5%. Needs a DFlash2-aware backend (llama.cpp PR #27342, or the SGLang/vLLM patch sets from the dflash repo).

Files

Usage

# llama.cpp (pulls straight from HF)
llama-cli -hf freakyskittle/qwen36-35b-a3b-distill-GGUF:Q4_K_M

# or download a specific file
huggingface-cli download freakyskittle/qwen36-35b-a3b-distill-GGUF     Qwen36-35B-A3B-Distill-Q4_K_M.gguf

Requires a recent llama.cpp with qwen35moe support (hybrid gated-deltanet + full-attention MoE, 256 experts, ~3B active params).