Qwen3.6 35B-A3B Distill (merged + GGUF)

Merged full model of the attention-LoRA adapter freakyskittle/qwen36-35b-a3b-distill-lora-l4 into huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated — the trainable safetensors equivalent of the GGUF-only HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive.

• Safetensors (root): text-only Qwen3_5MoeForCausalLM, bf16 — vision tower stripped, MTP block kept at mtp.* (ignored by the HF ForCausalLM, folded into every GGUF below as a nextn layer).
• GGUF: this repo's gguf/ folder has every quant; the dedicated freakyskittle/qwen36-35b-a3b-distill-GGUF repo mirrors them at root level so llama-cli -hf, LM Studio and other tools detect them. Produced by llama.cpp convert_hf_to_gguf.py + llama-quantize. The mtp-*.gguf file is the standalone MTP draft for speculative decoding.

GGUF files

All main-model GGUFs embed the MTP (nextn, 1 layer) head — llama-cli --speculative-style MTP draft decoding is usable out of the box. A standalone mtp-*-f16.gguf draft export is included for DFlash/DFlash2-style speculative workflows.

Training

Trained with an Unsloth pipeline (fork, examples/distillation_sft/) on a Colab L4 (22 GiB, native bf16):

• attention-only LoRA r=32, alpha=64 on q/k/v/o_proj (10 full-attention layers; 6.9M params)
• 1 epoch, seq 2048, packing, effective batch 16, adamw_8bit, lr 2e-4
• 12,502 rows -> 3,001 packed examples, 188 steps, train_loss 0.81

Data mix (~69% code/code-adjacent): private freakyskittle/glm-5.3-distillation (reasoning/tool-use traces), Magicoder-OSS-75K, CodeFeedback, StarCoder2 self-oss-instruct, Codeforces R1 CoTs, SWE-rebench agent traces, security CoTs (OpenVul, SecCoderX, cybench/autopenbench), CTF-solver, plus benchmark training splits (GSM8K, MATH, ARC, MMLU auxiliary_train).

Sanity benchmarks (small samples, adapter eval)

Small-sample eval for sanity checking only — not a rigorous benchmark run.

Usage

# llama.cpp
llama-cli -m gguf/Qwen36-35B-A3B-Distill-Q4_K_M.gguf -cnv

# transformers (text-only merged weights)
from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained("<this repo>", torch_dtype="bfloat16")

Requires a recent transformers (qwen3_5_moe support) / llama.cpp (qwen35moe arch). UD/imatrix quants were not produced on the CPU conversion box — standard K/IQ/TQ quants are provided instead.

Merge: W' = W + (alpha/r) * B @ A applied on disk, shard by shard.

Speculative decoding

• MTP: every GGUF embeds the MTP nextn head; mtp-Merged-F16.gguf is the standalone draft.
• DFlash2: freakyskittle/qwen36-35b-a3b-distill-dflash2 — a trained 432M block-diffusion draft (top-16 containment 53.5%), GGUF at Qwen36-35B-A3B-Distill-DFlash2-F16.gguf in the -GGUF repo.