Qwen3.6-35B-A3B · lododo-Distill-v1.0 (GGUF)
A distilled fine-tune of the Qwen3.6-35B-A3B base (MoE, 3B active), trained on data generated by Qwen3.8-27B as the teacher — bringing the base model to (and beyond) the teacher's capability while keeping the MoE speed advantage.
Put simply: 27B-level capability at 35B-A3B speed.
| Metric | This model | Teacher: Qwen3.8-27B | Base: Qwen3.6-35B-A3B (original) |
|---|---|---|---|
| LCB code (AtCoder 112) | 69.6% | 52.7% (+16.9) | 50.9% (+18.7) |
| BFCL tool calling (100) | 67.0% | 53.0% (+14.0) | 62.0% (+5.0) |
| GSM8K math (500) | 96.0% | 95.2% (+0.8) | 94.4% (+1.6) |
| GPQA science (198) | 53.5% | 40.9% (+12.6) | 43.4% (+10.1) |
| IFEval instruction (150) | 33.3% | 36.7% (−3.4) | 24.0% (+9.3) |
| JSON Schema (50) | 100% | 100% | 100% |
| Chinese writing (chars) | 140 | 159 | 150 |
- 🔗 LoRA adapter (40 MB, mount on the base, saves 22 GB) — lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF-lora
- ⚠️ Community work, not affiliated with or endorsed by Qwen. Base model: Qwen3.6-35B-A3B (Apache-2.0).
🧰 Running this model on a small GPU? Use moefast
moefast is a small launcher that configures llama.cpp for your machine. It reads the model's GGUF
metadata, measures your free VRAM, computes the expert-cache size, context and batch parameters,
writes them into an editable config file next to the model, and starts llama-server.
The expert cache itself — keeping expert weights in host RAM and caching the hot ones in VRAM — is
llama.cpp's own (--moe-cache-mib, upstream PRs #29887/#30112). moefast makes sure it is
enabled, correctly sized, and remembered per model — the cache defaults to off in llama.cpp,
which roughly halves throughput.
git clone https://github.com/conlododo/moefast && cd moefast
./install.sh
./bin/moefast run /path/to/this-model.gguf
No patch, no fork — stock llama.cpp only. Repo: https://github.com/conlododo/moefast
Files
| File | Size | Notes |
|---|---|---|
Qwen3.6-35B-A3B-lododo-Distill-v1.0-UD-Q4_K_XL.gguf |
22.85 GB | Recommended: merged model (includes the MTP head), single file |
Quick start
llama-server -m Qwen3.6-35B-A3B-lododo-Distill-v1.0-UD-Q4_K_XL.gguf \
-ngl 99 --jinja -c 32768 --port 8080
Sampling parameters (only relevant for MTP / Strata)
The model has no special sampling requirement — set temperature / top_p as your task needs.
Only when you enable MTP speculative decoding in an engine that injects model-card sampling defaults (e.g. Strata) must the request carry:
{"temperature": 0, "presence_penalty": 0, "frequency_penalty": 0, "repeat_penalty": 1.0}
Why: Strata injects temperature=1.0, top_p=0.95, top_k=20, presence_penalty=1.5 for qwen35moe,
and its MTP gate requires temperature==0 && repeat==1 && freq==0 && present==0 — the defaults shut the
drafter off entirely (drafts accepted 0 of 0 in our tests).
llama.cpp's own --spec-type draft-mtp is not affected.
Evaluation
Settings: thinking disabled, Q4 quant, identical llama.cpp flags, LCB reported on the AtCoder 112-question subset (LeetCode 63 listed separately).
| Metric | 35B original | 27B original | empero | This model |
|---|---|---|---|---|
| LCB AtCoder (112) | 50.9% | 52.7% | 50.0% | 69.6% (78/112) |
| GPQA Diamond (198) | 43.4% | 40.9% | 59.6% | 53.5% (106/198) |
| IFEval strict (150) | 24.0% | 36.7% | 23.3% | 33.3% |
| BFCL tool calling (100) | 62.0% | 53.0% | 59.0% | 67.0% (67/100) |
| GSM8K (500) | 94.4% | 95.2% | 96.4% | 96.0% (480/500) |
| JSON Schema (50) | 100% | 100% | 100% | 100% (50/50) |
| Chinese writing (chars) | 150 | 159 | 136 | 140 |
| LeetCode (63, supplementary) | 6.3% | 6.3% | 9.5% | 7.9% |
Versus the original 35B base: better on 7/7 metrics (LCB +18.7, GPQA +10.1, IFEval +9.3, BFCL +5.0, GSM8K +1.6).
Three deployment forms (same weights)
| Form | Size | LCB AtCoder |
|---|---|---|
| Merged (this repo) | 22.85 GB | 69.6% |
| LoRA f16 mount | 40 MB | 67.9% |
| LoRA f32 mount | 79 MB | 66.1% |
| LoRA q8_0 mount (not recommended) | 31 MB | 62.5% |
A LoRA delta is a small-magnitude tensor; q8_0 block quantization amplifies its relative error. If you want a small file, use f16.
Training summary
| Item | Value |
|---|---|
| Teacher → base | Distillation data from Qwen3.8-27B → Qwen3.6-35B-A3B |
| Data | 48.73M tokens (~52k samples) |
| Method | QLoRA r=16 / α=32 (attention layers + shared experts) |
| Training | 0.5 epoch (747 steps), single uninterrupted run |
| Stability | Equal-weight average of two independent runs (reduces random-init variance) |
Reproducibility (public benchmarks)
| Metric | Dataset | Link |
|---|---|---|
| LCB | LiveCodeBench code_generation_lite (release_v6) |
https://huggingface.co/datasets/livecodebench/code_generation_lite |
| GPQA | GPQA Diamond | https://huggingface.co/datasets/Idavidrein/gpqa |
| IFEval | Google IFEval (lm-eval v0.4.13, limit 150) | https://huggingface.co/datasets/google/IFEval |
| BFCL | Berkeley Function Calling Leaderboard | https://huggingface.co/datasets/gorilla-llm/Berkeley-Function-Calling-Leaderboard |
| GSM8K | OpenAI GSM8K (first 500 of test) | https://huggingface.co/datasets/openai/gsm8k |
| JSON Schema | In-house probe (50 items, Draft-7 validation) | not a public benchmark |
| Chinese writing style | In-house probe (31 prompts) | not a public benchmark |
Determinism: --workers 1 + temperature 0; repeated runs on the same weights match item by item.
Known limitations
| Item | Notes |
|---|---|
| LeetCode format | 7.9% (training data is AtCoder stdin/stdout style; weak at class Solution style) |
| IFEval | 33.3%, 3.4pp below the 27B teacher (mixing instruction data cost 11-14pp of coding ability in our experiments, so it was left out) |
| Long context | Supports up to the base model's native 256K context; the usable length depends on available VRAM (KV cache) |
| Languages | Strong in English and Chinese; other languages not systematically evaluated |
License & acknowledgements
- Released under Apache-2.0, inherited from Qwen/Qwen3.6-35B-A3B.
- The base GGUF quantization comes from unsloth/Qwen3.6-35B-A3B-MTP-GGUF (Apache-2.0); this merged model was produced by tensor-level merging into that UD-Q4_K_XL file.
- Changes: supervised fine-tuning (distillation data) and weight averaging relative to the base, then tensor-level merging into GGUF.
- Not affiliated with or endorsed by the Qwen team; "Qwen" is used only to describe the base model.
- Use at your own risk; no warranty of any kind.