leonsarmiento/Qwen3.6-35B-A3B-6bit-XL-MTP-mlx

⚡ oMLX REQUIRED: These quants are designed to be used with oMLX and they will not work on LM studio. The bundled MTP (Lightning MTP) speculative-decoding heads only execute on oMLX — LM Studio loads the base model but ignores the MTP heads, so the acceleration is lost. For MTP speed, serve this model with oMLX.

This model was converted to MLX format from Qwen/Qwen3.6-35B-A3B using BaseQuant_XL 6/8-bit mixed quantization, with the native MTP (Multi-Token Prediction) heads grafted in from the same source bf16 repo — no donor model, no transplant. The base weights are byte-identical to leonsarmiento/Qwen3.6-35B-A3B-6bit-XL-mlx (the "honest king" of the Qwen3.6 family), plus 36 grafted language_model.mtp.* tensors (+0.68 GB).

Measured Speed on oMLX (M4 Pro 48GB, Oct 2026)

Draft acceptance is content-dependent — and acceptance decides everything. Three workloads, identical prompts, server-reported tok/s, A/B against the vanilla build:

How to read this: MoE verification costs ~1.66× a plain decode step (each draft position routes its own experts), so the yield (tok/cycle) must clear that bar. Templated reasoning text drafts at 90%+ → decisive win. Long-form prose settles in the low 80s → solid win. The oMLX benchmark's synthetic continuation drafts at only ~67% over a tiny sample → the bench reports parity-to-slightly-slower, plus its headline TTFT includes ~300–400 ms of draft-layer KV priming across the 8k prefill (pp 735.6 vs vanilla 762.6 tok/s; peak memory +2 GB).

Rule of thumb: prefill/TTFT-bound workloads → use the vanilla build. Decode-heavy workloads (long agentic runs, big generations, batch serving) → this build, with mtp_enabled: true and TurboQuant off.

Why This Build Works (when the last MoE MTP experiment didn't)

• Native heads, not transplanted. The 19 mtp.* tensors ship inside Qwen's own bf16 repo (shards 25–26). They were calibrated with the model during training — not borrowed from a different quant.
• 92.2% acceptance (306/332 drafts, depth d1=112/121, d2=105/111, d3=89/100) vs ~64% on the previous MoE graft attempt.
• Requantized to match the XL recipe — the MTP layer is quantized with the same role-based allocation as the main model (see table below), not carried at a mismatched bit depth.

About XL Quantization

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.

Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.

Native MTP Graft — Construction Details

mlx_vlm's qwen3_5_moe sanitizer drops mtp.* keys during conversion (by design), which is why the vanilla XL carries none. This build re-adds them in the oMLX-proven language_model.mtp.* layout:

Split-order verified numerically: reassembling cat([gate, up], axis=rows) reproduces the source fused tensor at rel-L1 0.0240, while the swapped order gives 1.4177 — unambiguous.

Offline verification (9 checks, all PASS): 36/36 expected keys, format: mlx metadata, norms exactly round_bf16(src + 1.0), bf16 passthroughs bit-identical to source, dequant rel-L1 vs source: attention 0.0072–0.0082 (8-bit), experts 0.0244 (6-bit, full-tensor comparison of all 256 experts), config/index parity, base weights untouched (spot-checked against the vanilla XL).

oMLX Setup

1. mtp_enabled: true — fresh oMLX model entries default to false and will silently run at vanilla speed. Set via Admin → model settings, then reload.
2. turboquant_kv_enabled: false — TurboQuant KV cache eliminates the MTP advantage. Keep it off for MTP speed.
3. Verify engagement in the server log: MTP path activated (model has mtp_forward) per request, and an MTP[..] finish= line with acceptance stats.
4. mtp_enabled is a live toggle on current oMLX builds: false + reload returns this model to exact vanilla speed (verified 57.7 tok/s, no MTP log line).

Thinking-mode guidance

MTP boosts throughput, but reasoning models control their own output length — faster tokens can mean longer thinking, not faster answers, and the 4-bit family showed a −7pp MATHQA accuracy cost with MTP engaged. For deep thinking sessions, set mtp_enabled: false. For instruct/agentic/coding workloads, leave it on and enjoy +30%.

Intelligence Benchmarks

The base weights are bit-identical to Qwen3.6-35B-A3B-6bit-XL-mlx, so its measured results carry over: 74.8% average instruct (n=50: MMLU 64, MMLU_PRO 64, ARC 90, HUMANEVAL 78, MBPP 78) — the best data-agnostic (non-calibrated) result in the family — and 30/30 verified on corrected MATHQA with the lowest reasoning waste of all variants tested (5,111 chars/correct). Full analysis on the vanilla model card.

This variant adds the draft layer only; it changes nothing about the backbone's intelligence — it changes how fast tokens arrive.

BaseQuant_XL Quantization Strategy

• Quantization type: BaseQuant_XL mixed (multimodal, vision preserved) + native MTP graft
• Bits per weight: 6.808 (base) + 0.68 GB MTP
• Total size: 30.6 GB (6 shards)
• Group size: 64

Recommended Inference Parameters - Add to Jinja template on LM studio or Chat Template Kwargs on oMLX

Thinking Preserve ({%- set preserve_thinking = true %}):

• General tasks: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
• Coding tasks: temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

Instruct Mode ({%- set enable_thinking = false -%}):

• General tasks: temperature=0.7, top_p=0.8, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
• Reasoning tasks: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Use with mlx

pip install -U mlx-vlm

python -m mlx_vlm.generate --model leonsarmiento/Qwen3.6-35B-A3B-6bit-XL-MTP-mlx --max-tokens 256 --temperature 0.6 --top-p 0.95 --prompt "Hello"

⚠️ mlx_vlm runs the backbone only — MTP acceleration requires serving with oMLX.

Family

• Qwen3.6-35B-A3B-6bit-XL-mlx — vanilla (LM Studio compatible)
• Qwen3.6-35B-A3B-oQ4e-mtp-XL-mlx — 4-bit data-aware experts + donor MTP (Frankenstein)
• Qwen3.6-35B-A3B collection