leonsarmiento/Sage-A1.5-6bit-XL-MTP-mlx

⚡ oMLX REQUIRED: These quants are designed to be used with oMLX and they will not work on LM studio. The bundled MTP (Lightning MTP) speculative-decoding heads only execute on oMLX — LM Studio loads the base model but ignores the MTP heads, so the acceleration is lost. For MTP speed, serve this model with oMLX.

This model was converted to MLX format from InternScience/Sage-A1.5 using BaseQuant_XL 6/8-bit mixed quantization, with the model's own native MTP (Multi-Token Prediction) head grafted in from the same source repo — Sage-A1.5 ships a dedicated model-mtp.safetensors shard (785 mtp.* tensors), so no donor model and no cross-model transplant were needed. The base weights are bit-identical to the clean XL conversion; the graft adds 36 language_model.mtp.* tensors (+0.71 GB).

Sage-A1.5 (Shanghai AI Laboratory / InternScience) is a 35B MoE agentic model built on Qwen3.6 + Agents-A1, targeting long-horizon scientific problem solving through adaptive capability composition: agentic orchestration (parallel subagents with isolated contexts), verification mode (mid-trajectory self-review inside the model's own chain of thought), and open scientific discovery. Source-reported results include GAIA 95.1, BrowseComp 81.7, IFBench 81.6, SWE-bench Pro 61.2, and FrontierScience-Olympiad 81.0 — first among ~35B models on most agentic axes.

Measured Speed on oMLX (M4 Pro 48GB, Oct 2026)

Acceptance lands below the vanilla Qwen3.6 graft (92.2%) and the Huihui build (89.1%) — expected, since Sage-A1.5's backbone drifted further from the drafter's training distribution through its agentic fine-tuning. It is still far above the ~64% threshold where MoE bundled-MTP stops paying (tok/cycle 2.31 comfortably clears the ~1.66× MoE verification cost).

Acceptance is content-dependent (full three-regime analysis on the Qwen3.6 sibling): templated reasoning drafts highest, long-form prose lower, synthetic benchmark continuation lowest. Rule of thumb: prefill/TTFT-bound workloads → vanilla build (or mtp_enabled: false). Decode-heavy workloads (long agentic runs, big generations) → this build.

Native MTP Graft — Construction Details

mlx_vlm's qwen3_5_moe sanitizer drops mtp.* keys during conversion (by design), which is why the clean XL carries none. Sage stores its MTP experts individually (mtp.layers.0.mlp.experts.{0..255}.{gate,up,down}_proj.weight) — unlike Qwen's fused (256,1024,2048) layout — so this graft stacks them in expert-index order into the switch_mlp runtime format (the same transform the sanitizer applies to the main layers), then requantizes to match the XL recipe:

Offline verification (9 checks, all PASS): 36/36 expected keys, format: mlx metadata, norms exactly round_bf16(src + 1.0), bf16 passthroughs bit-identical to source, dequant rel-L1 vs source: attention 0.0072–0.0082 (8-bit), stacked experts 0.0239–0.0244 (6-bit, full 256-expert tensors; per-expert spot checks e0/e127/e255 uniform — ordering proven), config/index parity, base weights untouched.

Coherence with MTP engaged: 15×37 = 555 ✓ (clean thinking trace), iterative Fibonacci function with docstring ✓.

A note on thinking length

Sage-A1.5 is a deep-thinking agentic model: in thinking mode it routinely spends thousands of tokens on verification-heavy reasoning before answering (a 12K-char thinking trace on a simple coding prompt is normal). Give it generous max_tokens budgets in thinking mode, or switch to instruct mode (enable_thinking: false) for concise direct answers — both modes verified working on this build.

About XL Quantization

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.

Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.

oMLX Setup

1. mtp_enabled: true — fresh oMLX model entries default to false and will silently run at vanilla speed. Set via Admin → model settings, then reload.
2. turboquant_kv_enabled: false — TurboQuant KV cache eliminates the MTP advantage. Keep it off for MTP speed.
3. Verify engagement in the server log: MTP path activated (model has mtp_forward) per request, and an MTP[..] finish= line with acceptance stats.
4. mtp_enabled is a live toggle: false + reload returns this model to exact vanilla speed (verified ~58.5 tok/s, no MTP log line).

Thinking-mode guidance

MTP boosts throughput, but reasoning models control their own output length — faster tokens can mean longer thinking, not faster answers, and the 4-bit family showed a −7pp MATHQA accuracy cost with MTP engaged. For deep thinking sessions, set mtp_enabled: false. For instruct/agentic/coding workloads, leave it on.

BaseQuant_XL Quantization Strategy

• Quantization type: BaseQuant_XL mixed (multimodal, vision preserved) + native MTP graft
• Bits per weight: 6.808 (base) + 0.71 GB MTP
• Total size: 30.60 GB (6 shards)
• Group size: 64
• Method: Custom quant_predicate via mlx_vlm with individual-expert stacking sanitize, additive graft (base untouched)

Recommended Inference Parameters - Add to Jinja template on LM studio or Chat Template Kwargs on oMLX

Thinking Preserve ({%- set preserve_thinking = true %}):

• General tasks: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
• Coding tasks: temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

Instruct Mode ({%- set enable_thinking = false -%}):

• General tasks: temperature=0.7, top_p=0.8, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
• Reasoning tasks: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Use with mlx

pip install -U mlx-vlm

python -m mlx_vlm.generate --model leonsarmiento/Sage-A1.5-6bit-XL-MTP-mlx --max-tokens 256 --temperature 0.6 --top-p 0.95 --prompt "Hello"

⚠️ mlx_vlm runs the backbone only — MTP acceleration requires serving with oMLX.

Family

• Agents-A1 6bit-XL — Sage-A1.5's direct predecessor (InternScience)
• Qwen3.6-35B-A3B-6bit-XL-MTP-mlx — the vanilla Qwen3.6 MTP build (full three-regime acceptance analysis)
• Qwen3.6-35B-A3B collection