Qwen3.6-35B-A3B-Escha-W2-EXL3-MTP

IMPORTANT: This is an experimental quant, so far I had positive results on my testing but use it at your own discretion. Feedback is appreciated.

EschaLabs' Escha-W2 quality in EXL3 at 2.0 bpw, with the MTP head embedded — calibrated on EschaLabs' own KLD corpus.

This conversion packs the full model — decoder, head, and MTP block — into 11.2 GB of trellis-quantized safetensors, calibrated against the exact data EschaLabs used to certify their own 2-bit release.

• Small footprint: 11.2 GB at 2.0 bpw, complete model including MTP
• EschaLabs-calibrated: uses the exact 256×2048-token KLD corpus EschaLabs published with Escha-W2 (kld_corpus_v1, sha256-verified against their published provenance)
• Embedded MTP: the multi-token-prediction head is kept and quantized at 8 bpw — 3,113 MTP tensors audited present
• EXL3 2.0 bpw with 8-bit head (-hb 8) — trellis quantization, per-tensor bit budget from exllamav3's optimizer
• Quantized from: Qwen/Qwen3.6-35B-A3B (Apache-2.0, fresh bf16)
• Quality target: EschaLabs/Qwen3.6-35B-A3B-Escha-W2
• 256-expert MoE (top-8) · 2.0 bpw decoder · 8-bit head · MTP at 8 bpw · text-only
• Companion repo: Qwen3.6-35B-A3B-Escha-W2-GGUF-MTP — the same treatment for stock llama.cpp

Why this exists

The objective is the same as the GGUF companion: the smallest practical footprint following the EschaLabs method, with the MTP head embedded to add speculative-decoding value — here in the EXL3 format for exllamav3 and TabbyAPI users.

Three things make this conversion different from a generic 2-bit EXL3:

1. MTP head included. The base ships a multi-token-prediction head; this conversion keeps it, quantized at 8 bpw (-mb 8), instead of discarding it.
2. Calibrated on EschaLabs' own KLD corpus. The conversion used the exact 256×2048-token corpus EschaLabs published with their W2 quant (kld_corpus_v1, sha256-verified against their published provenance) — not a generic wiki/c4 mix.
3. EXL3 2.0 bpw with 8-bit head (-hb 8) — trellis quantization, per-tensor bit budget from exllamav3's optimizer.

The result is Escha-W2-targeted quality at 2.0 bpw, the MTP head intact, and a drop-in EXL3 distribution for exllamav3 / TabbyAPI.

Files

Usage (exllamav3 / TabbyAPI)

from exllamav3 import Config, Model, Cache, Tokenizer, Generator

config = Config.from_directory("<repo folder>")
tokenizer = Tokenizer.from_config(config)
model = Model.from_config(config)
cache = Cache(model, max_num_tokens=8192)
model.load()   # autosplits across available GPUs
gen = Generator(model, cache, tokenizer)
print(gen.generate(prompt="Hello", max_new_tokens=128))

Quality

Perplexity on the full wikitext-2 test set (llama.cpp perplexity windowing, 512-token windows, ~296k tokens) — same protocol as the companion GGUF repo's table, so rows are directly comparable:

Reading: at 2.0 bpw this lands between the GGUF IQ2_XXS and IQ3_XXS points on the size/quality curve — smaller than IQ3_XXS with quality approaching it, and both smaller and better-reading than IQ2_XXS.

Notes for honest interpretation:

• All rows were measured under the same windowing protocol as the companion GGUF repo, so the two tables can be read as one curve. PPLs from other toolchains or protocols are not comparable to these numbers.
• 2.0 bpw is a deep cut. The calibration keeps the conversion tracking the Escha-W2 target as closely as 2-bit storage allows, but it remains a 2-bit model — the table shows exactly where it lands.

Credit & license

• EschaLabs — the Escha-W2 2-bit calibration work and the published KLD corpus used as this conversion's calibration data. This repo depends on their open-sourced work.
• Qwen — the base model, released under Apache-2.0.
• Weights: Apache-2.0 (inherited).

Limitations

• Text-only: Escha-W2 is a text distribution (no vision tower). Base Qwen3.6 is multimodal; this distribution is not.
• 2.0 bpw is a deep cut: quality tracks the Escha-W2 target as closely as 2-bit storage allows (see table), but it is a 2-bit model.
• MTP speculative decoding depends on runtime support (TabbyAPI / exllamav3 speculative); the head is embedded and verified present (3,113 MTP tensors audited) regardless.