Qwen3.6-27B-MXFP4-MTP

This model extends mlx-community/Qwen3.6-27B-mxfp4 with native MTP (Multi-Token Prediction) heads stored in bfloat16 precision, enabling speculative decoding for ~1.5–2× faster generation.

What's different from the base model

The base mlx-community mxfp4 conversion strips the MTP decoder layers. This checkpoint re-injects them:

Component	Precision	Size
Main model (64 layers, vision encoder)	mxfp4 group_size=32	~15.0 GB
MTP decoder layer (×1)	bfloat16 (unquantized)	~0.81 GB

Total: ~15.8 GB.

MTP layer details

The MTP head consists of 1 decoder layer with self-attention, MLP, and projection:

Tensor	Shape	Size
`mtp.layers.0.self_attn.q_proj`	[12288, 5120]	120 MB
`mtp.layers.0.mlp.gate_proj`	[17408, 5120]	170 MB
`mtp.layers.0.mlp.up_proj`	[17408, 5120]	170 MB
`mtp.layers.0.mlp.down_proj`	[5120, 17408]	170 MB
`mtp.fc` (head projection)	[5120, 10240]	100 MB
Other (norms, o/k/v proj)	—	~80 MB
Total MTP		~810 MB

All MTP tensors are stored in bfloat16 — the overhead is small relative to the 15 GB main model, and keeping them unquantized preserves speculative decoding accuracy.

Model specs

Architecture: Qwen3.6-27B (Qwen3.5 multimodal)
Layers: 64 (16 full-attention, 48 linear-attention)
Context: 262,144 tokens
Vision: 27-layer ViT, image + video
MTP: 1 decoder layer, shared embeddings
Quantization: mxfp4 (OCP Microscaling FP4), group_size=32
KV cache: turboquant 4-bit

Use with MLX

pip install -U mlx-vlm

python -m mlx_vlm.generate \
  --model <hf-repo-id> \
  --max-tokens 100 \
  --temperature 0.0 \
  --prompt "Describe this image." \
  --image <path_to_image>

MTP speculative decoding

The MTP heads are already in the checkpoint — no separate drafter model or sidecar needed. Use with oMLX 0.39 (beta) which supports native Qwen3.6 MTP speculative decoding for ~1.5–2× faster generation.

Provenance

Base weights: mlx-community/Qwen3.6-27B-mxfp4
MTP tensors: extracted from Qwen/Qwen3.6-27B and merged in bf16

sleepyeldrazi/Qwen3.6-27B-MXFP4-MTP