Qwen3.5-9B MLX W4/g64 + Embedded W4 MTP
This is a full vision-language MLX conversion of
Qwen/Qwen3.5-9B with the model's
native one-layer MTP proposal head embedded in the same indexed safetensors
weight set under the mtp.* prefix.
- Language target and vision tower: byte-identical to
mlx-community/Qwen3.5-9B-MLX-4bit(W4/g64 affine, BF16 scales/biases). - MTP head: the 15 native
mtp.*tensors from the source checkpoint, RMSNorm weights converted to the MLXx·wconvention (+1) and the 8 linear modules quantized to W4/g64 affine with BF16 scales/biases (~137 MB). - Declared via
mtplx_mtp/mtplx_mtp_quantizationinconfig.json(prefix: "mtp.",block_size: 3). The head shares the target embedding and LM head, as intended by the source architecture.
There is no separate MTP repository: the proposal head travels inside the checkpoint, so the draft/target pairing cannot drift.
Measured (M4 Max, 546 GB/s, temperature 0)
| Configuration | decode tok/s |
|---|---|
| target only | 47.9–48.5 |
| embedded MTP | 73–83 |
Conversion notes
Raw source mtp.* norms are stored in the Qwen3-Next x·(1+w) convention and
MUST be shifted by +1 for MLX runtimes; quantization scales/biases MUST be
bfloat16. Raw norms produce a head whose drafts are rejected (slower than no
MTP at all); float16 scales fail the assistant load, which serving stacks
should surface as a logged fallback to target-only decoding.