EigenLabs/Qwen3.8-27B-MTP-4bit
This is a non-standalone, one-layer native MTP proposal head for the pinned target backbone EigenLabs/Qwen3.8-27B-4bit@301e9e2767fd0efcfab7883004720ba3c9a552a1. It proposes tokens only. The target backbone remains authoritative for language tensors, embeddings, the LM head, token acceptance, and final output.
The official model lineage is Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. This artifact was derived locally from the exact EigenLabs BF16 MTP artifact EigenLabs/Qwen3.8-27B-MTP-bf16@26a328e070875b0314d652a039b6b59902690f03, never by copying a third-party quantized checkpoint.
Artifact contract
model_type:qwen3_5_mtp- MTP layers: 1 (
layers.0only) block_size: 3- 31 tensor keys, all bare (no
mtp.prefix) - Eight 2-D matrices stored as canonical
.weight,.scales, and.biasestensors - Seven 1-D normalization tensors retained in BF16
- No tokenizer, target embeddings, target LM head, target-backbone weights, or vision weights
This repository must be paired with the immutable target revision above. Updating or downloading this repository alone does not register it in a runtime catalog or enable speculative decoding.
Quantization
Quantization used MLX 0.31.2 directly:
weight, scales, biases = mlx.core.quantize(
source_bfloat16, group_size=64, bits=4, mode="affine"
)
The packed weights are uint32; affine scales and biases remain BF16. MLX-LM 0.31.3, safetensors 0.7.0, and huggingface-hub 1.7.2 were installed in the conversion environment. Each matrix was independently dequantized with mlx.core.dequantize(..., group_size=64, bits=4, mode="affine", dtype=mlx.float32) and compared with its exact BF16 source.
| Matrix | Source shape | Packed shape | Scale/bias shape | Max abs error | Mean abs error |
|---|---|---|---|---|---|
fc.weight |
5120×10240 | 5120×1280 | 5120×160 | 0.041015625 | 0.0008614016 |
layers.0.mlp.down_proj.weight |
5120×17408 | 5120×2176 | 5120×272 | 0.0546875 | 0.0010076930 |
layers.0.mlp.gate_proj.weight |
17408×5120 | 17408×640 | 17408×80 | 0.017578125 | 0.0009606532 |
layers.0.mlp.up_proj.weight |
17408×5120 | 17408×640 | 17408×80 | 0.0341796875 | 0.0010548841 |
layers.0.self_attn.k_proj.weight |
1024×5120 | 1024×640 | 1024×80 | 0.0166015625 | 0.0012780037 |
layers.0.self_attn.o_proj.weight |
5120×6144 | 5120×768 | 5120×96 | 0.09765625 | 0.0016031306 |
layers.0.self_attn.q_proj.weight |
12288×5120 | 12288×640 | 12288×80 | 0.02734375 | 0.0011431416 |
layers.0.self_attn.v_proj.weight |
1024×5120 | 1024×640 | 1024×80 | 0.021484375 | 0.0023364164 |
All source, packed, scale, bias, and dequantized values passed finite-value checks. After the local derivation was complete, all 31 local tensor payloads were compared with the independent pinned reference mlx-community/Qwen3.8-27B-MTP-4bit@b643c01b6d3b094e325edb6ebd832e16c486c575; names, dtypes, shapes, and raw tensor bytes matched exactly.
These checks establish artifact structure and quantization fidelity only. They are not end-to-end generation, acceptance-rate, quality, or speedup claims.