Qwen3.8-27B-Abliterated — MLX bf16 with native MTP

Unquantized MLX conversion of windowsxp811203/Qwen3.8-27B-Abliterated (an abliterated Qwen/Qwen3.8-27B) with the MTP draft head and the vision tower kept in the checkpoint. 51.75 GiB (55.56 GB) of safetensors — this is the reference the quantized builds were made from, and the right starting point if you want to run your own oQ / mlx-vlm quantization with MTP preserved.

runtime	what you get
oMLX	native MTP speculative decoding (`mtp_enabled: true`) + vision, at full bf16 quality
mlx-vlm 0.6.17 (tested on the oQ siblings)	loads as a normal bf16 VLM — its `sanitize()` drops the in-file MTP tensors; add `--draft-model <the downloaded drafter/ folder>` for MTP. Releases that skip sanitize on MLX-format files (e.g. 0.6.3) reject the `language_model.mtp.*` keys

Quantized siblings (same conversion, oMLX oQ with preserve_mtp=True — MTP head kept, its Linears at the base bits, mtp.fc left bf16): oQ4e/ ((15.81 GiB)) · oQ6e/ ((22.09 GiB)) · oQ8e/ ((27.94 GiB)) — all in the project repo.

What is in the file

1199 tensors, all bf16: language model (64 layers, 48 GatedDeltaNet + 16 full attention), vision tower 333 tensors, MTP head 15 tensors (language_model.mtp.*, mtp_num_hidden_layers: 1), lm_head untied.
MLX conventions applied by the converter: the zero-centered RMSNorm weights carry the +1 offset (input/post-attention layernorms, q_norm/k_norm, the final norm and every MTP norm including pre_fc_norm_*; the GDN linear_attn.norm gated RMSNorm is left as-is), the GDN conv1d is transposed to (channels, kernel, 1), and the vision patch_embed.proj.weight is moved to channels-last (out, T, H, W, in).
Provenance: converted with mlx_vlm.convert(quantize=False, dtype=bfloat16) under oMLX 0.6.4's MTP sanitize patch (mlx-vlm 0.6.3 @ 78b96eb, mlx 0.32), straight from the HF bf16 parent with no dtype change (all 1199 tensors bf16); apart from the three MLX layout conventions above, tensors are copied unchanged.

Verification

Probe against the HF parent: 15/15 MTP and 333/333 vision tensors present, all 1199 tensors bf16, and the sampled RMSNorm offsets (7 MTP + 3 trunk norms) are +1.000 within bf16 rounding.
MMLU (oMLX accuracy bench: 5-shot, its seeded 400-question sample stratified by subject, temperature 0, non-thinking, MTP enabled at depth 3 — the same settings as the oQ runs; single-letter answers barely exercise the draft path, so the score is effectively independent of MTP): 82.50 % (330/400) — the reference the oQ cards compare against.
Refusal was not re-run on this bf16 build. The parent's numbers carry over (same weights, MLX layout): AdvBench 520 → 1/520 (0.19 %) non-thinking, 0/520 thinking; HarmBench safety categories → 0/293. The oQ4e/oQ6e/oQ8e siblings quantized from it each measured 0/80 on AdvBench and 0/119 on HarmBench-safety (no prefill, non-thinking).

Usage (oMLX)

brew tap jundot/omlx https://github.com/jundot/omlx && brew install jundot/omlx/omlx
hf download windowsxp811203/Qwen3.8-27B-Abliterated-MLX-MTP --include "bf16/*" --local-dir ~/.omlx/models
omlx serve --model-dir ~/.omlx/models     # the model id is the folder name: bf16

Then in ~/.omlx/model_settings.json (note the models wrapper — a top-level model key is silently ignored):

{"version": 1, "models": {"bf16": {"mtp_enabled": true, "mtp_num_draft_tokens": 3}}}

The server log should print Speculative backend selected … Lightning MTP (model_type=qwen3_5, active) when it loads.

Support / 打賞

If these models are useful to you, tips are appreciated — they pay for the GPU time. 如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。

USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888

Disclaimer