Qwen3.6-27B-MLX-6bit
A 6-bit quantization of Qwen/Qwen3.6-27B for Apple Silicon (MLX). In our evaluation this is the low-degradation sweet spot: statistically indistinguishable from the BF16 model while being ~2.6× smaller and faster.
- 6.50 bits/weight, ~21 GB on disk
- ~25 tok/s decode, ~22 GB peak memory on an Apple M5 Max
- Weight-only affine quant (activations stay FP16),
group_size=64 - Effectively lossless (see measurements)
Want smaller/faster and can accept a measurable quality cost? See the mixed-4bit build (17 GB, ~30 tok/s). Want maximum fidelity? See the 8-bit build.
Measured quality (vs BF16)
Perplexity, KL-divergence, and top-1 agreement against the BF16 model on a fixed 650-token multi-domain passage. Indicative smoke test, not a full benchmark — a relative comparison between builds.
| Build | bpw | size | ppl | Δppl vs BF16 | KL(bf16‖q) | top-1 agree |
|---|---|---|---|---|---|---|
| BF16 (reference) | 16 | 55.6 GB | 6.622 | — | — | — |
| 8-bit | 8.5 | 27 GB | 6.635 | +0.20% | 0.0018 | 97.7% |
| 6-bit (this model) | 6.5 | 21 GB | 6.595 | −0.41%¹ | 0.0087 | 96.6% |
| mixed-4bit | 5.36 | 17 GB | 6.977 | +5.36% | 0.0405 | 91.5% |
| uniform-4bit | 4.50 | 15 GB | 6.989 | +5.53% | 0.0651 | 89.8% |
¹ within noise — 6-bit is statistically indistinguishable from BF16 on this sample. There is a sharp quality "cliff" below 6-bit: dropping to 4-bit costs ~+5% perplexity and ~6 points of top-1 agreement.
Usage
pip install mlx-lm
mlx_lm.generate --model EigenLabs/Qwen3.6-27B-MLX-6bit \
--prompt "Explain gated delta networks in two sentences." --max-tokens 256
from mlx_lm import load, generate
model, tokenizer = load("EigenLabs/Qwen3.6-27B-MLX-6bit")
messages = [{"role": "user", "content": "What is 17*23?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))
Qwen3.6 is a reasoning model — the chat template opens a <think> block, so responses begin with a chain of thought.
About the model
Qwen3.6-27B is a hybrid architecture: 48 of 64 layers use Gated-DeltaNet linear attention and 16 use full softmax attention, with a 262K context window. At 6-bit, quantization error is negligible across all layer types, so this build uses a uniform 6-bit affine scheme (group_size=64) rather than mixed precision.
Requirements & notes
- Apple Silicon with MLX; ~24 GB free unified memory. Comfortable on 32 GB+, ideal on 48 GB+ for long context.
- Text-only. The MLX
qwen3_5implementation does not load the model's MTP head or its vision tower (usemlx-vlmseparately for images). - Hybrid Gated-DeltaNet layers run on the Metal GPU.
How it was made
from mlx_lm import convert
convert("Qwen/Qwen3.6-27B", mlx_path="Qwen3.6-27B-MLX-6bit", quantize=True, q_bits=6, q_group_size=64)
Tokenizer, chat template, and generation config are copied verbatim from the base model.
License & attribution
Apache 2.0, inherited from Qwen/Qwen3.6-27B (© Alibaba / Qwen team). This is a quantized derivative produced by EigenLabs. Please cite the original Qwen3.6 model.