Qwen3.8-27B-Coder390-MLX-slq-4.5bpw

Mixed-bitwidth SLQ quantization of nerkyor/Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-MTP-DFlash2, packed for MLX with nn.QuantizedLinear. The checkpoint stays packed in memory and on disk (~1.85× smaller than BF16).

Note on speculative decoding (MTP / DFlash2): This checkpoint uses SLQ quantization, which excludes the MTP head from quantization (it is grafted back as BF16). Because the MTP head was trained on BF16 backbone hidden states but runs against a quantized backbone, its accept rate is low (~1.4 tok/cycle vs ~2.5+ for a matched quantization). MTP and DFlash2 do not work well with this model. For speculative decoding, prefer an oQ-quantized variant where the MTP head and backbone are quantized uniformly through oMLX's engine.

Quick start

pip install mlx-lm
from mlx_lm import load

model, tokenizer = load(
    "hermitdave/Qwen3.8-27B-Coder390-MLX-slq-4.5bpw",
    trust_remote_code=True,
)

Model details

Bitwidth distribution

Exact per-layer allocation is stored in config.json → slq_bits.

Fidelity (measured)

Calibration: FineWeb-Edu, 131,072 tokens (128×1024). Allocator: per-layer greedy. Eval: paired zero-shot, 400 examples/task.

Per-task recovery (capped at 400 examples/task)

Note: full benchmark sets are ~15.5k examples/task; this run uses a 400-example cap for speed.

Size

Repo

Generated with slq-mlx. Pipeline: chunked per-block multi-bitwidth Stage 1, streaming error DB, heuristic allocation, MLX packed export.