Qwen3.5-2B-MLX-slq-4.5bpw

Mixed-bitwidth SLQ quantization of Qwen/Qwen3.5-2B, packed for MLX with nn.QuantizedLinear. The checkpoint stays packed in memory and on disk (~2.3× smaller than BF16).

Quick start

pip install mlx-lm
from mlx_lm import load

model, tokenizer = load(
    "hermitdave/Qwen3.5-2B-MLX-slq-4.5bpw",
    trust_remote_code=True,
)

Model details

Field	Value
Base model	Qwen/Qwen3.5-2B
Parameters	~2B
Layers	24
Hidden size	2048
Architecture	`qwen3_5` (Gated DeltaNet + hybrid attention)
Quantization	SLQ, affine, group_size=64
Average bitwidth	4.49 bpw
Format	MLX packed (`mlx-packed-affine`)

Bitwidth distribution

Bitwidth	Share
4-bit	~35%
5-bit	~50%
6-bit	~12%
8-bit	~3%

Exact per-layer allocation is stored in config.json → slq_bits.

Fidelity (measured)

Calibration: FineWeb-Edu, 131,072 tokens (128×1024). Allocator: per-layer greedy. Eval: paired zero-shot, 400 examples/task.

Metric	Value
Mean zero-shot recovery	0.9879
Task-lossless verdict	No (threshold ≥ 0.99)

Per-task recovery (capped at 400 examples/task)

Task	BF16	SLQ 4.49bpw	recovery	95% CI
arc_challenge	0.3825	0.3775	0.9869	[0.9710, 1.0028]
arc_easy	0.7175	0.6775	0.9443	[0.9300, 0.9586]
hellaswag	0.4150	0.4225	1.0181	[1.0049, 1.0313]
winogrande	0.5275	0.5075	0.9621	[0.9462, 0.9780]
mmlu	0.3575	0.3675	1.0280	[1.0082, 1.0478]

Note: full benchmark sets are ~15.5k examples/task; this run uses a 400-example cap for speed.

Size

Artifact	Size
Packed SLQ	~1.4 GiB
BF16 base	~3.2 GiB

Repo

Generated with slq-mlx. Pipeline: chunked per-block multi-bitwidth Stage 1, streaming error DB, heuristic allocation, MLX packed export.

hermitdave/Qwen3.5-2B-MLX-slq-4.5bpw