SenseNova-Loop-9B-Preview-MLX-slq-4.5bpw

Statistically-lossless quantization (SLQ) of SenseNova-Loop-9B-Preview to 4.5 bpw using per-layer mixed-bitwidth allocation. Packed artifact for MLX-LM on Apple Silicon.

Parent model

This model is a quantized variant of SenseNova-Loop-9B-Preview.

Model details

Property	Value
Base model	SenseNova-Loop-9B-Preview
Base model relation	quantized
Quantization method	SLQ (Statistically-Lossless Quantization)
Average bitwidth	4.5 bpw
Packed size	7.631 GiB
BF16 base size	38.1 GiB
Compression ratio	~5.00×
Parameters	~9B total (6.912B quantizable)
Architecture	Qwen3.5-text backbone, 32 distinct layers (56 virtual executions), hybrid linear + full attention, two-stream mHC
Quantizable projections	200
Excluded from quantization	mHC routers (FP32-critical, Sinkhorn-normalized), norms, embedding, LM head, vision encoder, MTP head

Bitwidth distribution

Bitwidth	Layers	Percentage
6-bit	16	8.0%
5-bit	114	57.0%
4-bit	70	35.0%

Fidelity (measured)

Calibration: FineWeb-Edu, 8192 tokens (32 samples × 256 tokens). Allocator: per-layer greedy heuristic, target 4.5 bpw.

Metric	Value
PPL ratio (FineWeb-Edu 256-seq)	1.0141
Symmetric KLD (FineWeb-Edu 256-seq)	0.1699
EAR (FineWeb-Edu 256-seq)	0.9382
Zero-shot mean recovery (3 tasks, n=100)	0.9623
Task-lossless verdict	No

Per-task zero-shot recovery (capped at 100 examples/task)

Task	BF16	SLQ 4.5bpw	recovery	paired diff	95% CI
arc_challenge	0.4700	0.4200	0.8936	−0.0500 ± 0.0332	[−0.1150, +0.0150]
hellaswag	0.4700	0.4500	0.9574	−0.0200 ± 0.0141	[−0.0477, +0.0077]
winogrande	0.5600	0.5800	1.0357	+0.0200 ± 0.0316	[−0.0420, +0.0820]

Note: the full benchmark sets are ~15.5k examples/task; this run uses a 100-example cap for speed.

Performance (measured)

Configuration	Throughput (tok/s)
BF16	TBD
SLQ 4.5bpw (packed)	TBD
4-bit uniform RTN	TBD

Note: TPS benchmarks pending. Warmup 8–16 tokens, measure 128–256 tokens at temperature 0.0 on M3 Max 64 GB.

Methodology

This checkpoint was produced with the slq-mlx pipeline:

Stage 1 (GPTQ): Per-layer GPTQ at bitwidths {2, 3, 4, 5, 6, 8} with group size 128, calibration on FineWeb-Edu (8192 tokens, seq len 256).
Stage 2 (Error DB): Built per-layer error database from all bitwidths.
Stage 3 (Search): Per-layer greedy heuristic allocation targeting 4.5 bpw average.
Stage 4 (Pack): Streaming layer-by-layer pack (run_pack_mlx.py --streaming) to avoid OOM on 64 GB M3 Max.
Stage 5 (Validation): Zero-shot recovery measured on arc_challenge, hellaswag, winogrande (100 examples/task).

The per-layer bitwidth map is stored in slq_bits inside config.json. The artifact uses the packed MLX format (nn.QuantizedLinear + mx.quantized_matmul), so it stays quantized in memory and on disk.

Model lineage

sensenova/SenseNova-Loop-9B-Preview (BF16 base)
└── hermitdave/SenseNova-Loop-9B-Preview-MLX-slq-4.5bpw (this model)

Limitations

Not task-lossless at 4.5 bpw (paper's Task-Lossless target is ≥0.99 recovery and requires Shapley+ILP at 512×2048 calibration).
Validation was performed on a small calibration set (32×256). Larger calibration may improve EAR.
The model uses a custom architecture file (slq_model.py) that wraps mlx_lm.models.qwen3_5 to bind the SLQ QuantizedLinear layers. Loading it requires trust_remote_code=True.
Looped execution (layers [4,28) run twice) increases KV cache footprint (~728 KB/token, ~3.05 GB @ 4K context).

Attribution

Quantization pipeline and streaming pack implementation by Hermes Agent.

License

MIT

hermitdave/SenseNova-Loop-9B-Preview-MLX-slq-4.5bpw