LFM2.5-2.6B-MLX-SLQ-4.49bpw
SLQ (Statistically-Lossless Quantization) packed checkpoint of LiquidAI/LFM2.5-2.6B, at 4.49 bpw via MLX-LM's QuantizedLinear.
- Base:
LiquidAI/LFM2.5-2.6B - Architecture: LFM2 hybrid (22 short-conv blocks + 8 full-attention blocks)
- Parameters: 2.69B
- Layers: 30
- Quantization: per-layer non-uniform affine (SLQ), group_size=128
- Target bpw: 4.50 → measured 4.4948 bpw
- Format: mlx-packed-affine (runs on MLX fused
quantized_matmul) - Context: 128K
Fidelity (measured)
Calibration: FineWeb-Edu, 1,048,576 tokens (32×1024), 512 seq len. Allocator: per-layer greedy, EAR target 0.99.
| Metric | Value |
|---|---|
| PPL ratio (FineWeb-Edu 512-seq) | 1.0456 |
| Symmetric KLD (FineWeb-Edu 512-seq) | 4.31 |
| Zero-shot mean recovery (4 tasks, n=400) | 1.0128 |
| Task-lossless verdict | ✅ Yes |
Per-task recovery (capped at 400 examples/task)
| Task | BF16 | SLQ 4.49bpw | recovery | 95% CI |
|---|---|---|---|---|
| arc_challenge | 0.3600 | 0.3850 | 1.0694 | [-0.0068, +0.0568] |
| arc_easy | 0.5925 | 0.5800 | 0.9789 | [-0.0439, +0.0189] |
| hellaswag | 0.4175 | 0.4100 | 0.9820 | [-0.0238, +0.0088] |
| winogrande | 0.4800 | 0.4900 | 1.0208 | [-0.0304, +0.0504] |
Note: the full benchmark sets are ~15.5k examples/task; this run used a 400-example cap for speed.
Size
| Artifact | Size | vs BF16 |
|---|---|---|
| BF16 base | ~5.0 GiB | — |
| SLQ packed | ~1.9 GiB | ~2.6× smaller |
Pipeline
Stage 1: GPTQ → per-layer fake-quant weights at bitwidths 2–8
Stage 2: build → error_db.json (relative Frobenius error per layer)
Stage 3: search → heuristic greedy allocator (EAR ≥ 0.99 target)
Stage 4: build → packed MLX checkpoint (QuantizedLinear)
Usage
pip install mlx mlx-lm
python -m mlx_lm.generate --model hermitdave/LFM2.5-2.6B-MLX-SLQ-4.49bpw \
--prompt "The capital of France is" --max-tokens 64
Requires MLX ≥ 0.27 (Apple Silicon only).
This checkpoint uses a custom architecture file (slq_model.py) that wraps the base model with per-layer QuantizedLinear layers. Loading it requires trust_remote_code=True.
How this was built
- Tool: slq-mlx-port (Hermes Agent workflow)
- Base model: LiquidAI/LFM2.5-2.6B
- Calibration data: FineWeb-Edu (subset)
- Allocator: per-layer greedy (EAR target 0.99)
- Packing: MLX
QuantizedLinearwith affine integer grid
Attribution
Quantization pipeline and validation harness built with Hermes Agent (by NousResearch). Full benchmark results and reproduction steps documented in the upstream slq-mlx repo.