LFM3-1.7B-MLX-MACaQ-R-4.25bpw

MACaQ-R weight-only refinement of MLX's native MXFP4 quantization of LiquidAI/LFM3-1.7B, packed for MLX.

Unlike the standard MLX mxfp4 path, which is plain round-to-nearest, this checkpoint has its E2M1 codes re-optimised against the BF16 teacher using MACaQ's Flip-Refinement and Hessian-weighted exponent screening. It loads with stock mlx-lm — no custom runtime, no custom kernels, no trust_remote_code.

Quick Start

from mlx_lm import load, generate

model, tokenizer = load("hermitdave/LFM3-1.7B-MLX-MACaQ-R-4.25bpw")
print(generate(model, tokenizer, prompt="The capital of France is", max_tokens=32))

Or from the CLI:

mlx_lm.generate --model hermitdave/LFM3-1.7B-MLX-MACaQ-R-4.25bpw \
    --prompt "The capital of France is"

Model Details

Fidelity (measured)

WikiText-2 test, 130,816 tokens, sequence length 512, identical evaluation data for every row.

• 22.1% lower perplexity than the stock MLX mxfp4 quantization of the same checkpoint.
• Closes 67% of the BF16 → mxfp4 gap.
• Better than affine-4 at a lower bit width (4.25 vs 4.50 bpw).
• 32.5% reduction in the total activation-Hessian reconstruction objective across all 153 quantized linear layers.

Size

Method

This is the weight-only variant of MACaQ (the "MACaQ-R" strategy from the paper's Appendix). For each of the 153 quantized linear layers:

1. Unpack MLX's raw E2M1 nibbles and E8M0 block scales.
2. Capture the activation Hessian H = Σ xᵀx from the BF16 teacher on 200K WikiText-2 training tokens, using forward hooks.
3. Re-screen the block exponents over offsets {−1, 0, +1} under the diagonal Hessian metric.
4. Flip-Refinement — greedy coordinate descent on the E2M1 codes with the closed-form gain 2Δg − Δ²H_jj, exponents held fixed (256 sweeps, with early stopping).
5. Repack into MLX's native mxfp4 layout and write back.

No neighbouring layer, normalization, or residual branch is modified, so the file layout, config, and runtime are unchanged. The output is a standard MLX quantized model.

Ablation on this checkpoint (all at 200K calibration tokens unless noted):

Limitations

• Not a fine-tune. This is lossy compression; perplexity remains above BF16.
• No Hadamard rotation. The paper's largest single ablation term is not available here, because applying it requires an online activation transform that MLX's quantized matmul does not expose. This is the weight-only variant.
• W4A16 only. MLX's qqmm W4A4 kernel is only faster than BF16 at batch ≤ 4 on Apple Silicon, so this checkpoint ships as weight-only.
• Single model validated. Measured on LFM3-1.7B only.

Attribution

• Base model: LiquidAI/LFM3-1.7B
• Quantization method: MACaQ (arXiv 2611.10649), Apache-2.0 — Flip-Refinement and Hessian-weighted exponent screening follow its design
• MLX port, packing layer, and benchmarks: developed with Hermes Agent — hermitdave/macaq-mlx
• Calibration data: WikiText-2 train split