ICE: Quantization by Error-Propagation Class in Sparse MoE Models

Technical report and full evidence for ICE (Isolation of Compounding Error), a quantization bit-allocation method for sparse Mixture-of-Experts checkpoints.

Author: Gökhan Buz (gbuzhf)

Read the paper

paper/ICE_Technical_Report.md: the method (ICE v1, report version 1.1)

paper/ICE2_Technical_Report.md: ICE 2.0, format-preserving expert repair

Version history

What ICE is

Every quantizer in the GGUF ecosystem minimizes the same objective for every tensor: the importance-matrix-weighted error of that tensor's output, for the current token. That is the right objective for a tensor whose error dies with the token. It is the wrong objective for three other kinds, and no published method separates them.

ICE classifies every tensor by how far its error travels:

The first three are 0.14% of the model, so freezing them is a line item rather than a trade-off. In one line: freeze what propagates, spend everything else on the library.

Headline results

Mean KL divergence against the bf16 checkpoint, WikiText-2, one harness for all files.

Across the twelve-tier comparison, nine tiers are Pareto-optimal and three are strictly dominated. ICE does not win at the top of the ladder: UD-Q5_K_S and UD-Q6_K are the two best files measured, which the paper explains rather than omits (Law 4).

The four laws

1. Sparsity. A bit on the always-on core is worth E/k bits on the expert bank. Measured 31.8 against a predicted 32.
2. Convexity. Error falls as 4^-b, so allocation cleverness is capped at +0.139 bpw. Two consequences: do no depth grading, and at a fixed average always pick the narrower type bracket (measured +16.6% and +7.0% penalty for widening).
3. Placement. Inside a fixed bracket, shallow-first is worth about −8.5% KLD per bpw of gap and reverses below 0.47 bpw. Depth gain measured directly as g(t) ≈ exp(−t/9.95).
4. The floor is epistemic. Fitted on two independent harnesses at k₀ = 0.0205 and 0.0219, and confirmed by a direct probe at 0.018297. The best file measured is 8% above it. You cannot out-bit a wrong prior.

What is in this repository

paper/ICE_Technical_Report.md      the report
appendix/original-ICE/             PRINCIPLE.md and RECIPE.md, the method as first written
appendix/recipes/ICE-1.0/          every tensor-type-file cited in the ICE v1 report
appendix/recipes/ICE-1.6/          ICE 1.6 recipes for Ornith-1.5-35B-A3B, 12G to 25G (README inside)
appendix/measurements/             raw llama-perplexity output for every KLD quoted
appendix/ice-v1.1-ice2-evidence.tar.gz   raw KLD logs for report v1.1 and the ICE 2.0 report (INDEX.md inside)
appendix/SHA256SUMS.txt            checksum for every file above

appendix/recipes/ICE-1.0/

cfg_21G-ICE_ICEbase.txt and cfg_21G-ICE_previous.txt have the same SHA-256 (93456d45b1fc0dab...). That is not an oversight. 21G is the tier where Law 3's condition does not hold, so the method's output is to change nothing, and the identical checksum is the proof that nothing was changed.

Recipes for the three dominated tiers are not included: they were dropped after the Pareto analysis and none was retained. Their measured numbers are in the paper.

appendix/recipes/ICE-1.6/

The five ICE 1.6 tensor-type files for Ornith-1.5-35B-A3B (12G, 15G, 19G, 23G, 25G), the recipes behind Section 3.3 of the ICE 2.0 report. Its README.md gives the command line and the resulting file sizes.

Reproducing a number

llama-perplexity -m <file>.gguf -f wiki.test.raw \
    -c 2048 --chunks 64 --kl-divergence-base base.kld --kl-divergence

against a base.kld produced once from the bf16 checkpoint with the same corpus and chunk count. Protocol details, including how the byte-identical control variants are constructed, are in Appendix B.

The same recipe measured through three different paths gave 0.0345, 0.034162 and 0.034290, a spread under 1%.

Negative results

Seven are documented at the same weight as the positive ones, including the retraction of a rule this work itself derived and shipped: bumping ffn_down above its sibling projections, which is standard practice, measured 11.3% worse than uniform at identical size. One of three registered predictions also failed, and it is scored as such.

On the comparison ladders

Unsloth Dynamic 2.0 and LocalAI APEX are the work of their respective authors. They are measured here, not reproduced or modified. The comparison exists because no individual publication can provide it: each ladder is published with its own harness and its own reference, so the tiers are not comparable until someone puts them on one. The same analysis that finds ICE tiers dominating three others also finds two UD tiers to be the best files on the board and two APEX tiers to be the only options below 18.5 GB.

Citing

Cite this repository. Model cards for ICE-quantized GGUFs link here for the method description.