EXL3 quants of google/gemma-4-31B-it, quantized with exllamav3 1.6.6 from per-tensor sensitivity measurements on chat data rather than a uniform bit width. One revision per quant:
| revision | bits/weight | head | vision | size | KLD ↓ | × noise floor | p90 | PPL |
|---|---|---|---|---|---|---|---|---|
| SC_6.00bpw_H6_V8 | 6.00 | 6 | 8 | 26.5 GB | 0.001101 | 1.34× | 0.002450 | 1.18304 |
| SC_5.00bpw_H6_V6 | 5.00 | 6 | 6 | 22.7 GB | 0.002352 | 2.85× | 0.04654 | 1.18454 |
| SC_4.00bpw_H5_V6 | 4.00 | 5 | 6 | 18.8 GB | 0.007176 | 8.7× | 0.014008 | 1.18898 |
| 4.00 bpw uniform — turboderp | 4.00 | 6 | — | 19.7 GB | 0.012337 | 15.0× | 0.025374 | 1.19582 |
| UD-Q4_K_XL — Unsloth GGUF | 4.87 | 5.5 | — | 17.5 GB | 0.015032 | 18.2× | 0.031242 | 1.20010 |
Per-token KL divergence against bf16, histogram per model: 120 shared bins, uniform on the log x axis, cropped to each model's p1–p99 range. The dotted line is the noise floor — bf16 re-run against itself with rounding-scale perturbation, i.e. the divergence any lossy scheme inherits for free.
Also in this repo: KLD spread per layer type, per-model histograms, raw results, project file.
| revision | bits/weight | head | size | source |
|---|---|---|---|---|
| assistant_4.00bpw | 4.00 | 4 | 256 MB | google/gemma-4-31B-it-assistant |
EXL3 4.00 bpw quant of Google's assistant (multi-token-prediction) draft, quantized with the same exllamav3 1.6.6 build and calibration set. Pair it with any revision above — it proposes candidate tokens that the 31B target then verifies, so its own logits never reach the user. It is deliberately absent from the KLD table: scoring a draft model's single-token distribution against the target measures the wrong thing.
Calibration data from (also have 3.0 bpw quant) Dino0040/gemma-4-31b-it-exl3-optimized
google/gemma-4-31B-it