Near-lossless GGUF quants of:
Chat template has been updated (2026-07-15 version).
Original QAT Specifications
The base QAT model was trained with the following specifications:
- Text & Draft Models: 4-bit
- Multimodal Projector (
mmproj): BF16
Quantization Error Evaluation
- Dequantization: Quantized weights were dequantized to F32 for evaluation.
- Error Calculation: Quantization error metrics (compared to the BF16 baseline) were computed and accumulated in F64 precision to prevent numerical underflow and precision loss.
Metrics:
- MAE (Mean Absolute Error)
- RMSE (Root Mean Squared Error)
- Max Error
[!NOTE] Interpretation of Metrics: These error metrics do not directly reflect actual model performance. This is because imatrix optimization intentionally increases measured error by downweighting less important parameters to improve overall performance. However, these metrics remain useful for estimating how close the quantized model is to the original, lossless quality.
gemma-4-31B-it-qat
| Model | MAE | RMSE | Max Error |
|---|---|---|---|
JMingo/gemma-4-31B-it-qat-BF16.gguf (Baseline) |
0.00000000 | 0.00000000 | 0.00000000 |
JMingo/gemma-4-31B-it-qat-Q4_0.gguf |
0.00000973 | 0.00001633 | 0.00061035 |
idkwhattoputherenow/gemma-4-31B-it-qat-q4_0-unquantized-q4_0-maxerr.gguf |
0.00001104 | 0.00001856 | 0.00062561 |
unsloth/gemma-4-31B-it-qat-UD-Q4_K_XL.gguf |
0.00001388 | 0.00002110 | 0.00195312 |
google/gemma-4-31B_q4_0-it.gguf |
0.00027934 | 0.00057681 | 0.01635742 |
lmstudio-community/gemma-4-31B-it-QAT-Q4_0.gguf |
0.00027934 | 0.00057681 | 0.01635742 |
mradermacher/gemma-4-31B-it-qat-q4_0-unquantized.i1-Q4_0.gguf |
0.00001890 | 0.00006541 | 0.02943420 |
ggnoy/gemma-4-31B-it-qat-q4-bim.gguf |
0.00001890 | 0.00006589 | 0.02049255 |
mtp-gemma-4-31B-it-qat
| Model | MAE | RMSE | Max Error |
|---|---|---|---|
JMingo/mtp-gemma-4-31B-it-qat-BF16.gguf (Baseline) |
0.00000000 | 0.00000000 | 0.00000000 |
JMingo/mtp-gemma-4-31B-it-qat-Q4_0.gguf |
0.00002260 | 0.00003839 | 0.00128174 |
unsloth/mtp-gemma-4-31B-it.gguf |
0.00003191 | 0.00004994 | 0.00390625 |