gemma-4-E4B-it-BaseQuant-V1
A quantized build of google/gemma-4-E4B-it. It is closer to the BF16 model than the unsloth quant of the same size.
| Size | baseRT bundle | Same size as |
|---|---|---|
| Medium | gemma-4-E4B-it-BaseQuant-V1-Medium.base (4.98 GB of quantized text weights) |
unsloth Q4_K_M (4.98 GB) |
The bundle also carries the vision and audio towers at 16-bit (0.96 GB; 5.97 GB file). The size comparison is text weights against text weights: unsloth's GGUFs hold the text model only.
Usage
hf download basecompute/gemma-4-E4B-it-BaseQuant-V1 gemma-4-E4B-it-BaseQuant-V1-Medium.base --local-dir .
basert serve ./gemma-4-E4B-it-BaseQuant-V1-Medium.base
Results
KL divergence of each quant's next-token distribution from the BF16 model. Lower is closer to BF16. The percentage over each pair is our change against unsloth.
Public prompts
The prompts come from seven public datasets (OpenHands, SWE-agent, Glaive function calling, OpenOrca, Tulu 3, OpenHermes, UltraChat), published as basecompute/quant-fidelity-eval. The BF16 model writes each continuation, and every quant is scored along it. BaseQuant is 81% closer to BF16 than unsloth's quant of the same size, and closer on 139 of the 140 prompts.
| Source | unsloth Q4_K_M (4.98 GB) | unsloth UD-Q4_K_XL (5.13 GB) | BaseQuant Medium (4.98 GB) | Change vs Q4_K_M |
|---|---|---|---|---|
| OpenHands | 0.03348 | 0.02431 | 0.00391 | -88% |
| SWE-agent | 0.03219 | 0.02290 | 0.00506 | -84% |
| Glaive function calling | 0.03453 | 0.02990 | 0.00609 | -82% |
| OpenOrca | 0.03141 | 0.02451 | 0.00561 | -82% |
| Tulu 3 | 0.02444 | 0.02085 | 0.00450 | -82% |
| OpenHermes | 0.02445 | 0.01978 | 0.00498 | -80% |
| UltraChat | 0.04787 | 0.03760 | 0.01003 | -79% |
| All seven | 0.03191 | 0.02551 | 0.00597 | -81% |
| All but UltraChat | 0.02807 | 0.02261 | 0.00499 | -82% |
Scored along the dataset's own text instead of the model's continuations, BaseQuant is 77% closer than Q4_K_M and 69% closer than UD-Q4_K_XL:
| Dataset text | unsloth Q4_K_M (4.98 GB) | unsloth UD-Q4_K_XL (5.13 GB) | BaseQuant Medium (4.98 GB) | Change vs Q4_K_M |
|---|---|---|---|---|
| OpenHands | 0.03237 | 0.02386 | 0.00589 | -82% |
| SWE-agent | 0.02503 | 0.01755 | 0.00422 | -83% |
| Glaive function calling | 0.03132 | 0.01973 | 0.00948 | -70% |
| Tulu 3 | 0.01768 | 0.01452 | 0.00540 | -69% |
| OpenHermes | 0.03764 | 0.02977 | 0.00947 | -75% |
| UltraChat | 0.05279 | 0.03862 | 0.01172 | -78% |
| All six | 0.03194 | 0.02394 | 0.00734 | -77% |
Mean KL over every scored token; each source is position-weighted. BaseQuant runs in baseRT and unsloth's quants in llama.cpp, all against the same BF16 references. Bootstrapping over prompts (10,000 resamples), BaseQuant's KL is 0.19x unsloth Q4_K_M's [0.18, 0.20] on the model's continuations and 0.23x [0.21, 0.26] on dataset text.
Evaluation sets
On the model's own continuations and on written text, BaseQuant is 73-88% closer to BF16 than unsloth's quants, including UD-Q4_K_XL, which is 150 MB larger.
| Evaluation set | unsloth Q4_K_M (4.98 GB) | unsloth UD-Q4_K_XL (5.13 GB) | BaseQuant Medium (4.98 GB) |
|---|---|---|---|
| Model's own continuations | 0.02097 | 0.01614 | 0.00339 |
| Written text | 0.03336 | 0.02654 | 0.00721 |
| Tool use | 0.01276 | 0.00996 | 0.00175 |
| Agentic coding | 0.02462 | 0.01774 | 0.00294 |
| Top-1 agreement, 32-token greedy continuations | 97.31% | 98.19% | 99.07% |
Bold marks the lowest KL (highest agreement).