SPLIT UNTIED · MAXOPTIMA
31B · Gemma 4 · Custom Quant “Preserving the soul, optimizing the memory.” MIXED QUANTMEMORY OPTIMIZEDDATASET REAP V2ROLEPLAY FOCUSED ◖◗Release Status Hugging Face takes a significant amount of time to publish model weights. The models are finished, but uploading 120GB of data is a slow process.☰Available Quants Files follow the naming convention:Split-Untied-31B-{QuantType}.gguf
Quant TypeDescription
MaxOptima (Mixed)Custom mix. Highest quality-to-size ratio. Uses a sandwich structure with IQ3_K and IQ4_K.
IQ4_KSStandard 4-bit imatrix quant.
IQ4_KS_R4Alternative 4-bit imatrix variant (R4).
IQ5_KHigher fidelity, heavier memory footprint.
IQ6_KNear-lossless. For systems with sufficient VRAM or RAM.
⚡The MaxOptima Mix
ILayer Analysis
How it was builtI spent hours, both with myself and AIs, analyzing which layers should have higher importance, and which can be left to have lower.
The result came out as this mix. It uses a newly computed imatrix, with FFN.up and FFN.gate mostly at IQ3_K, creating a sandwich structure where the first and ending layers are IQ4_K.
More importance was attached to attention, so the model won't forget where it is and will have near-ideal retention.
IIDataset Reap V2
No fillerAll Polyglot, CoT, and Factual data was removed. Now retaining only Uncensored and RP.
110/200 chunks this time. It's still over the standard amount of 100. For a 31B Dense model, it takes a significant amount of time to make an imatrix (approximately 50 seconds per chunk).
IIISystem Focused
Smooth inferenceThe Q4_K_M from mradermacher is great, but for a 31B Dense model, memory efficiency is everything. On many systems, standard 4-bit quants can spill over into system RAM or cause swapping, which severely bottlenecks generation.
I made this quant to decrease the memory footprint while strategically increasing attention precision. It trades a very small amount of word variety for much better memory headroom.
⚡Speed Differences Compared to mradermacher's Q4_K_M quant. MetricDifference Token GenerationAverage ~4% decrease Prompt ProcessingAverage ~14% decrease◆ The Real-World Takeaway
While raw compute speed is slightly lower per token, the reduced memory footprint prevents severe generation bottlenecks caused by RAM swapping. In practice, overall generation throughput is often higher.
Benchmark Caveat: Benchmarked on an Intel Core Ultra 7 265K (CPU). On AVX-512-capable CPUs or discrete GPUs, the speed penalty will likely disappear, or shrink substantially.
◖◗Credits◆ Nimbz: For creating the Split Untied model and the original model card design layout.
◆ mradermacher: For the original Q4_K_M quant.
◖ · ◗