Bespoke Quantization // ik_llama.cpp
Split-Untied-31B
A meticulously crafted IQK mix for the darker side of roleplay.
150
Cycles
300
Chunks
1024
Context
4h+
Compute Time
This repository hosts a highly specialized ik_llama.cpp quantization of Blazed-Forge/Split-Untied-31B. The base model is Nimbz's 31B Gemma 4 merge, built through a three-phase StyleSwap process that prioritizes creative writing, darker themes, and expressive freedom. It excels at imaginative, less constrained narratives and deep character roleplay.
Standard quantization methods treat every layer with the same broad strokes. MaxOptimaV2 is different. This repository is dedicated entirely to this bespoke, layer-by-layer recipe. It was designed to compete directly with standard IQ4_KS, offering quality on par with or slightly better than a generic Q4_K_M while maintaining a smaller file size. Every tensor was deliberately assigned to protect the model's creative voice.
The MaxOptimaV2 Strategy
To preserve the expressive nature of Split-Untied, standard uniform 4-bit compression is insufficient. The MaxOptimaV2 script uses a surgical sandwich approach. It safeguards the most critical attention and feed-forward layers at IQ5_K and Q6_K, while compressing the bulk of the model to IQ4_KSS and IQ4_KS.
The V3 Calibration Dataset
The importance matrix for this quantization was not generated lightly. It required over four hours of dedicated compute time, processing 300 chunks of domain-specific data at a 1024-token context window across 150 cycles.
Crucially, this imatrix was built using a custom V3 Calibration Dataset. We took the OpenIntelligenceNet Mixed Data and completely stripped out all Polyglot, Chain-of-Thought, and Factual knowledge data. What remains is a laser-focused dataset containing only Uncensored and Roleplay data. This ensures the quantization weights are perfectly optimized for creative and unfiltered inference, rather than wasting precision on general factual recall.
40%
25%
20%
15%
40% v2_stripped
25% DPO Roleplay (ath)
20% Character Cards (gry)
15% SynthRP (epi)
The v2_stripped source consists of the Uncensored Alpaca Blend and Multi-Turn Roleplay & Fiction. The ath source provides DPO Pairs for NSFW roleplay. The gry source is Sonnet 3.5 character card roleplay. The epi source is synthetic roleplay generations.
Split-Untied-31B
MaxOptimaV2
A meticulously crafted IQK mix for the darker side of roleplay.
150
Cycles
300
Chunks
1024
Context
4h+
Compute Time
This repository hosts a highly specialized ik_llama.cpp quantization of Blazed-Forge/Split-Untied-31B. The base model is Nimbz's 31B Gemma 4 merge, built through a three-phase StyleSwap process that prioritizes creative writing, darker themes, and expressive freedom. It excels at imaginative, less constrained narratives and deep character roleplay.
Standard quantization methods treat every layer with the same broad strokes. MaxOptimaV2 is different. This repository is dedicated entirely to this bespoke, layer-by-layer recipe. It was designed to compete directly with standard IQ4_KS, offering quality on par with or slightly better than a generic Q4_K_M while maintaining a smaller file size. Every tensor was deliberately assigned to protect the model's creative voice.
The MaxOptimaV2 Strategy
To preserve the expressive nature of Split-Untied, standard uniform 4-bit compression is insufficient. The MaxOptimaV2 script uses a surgical sandwich approach. It safeguards the most critical attention and feed-forward layers at IQ5_K and Q6_K, while compressing the bulk of the model to IQ4_KSS and IQ4_KS.
| Layer Group | Quant | Rationale |
|---|---|---|
| output.weight | Q6_K | Final projection kept at high precision to preserve vocabulary richness. |
| token_embd.weight | Q5_K | Embeddings balanced for size versus semantic depth. |
| blk.0 / blk.58 attn_v | IQ5_K | Edge attention layers protected at 5-bit. |
| blk.5, 11, 17, 23, 29, 35, 41, 47, 53, 59 attn_k | IQ5_K | Key attention layers kept sharp for multi-turn context tracking. |
| blk.0 / blk.59 ffn_down | IQ5_K | Boundary FFN layers preserved. |
| All other ffn_down | IQ4_KSS | Balanced 4-bit for the bulk of the feed-forward network. |
| blk.50–59 ffn_gate / ffn_up | IQ4_KSS | Late FFN layers kept at 4-bit to protect output style. |
| blk.0–19 ffn_gate / ffn_up | IQ4_KSS | Early FFN layers similarly balanced. |
| All remaining tensors | IQ4_KS | Default fallback for maximum compression at 4-bit. |
Competitive Positioning: MaxOptimaV2 targets the same niche as IQ4_KS but with a finer-grained weight distribution. In practice, it offers comparable or slightly better roleplay coherence than Q4_K_M while producing a smaller file. It is not a universal upgrade. It is a purpose-built tool for creative and uncensored inference.
Compatibility MaxOptimaV2 is exclusive to ik_llama.cpp. It will not load in mainline llama.cpp. If you require mainline compatibility or standard formats like BF16, please refer to my previous repository: Austriani/Split-Untied-31B-ik_llama.cpp. Austriani // Split-Untied-31B MaxOptimaV2 Base: Blazed-Forge/Split-Untied-31B // V3 Calibration