Austriani // ik_llama.cpp // GGUF

Kitchoon-Final-26B-A4B

MoE Roleplay Quantization // V3 Calibration Imatrix 26B Total Params 4B Active Params 1024 Imatrix Ctx 500 Cycles The Base Model This repository hosts ik_llama.cpp-compatible GGUF quantizations of SubMaroon/Kitchoon-Final-26B-A4B. This is the final iteration of the Kitchoon line, built on the Gemma 4 26B-A4B Mixture-of-Experts architecture (26B total, 4B active per token). It is specifically tuned for RP and ERP, focusing on character development and a neutral writing style. The "Final" release bakes in Anthropos v3 at 0.7. This is a LoRA trained via the BACo method to mix the instruction-tuned model with the pre-trained Gemma base, resulting in fewer stock phrases, fewer em dashes, and more diverse swipes without sacrificing formatting or reasoning. Thinking mode is preserved and fully functional. MaxOptima Custom Quant In addition to the standard IQK formats, this release includes MaxOptima. This is a bespoke, surgical quantization mix designed specifically for the 128-expert MoE architecture of Gemma 4. The goal is to protect the expert routing and critical attention layers while aggressively compressing the bulk of the expert FFNs. MaxOptima V1 Strategy: Embeddings and output layers are locked at Q6_K. Router inputs, norms, scales, and biases are kept at full precision (F32) to preserve routing integrity. Key attention layers are kept at IQ5_K. The bulk of the expert FFNs are compressed to IQ4_KSS and IQ4_NL, with boundary blocks protected at higher precision. View Full MaxOptima V1 Custom Quant Configuration ^output\.weight=Q6_K,^token_embd\.weight=Q6_K,ffn_gate_inp\.weight=F32,rope_freqs\.weight=F32,.*_norm\.weight=F32,.*\.scale=F32,.*_scale\.weight=F32,.*\.bias=F32,blk\.(0|1|2|3|4|5|6|11|17|18|23|24|25|26|27|28|29)\.attn_k\.weight=IQ5_K,attn_k\.weight=IQ4_KSS,attn_v\.weight=IQ5_K,blk\.(0|1|2|3|4|5|11|17|23|25|26|27|28|29)\.attn_output\.weight=IQ5_K,attn_output\.weight=IQ4_KS,attn_q\.weight=IQ5_K,blk\.(0|1|2|3|4|25|26|27|28|29)\.ffn_gate_up_exps\.weight=IQ5_K,ffn_gate_up_exps\.weight=IQ4_KSS,blk\.(0|1|2|3|4|25|26|27|28|29)\.ffn_down_exps\.weight=Q5_0,ffn_down_exps\.weight=IQ4_NL,ffn_down\.weight=Q5_0,ffn_(gate|up)\.weight=IQ5_K,.*=IQ4_KS MaxOptima V2 Custom Quant MaxOptima V2 is the refined successor to V1. It replaces the legacy Q5_0 and older IQ4_NL assignments on geometry-constrained tensors with the native IQK 32-block fallback (IQ4_KS_R16), and lifts all attention projections to IQ5_K across every layer. The result is a slightly smaller and consistently higher-fidelity build than V1 on the most sensitive expert and attention tensors. MaxOptima V2 Strategy: Embeddings and output layers are locked at Q6_K. Router inputs, norms, scales, and biases are kept at full precision (F32) to preserve routing integrity. All attention projections (attn_q, attn_k, attn_v, attn_output) are kept at IQ5_K across every layer. The bulk of the expert FFNs use IQ4_KS; tensors whose row width is not divisible by 256 (ffn_down_exps, dense ffn_down) automatically fall back to the native 32-block IQ4_KS_R16. Boundary expert layers are protected at IQ5_K. View Full MaxOptima V2 Custom Quant Configuration ^output\.weight=Q6_K,^token_embd\.weight=Q6_K,ffn_gate_inp\.weight=F32,rope_freqs\.weight=F32,.*_norm\.weight=F32,.*\.scale=F32,.*_scale\.weight=F32,.*\.bias=F32,attn_k\.weight=IQ5_K,attn_v\.weight=IQ5_K,attn_output\.weight=IQ5_K,attn_q\.weight=IQ5_K,blk\.(0|1|2|3|4|25|26|27|28|29)\.ffn_gate_up_exps\.weight=IQ5_K,ffn_gate_up_exps\.weight=IQ4_KS,ffn_down_exps\.weight=IQ4_KS,ffn_down\.weight=IQ4_KS,ffn_(gate|up)\.weight=IQ5_K,.*=IQ4_KS Available Quants MaxOptima V2 // Custom MoE Mix MaxOptima V2 Custom mix. F32 routing/scales, IQ5_K attention across all layers, IQ4_KS expert FFNs with native IQ4_KS_R16 fallback on non-256-divisible tensors. Best quality-to-size ratio for MoE inference. MaxOptima V1 // Custom MoE Mix (Legacy) MaxOptima V1 Custom mix. F32 routing/scales, IQ5_K/IQ4_KSS attention, IQ4_KSS/IQ4_NL expert FFNs. Superseded by V2; kept for existing downloads and reference. Standard IQK // High & Mid Fidelity IQ5_K Higher fidelity 5-bit. Best quality-to-size for systems with sufficient memory. IQ4_K Standard 4-bit IQK. Uses 256-value super-blocks for scaling. IQ4_KS "Small" 4-bit variant with tighter packing. Lower memory footprint. IQ4_KS_R4 Row-interleaved variant of IQ4_KS, optimized for ik_llama.cpp's matmul kernels. Trellis // Alternative 4-Bit IQ4_KT Dynamic programming trellis quant. Uses a fundamentally different error-minimization strategy than IQK. Imatrix Specification This release utilizes the exact same V3 Calibration Dataset used across previous Austriani releases. No datasets were changed. The calibration was executed at 500 cycles per chunk with a 1024-token context window, processing approximately 1000 chunks. Compatibility These GGUF files are built exclusively for ik_llama.cpp. IQK, Trellis, and MaxOptima quants will not load in mainline llama.cpp. Please refer to the official base repository for mainline GGUF compatibility. Austriani // Kitchoon-Final-26B-A4B-RP // ik_llama.cpp Base: SubMaroon/Kitchoon-Final-26B-A4B