Austriani / ik_llama.cpp

Gemma-4-26B-A4B-it

Architectural Quantization Archive 26B Total Params 4B Active Params 1024 Imatrix Ctx 500 Cycles Base Architecture This repository hosts ik_llama.cpp-compatible GGUF quantizations of google/gemma-4-26B-A4B-it. This is Google DeepMind's 26B-parameter Mixture-of-Experts model, activating only 3.8B parameters per token through 128 fine-grained experts and top-8 routing. It is built for cost-efficient inference, native function calling, and multimodal input, supporting a 256K-token context window. The architecture uses 30 layers with a hybrid attention scheme: interleaved local sliding-window attention (1024 tokens) and global attention. It has 16 attention heads over 8 KV heads. The model is released under the Apache 2.0 license. MaxOptima Custom Quant This release includes MaxOptima, a bespoke, surgical quantization mix designed specifically for the 128-expert MoE architecture of Gemma 4. The goal is to protect the expert routing and critical attention layers while aggressively compressing the bulk of the expert FFNs, producing a file significantly smaller than a standard Q4_K_M. MaxOptima Strategy: Embeddings and output layers are locked at Q6_K. Router inputs, norms, scales, and biases are kept at full precision (F32) to preserve routing integrity. Key attention layers are kept at IQ5_K. The bulk of the expert FFNs are compressed to IQ4_KSS and IQ4_NL, with boundary blocks protected at higher precision. View Full Custom Quant Configuration ^output\.weight=Q6_K,^token_embd\.weight=Q6_K,ffn_gate_inp\.weight=F32,rope_freqs\.weight=F32,.*_norm\.weight=F32,.*\.scale=F32,.*_scale\.weight=F32,.*\.bias=F32,blk\.(0|1|2|3|4|5|6|11|17|18|23|24|25|26|27|28|29)\.attn_k\.weight=IQ5_K,attn_k\.weight=IQ4_KSS,attn_v\.weight=IQ5_K,blk\.(0|1|2|3|4|5|11|17|23|25|26|27|28|29)\.attn_output\.weight=IQ5_K,attn_output\.weight=IQ4_KS,attn_q\.weight=IQ5_K,blk\.(0|1|2|3|4|25|26|27|28|29)\.ffn_gate_up_exps\.weight=IQ5_K,ffn_gate_up_exps\.weight=IQ4_KSS,blk\.(0|1|2|3|4|25|26|27|28|29)\.ffn_down_exps\.weight=Q5_0,ffn_down_exps\.weight=IQ4_NL,ffn_down\.weight=Q5_0,ffn_(gate|up)\.weight=IQ5_K,.*=IQ4_KS Available Quants MaxOptima // Custom MoE Mix MaxOptima Custom mix. F32 routing/scales, IQ5_K attention, IQ4_KSS/IQ4_NL expert FFNs. Best quality-to-size ratio for MoE inference. Standard IQK // High & Mid Fidelity IQ5_K Higher fidelity 5-bit. Best quality-to-size for systems with sufficient memory. IQ4_K Standard 4-bit IQK. Uses 256-value super-blocks for scaling. IQ4_KS "Small" 4-bit variant with tighter packing. Lower memory footprint. IQ4_KS_R4 Row-interleaved variant of IQ4_KS, optimized for ik_llama.cpp's matmul kernels. Trellis // Alternative 4-Bit IQ4_KT Dynamic programming trellis quant. Uses a fundamentally different error-minimization strategy than IQK. Imatrix Specification This release utilizes the exact same V3 Calibration Dataset used across previous Austriani releases. No datasets were changed. The calibration was executed at 500 cycles per chunk with a 1024-token context window, processing approximately 1000 chunks. Compatibility These GGUF files are built exclusively for ik_llama.cpp. IQK, Trellis, and MaxOptima quants will not load in mainline llama.cpp. Please refer to the official base repository for mainline GGUF compatibility. Austriani // Gemma-4-26B-A4B-it // ik_llama.cpp Base: google/gemma-4-26B-A4B-it