Austriani // GGUF Release // llama.cpp
Kitchoon-Final-26B-A4B
MoE Roleplay Quantization // V3 Calibration Imatrix
26B
Total Params
4B
Active Params
500
Cycles
1024
Context
The Base Model
This repository hosts GGUF quantizations of SubMaroon/Kitchoon-Final-26B-A4B. This is the final iteration of the Kitchoon line, built on the Gemma 4 26B-A4B Mixture-of-Experts architecture (26B total, 4B active per token). It is specifically tuned for RP and ERP, focusing on character development and a neutral writing style.
The "Final" release bakes in Anthropos v3 at 0.7. This is a LoRA trained via the BACo method to mix the instruction-tuned model with the pre-trained Gemma base, resulting in fewer stock phrases, fewer em dashes, and more diverse swipes without sacrificing formatting or reasoning. Thinking mode is preserved and fully functional.
Quantization Matrix // Full Spectrum
Status: Release Complete. The imatrix was generated using the V3 Calibration Dataset. All baseline quants from Q8_0 down to IQ1_S are available. Click any tier below to expand the exact formats.
8-Bit & 6-Bit Tier // Near Lossless
Q8_0
Absolute maximum quality. Virtually indistinguishable from the original f16 weights. Large file size, requires significant VRAM/RAM.
Q6_K
Excellent high-fidelity baseline. Extremely low perplexity loss. Perfect for users with ample memory who want the best possible output.
5-Bit Tier // High Fidelity
Q5_K_M
The "Medium" 5-bit. Recommended for high-quality inference if you have the VRAM. Outstanding balance of quality and size.
Q5_K_S
The "Small" 5-bit. Slightly smaller footprint than M, but maintains most of the high-fidelity performance.
4-Bit Tier // Standard & Advanced
Q4_K_M
The community standard. Highly recommended if you want a standard, cross-compatible quant that works in mainline llama.cpp.
Q4_K_S
Smaller variant of the 4-bit medium. Good for saving a bit of memory with minimal quality drop.
Q4_0
Legacy 4-bit format. Faster on some older hardware, but lower quality than K-quants.
IQ4_XS
Importance-aware 4-bit. Extremely efficient. Often outperforms standard Q4 formats at a smaller size.
IQ4_NL
Non-linear 4-bit. An alternative to IQ4_XS, optimized for different hardware instruction sets.
3-Bit Tier // Aggressive Compression
Q3_K_M
Medium 3-bit. Noticeable degradation but retains the core persona.
Q3_K_S
Small 3-bit. Highly compressed.
IQ3_XXS
Importance-aware 3-bit. Aggressive size reduction while trying to protect critical layers.
IQ3_XS
Slightly larger than XXS, offering better stability.
IQ3_S
Small 3-bit IQ. Good option for edge devices.
IQ3_M
Medium 3-bit IQ. The highest quality in the 3-bit IQ family.
2-Bit Tier // Severe VRAM Constraints
IQ2_XXS
Extreme 2-bit. Only recommended if you absolutely cannot fit anything larger.
IQ2_XS
Slightly larger than XXS, but still highly experimental for RP.
IQ2_S
Small 2-bit. Quality degradation will be very noticeable.
IQ2_M
Medium 2-bit. Pushes the absolute limits of acceptable quality.
1-Bit Tier // Experimental Extreme
IQ1_M
Extreme 1-bit. Highly experimental. The model will likely lose coherence quickly.
IQ1_S
Absolute smallest possible size. For testing only. Not recommended for actual roleplay.
Imatrix Specification
The importance matrix is calibrated on the same V3 dataset used in previous releases: 100% uncensored and roleplay data. General knowledge, code, and polyglot data are excluded to keep the weights focused on creative inference.
Note on Low-Bit Quants: Extreme low-bit quantization on a 26B MoE model is highly experimental. While the V3 imatrix helps preserve roleplay formatting and character voice, quality degradation is expected below 4-bit. Use Q4_K_M or IQ4_XS for the best balance of size and performance.
Compatibility
These are standard baseline GGUF files, fully compatible with mainline llama.cpp, LM Studio, KoboldCPP, and Ollama. No custom forks or special builds are required.
Austriani // Kitchoon-Final-26B-A4B-RP-GGUF
Base: SubMaroon/Kitchoon-Final-26B-A4B // V3 Imatrix