Quantized by Austrian_Conqueror [Austriani]

MINICPM5 · HERETIC

2.5B · Llama · GGUF Quants “Small model. Big context. No brakes.” CODE & MATH131K CONTEXTTOOL CALLINGEDGE READY ◖◗Release Status All quantization variants have been computed and uploaded. The full set is available in the files tab.☰Available Quants Files follow the naming convention: MiniCPM5-2B-{QuantType}.gguf Quant TypeDescription BF16Full precision. Reference quality, largest file size. IQ4_KStandard 4-bit imatrix quant. IQ4_KS4-bit imatrix, optimized for size/quality balance. IQ4_KS_R4Alternative 4-bit imatrix variant (R4). IQ4_KTTrellis quant. Novel integer-based trellis quantization offering superior compression and accuracy trade-offs. IQ5_KHigher fidelity, heavier memory footprint. ⚡The Process I

Imatrix Optimization

For standard quants

To ensure the best possible quality at lower bitrates, the IQ4_K, IQ4_KS, and IQ5_K quants were built using a freshly computed imatrix.

The imatrix process analyzes the model's activation patterns to determine which weights are most critical. This allows the quantization to prioritize important tensors while compressing the ones that matter less, preserving the model's reasoning and coding abilities.

II

Trellis Quantization

The IQ4_KT variant

IQ4_KT is not a standard imatrix quant. It uses a novel integer-based trellis quantization method from ik_llama.cpp.

Unlike traditional quants that use a fixed grid, Trellis quantisation uses a dynamic codebook approach. It achieves superior model compression and accuracy trade-offs, and can even offer faster CPU prompt processing. This makes it an excellent choice for those who want maximum quality retention at 4-bit.

III

Specialized Tuning

Coding, Math, and Tools

This model isn't just a general-purpose LLM. It has been specifically tuned on a Coding, Math, and Tool-Calling dataset. If you need a lightweight agent that can write scripts, solve equations, and format JSON function calls, this is the one.

The decensoring process (Abliteration) removed the refusals, meaning it will handle aggressive programming prompts, complex exploit-adjacent logic, and math proofs without unnecessary safety interruptions.

⚡Architecture & Ablation Built on the MiniCPM5-2B standard Llama architecture, optimized for extreme context lengths. SpecificationValue ArchitectureStandardLlamaForCausalLM Parameters2,516,756,480 Context Length131,072 tokens Refusal RateReduced from 99% to 4% Ablation Layers9 through 32 (ARA)

◆  The Real-World Takeaway

While raw compute speed is slightly lower per token than a non-imatrix quant, the reduced memory footprint prevents severe generation bottlenecks caused by RAM swapping. For agentic loops and long-context coding, this consistent throughput is vital.

◖◗Credits

◆  dalatexcoder: For creating the MiniCPM5-2B-Heretic-ARA model and the original ablation recipe.

---

◆  OpenBMB: For the base MiniCPM5-2B model and the UltraData training suite.

◖ · ◗