Quantized by Austrian_Conqueror [Austriani]

NEMOTRON BOLT

8B (4B Effective) · Gemma-4 · ik_llama.cpp “Edge intelligence. Uncensored.” 4B EFFECTIVE64K CONTEXTTOOL CALLINGTRELLIS QUANTS ◖◗Release Status All quantization variants have been computed and uploaded. The full set is available in the files tab. These files are specifically built for ik_llama.cpp.☰Available Files Files follow the naming convention: Nemotron-3.5-Bolt-E4B-Heretic-{QuantType}.gguf File TypeDescription .imatrixThe raw importance matrix file. Provided for anyone who wishes to generate their own custom quants. IQ4_KTTrellis quant. Novel integer-based trellis quantization offering superior compression and accuracy trade-offs. IQ4_KStandard 4-bit imatrix quant. IQ4_KS4-bit imatrix, optimized for size/quality balance. IQ4_KS_R4Alternative 4-bit imatrix variant (R4). IQ5_KHigher fidelity, heavier memory footprint. ⚡The Process I

Imatrix Optimization

Reap V1 Dataset

To ensure the best possible quality at lower bitrates, the standard quants (IQ4_K, IQ4_KS, IQ5_K) were built using a freshly computed imatrix.

For this release, the imatrix was calculated using the Reap V1 dataset. This allows the quantization process to prioritize the most critical weights and activation patterns, preserving the model's native reasoning and tool-calling abilities while compressing the ones that matter less.

II

Trellis Quantization

The IQ4_KT variant

IQ4_KT is not a standard imatrix quant. It uses a novel integer-based trellis quantization method unique to the ik_llama.cpp ecosystem.

Unlike traditional quants that use a fixed grid, Trellis quantization uses a dynamic programming approach to find the optimal sequence of discrete values. It achieves superior model compression and accuracy trade-offs, making it an excellent choice for those who want maximum quality retention at 4-bit.

III

Abliteration

What makes this "Heretic"

This is a decensored version of NVIDIA-Nemotron-3.5-Bolt-E4B, made using Heretic with the Arbitrary-Rank Ablation (ARA) method.

The ablation targets layers 12 through 27. The result is a refusal rate drop on JailbreakBench Harmful prompts from 89/100 to 3/100, with a KL divergence of only 0.0859. It handles aggressive prompts and complex logic without unnecessary safety interruptions, while retaining the base model's tool-calling capabilities.

⚡Architecture & Specs Built on NVIDIA's Nemotron-3.5-Bolt (post-trained from Gemma-4-E4B), optimized for edge devices. SpecificationValue Total Parameters8B (4B Effective) ArchitectureGemma-4-E4B (NVIDIA Post-train) Context Length64,000 tokens Refusal Rate (Harmful)Reduced from 89% to 3% Ablation Layers12 through 27 (ARA)

◆  The Real-World Takeaway

With 8B total parameters but only 4B active per token, this model is designed for fast local inference. The reduced memory footprint of these quants prevents severe generation bottlenecks caused by RAM swapping. For agentic loops and edge deployment, this consistent throughput is vital.

◖◗Credits

◆  grumd0: For creating the Nemotron-3.5-Bolt-E4B-Heretic model and the original ablation recipe.

---

◆  NVIDIA: For the base Nemotron-3.5-Bolt model and the post-training methodology.

---

◆  Google: For the upstream Gemma-4-E4B foundation model.

◖ · ◗