Austriani // Gemma 4 E2B // ik_llama.cpp

Gemma 4 E2B ARA Abliterated
ik_llama.cpp Quantizations

The ARA abliteration, ported to ik_llama.cpp with a custom uncensored imatrix and MaxOptima. ARA Abliteration ik_llama.cpp V3 Calibration MaxOptima IQ5_K IQ4_K IQ4_KS IQ4_KS_R4 IQ4_KT BF16 500 imatrix Cycles per chunk // higher than typical 2048 Context Window during imatrix computation

This repository contains ik_llama.cpp-compatible GGUF quantizations of Austriani/Gemma-4-E2B-it-ARA-Abliterated. The base model is an ARA abliteration of Gemma 4 E2B, producing a KLD of 0.1004 and a refusal rate of 14/100. The quants below are built with a custom importance matrix computed from a purpose-built uncensored calibration dataset, not a generic pile or wiki-text sample.

Important: IQK and Trellis quants are ik_llama.cpp-only. They will not load in mainline llama.cpp. If you need a mainline-compatible GGUF, use the BF16 release or convert from the source safetensors.

Quantization Options // ik_llama.cpp

The MaxOptima Mix: This is a custom, highly specific layer mix designed to maximize quality while keeping the overall file size smaller than a standard Q4_K_M. It uses a "sandwich" structure: Q6_K for embeddings and output, IQ6_K for the middle attention layers (blk 14), IQ5_KS for the outer attention layers and later FFN layers, and falls back to IQ4_KS and IQ3_KS for the bulk of the FFN weights. This preserves the most critical weights at high precision while aggressively compressing the rest.

Why both IQ4_KS and IQ4_KT? IQ4_KS is an IQK quant: it uses a fixed non-linear grid and a 256-value super-block structure. IQ4_KT is a Trellis quant: it uses dynamic programming to choose the optimal discrete value sequence for each weight block. They are not interchangeable variants; they represent two different approaches to minimizing 4-bit error. Testing both lets you see which one preserves the ARA-abliterated model's behavior better on your hardware.

Imatrix // V3 Calibration Dataset

The importance matrix for these quants was computed from V3 Calibration, a custom dataset derived from OpenIntelligenceNet/Uncensored-Imatrix-Calibration-Mixed-Data. Unlike the full mixed dataset, V3 retains only two domains: Uncensored Alpaca and Multi-Turn Roleplay & Fiction. This keeps the imatrix focused on the behaviors the abliterated model is actually used for, rather than diluting importance across code, math, or factual knowledge.

Composition: 40% v2_stripped / 25% ath / 20% gry / 15% epi. The mix deliberately weights the stripped uncensored Alpaca and roleplay blend highest, then layers in DPO roleplay, character-card roleplay, and synthetic RP. The goal is an importance matrix that reflects uncensored conversational and roleplay inference, not general-purpose text completion.

imatrix Parameters // 500 Cycles, 2048 Context

How this compares to common imatrix practice: Bartowski's widely used calibration_datav3.txt is on the order of 64k tokens and is often run with a smaller context window. mradermacher's guidance recommends a dataset roughly double Bartowski's size, but warns that beyond about 1 MB the returns diminish, and emphasizes that calibration data should match the intended use case. Community testing has also found that a small context size such as 512 often performs as well as or better than 4096 for imatrix generation. This release deliberately diverges from those defaults: 500 cycles and a 2048-token context are heavier settings, paired with a dataset that is purpose-built for uncensored roleplay rather than general-purpose calibration. The trade-off is longer imatrix computation time in exchange for importance weights that are more tightly aligned to the abliterated model's target domain.

Why This Release Exists

The safetensors ARA abliteration is the source of truth. This repository exists because ik_llama.cpp's quant family offers roughly 40% lower quantization error than mainline Q4_K_S at equivalent bits-per-weight, and because a general-purpose imatrix would blur the abliterated model's specific strengths. The V3 calibration and the 500-cycle / 2048-context settings are an attempt to build an importance matrix that knows what this model is for.

If you want the smallest files for edge deployment, start with IQ4_KS or IQ4_KT. If you have more memory and want the highest fidelity quant, use IQ5_K. If you want the best balance of size and quality without the extreme compression of pure 4-bit, use MaxOptima. If you need mainline compatibility or a reference for comparison, use BF16.

Austriani // Gemma 4 E2B ARA Abliterated // ik_llama.cpp Base: Austriani/Gemma-4-E2B-it-ARA-Abliterated // V3 Calibration