ExLlamaV3 4.0 bpw quantization of richardyoung/gemma-4-31B-it-ARA-heretic, a Heretic v2.0.0 Arbitrary-Rank Ablation (ARA) of google/gemma-4-31B-it. All credit for the abliteration goes to Richard J. Young; see the original card for method, parameters and the accompanying paper.
| Metric (from the original card) | ARA heretic | google/gemma-4-31B-it |
|---|---|---|
| Refusals | 28/100 | 99/100 |
| KL divergence | 0.0845 | 0 |
| WildGuard refusal (972 harmful prompts) | 0.6% | 89.3% |
| Capability change (MMLU-Pro, GSM8K, IFEval, HumanEval, MBPP) | −2.18 pp | – |
mul1, out_scales autoconfig.json is not the one from the source repo. The source config was saved by transformers 5.18, which stores Gemma 4's global-attention layers as per_layer_config; exllamav3 does not read that and fails on layer 5 (q_proj 16384 vs 8192). It was replaced with the stock google/gemma-4-31B-it config, which describes the same architecture (identical tensor names and shapes). The original is kept as config.ara-orig.json. All other files are unchanged from the source.
Weights follow the Gemma 4 license of the base model. The source release additionally attaches the OpenRAIL++ use-based restrictions in LICENSE_OpenRAIL++.md, which apply to this quant as a derivative work. Where the two conflict, the base licence prevails.
model-00001-of-00005.safetensors 4.3 GBmodel-00004-of-00005.safetensors 4.2 GBmodel-00003-of-00005.safetensors 4.2 GBmodel-00002-of-00005.safetensors 4.1 GBmodel-00005-of-00005.safetensors 2.3 GBrichardyoung/gemma-4-31B-it-ARA-heretic