google/gemma-3-1b-it with its refusal behaviour removed by
Heretic 2.x — training-free modification via Arbitrary-Rank Ablation (ARA)
with TPE hyperparameter search.
Produced on a single 8 GB Apple M1 MacBook Air: 100 trials, 3 h 21 min, MPS backend, batch size 64.
Heretic's scorers, mlabonne/harmful_behaviors (100 prompts) and mlabonne/harmless_alpaca (100 prompts):
| Metric | Base (bf16) | This model (trial 97) |
|---|---|---|
| Refusals | 98/100 | 3/100 |
| KL divergence | 0 (by definition) | 0.0180 |
Selected trial parameters: start_layer_index=8, end_layer_index=23, preserve_good_behavior_weight=0.7985, steer_bad_behavior_weight=0.0001, overcorrect_relative_weight=0.8775, neighbor_count=4.
Independent check through llama.cpp (greedy, model chat template, 6 prompts from harmful_behaviors):
| Arm | Refusals | Normal queries |
|---|---|---|
| base, bf16 | 4/6 | intact |
| base, Q4_0 | 6/6 | intact |
| this model, bf16 | 0/6 | intact |
| this model, Q4_0 | 0/6 | intact |
Benign prompt ("explain what a CPU cache is in one sentence") stays correct at both precisions.
Heretic's --trial-index 0 sorts the Pareto front by refusals first, so it auto-selects the zero-refusal
extreme. On this model that trial (36) reached 0/100 refusals at KL 9.8395 with weights up to
2.4e9 in magnitude — it is not merely degraded, it is numerically broken: llama-quantize rejects it
(found inf value … quantized data validation failed) and it emits okay okay okay… on any prompt.
This repository ships trial 97 (3/100 refusals, KL 0.0180) instead, which quantizes cleanly.
| File | Size | Notes |
|---|---|---|
model.safetensors |
1.9 GB | bf16, HF Transformers format |
gguf/gemma-3-1b-it-heretic-Q4_0.gguf |
688 MB | llama.cpp, recommended |
gguf/gemma-3-1b-it-heretic-BF16.gguf |
1.9 GB | llama.cpp, reference precision |
llama-server -m gguf/gemma-3-1b-it-heretic-Q4_0.gguf -ngl 99 --port 8080
Load in bfloat16. In float32 with transformers 5.x, Gemma 3 models of this generation emit a bare
<end_of_turn> and produce no text.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tok = AutoTokenizer.from_pretrained("AJ-Gonzalez/gemma-3-1b-it-heretic")
model = AutoModelForCausalLM.from_pretrained("AJ-Gonzalez/gemma-3-1b-it-heretic", dtype=torch.bfloat16)
msgs = [{"role": "user", "content": "Explain what a CPU cache is in one sentence."}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt", add_special_tokens=False)
out = model.generate(**ids, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True))
google/gemma-3-1b-it (instruction-tuned, bf16), obtained via the unsloth/gemma-3-1b-it mirror.heretic.modifiers.ara.ARA, scorers
keyword_rate.KeywordRate (minimize refusals) and kl_divergence.KLDivergence (minimize KL),
100 trials, --seed 0, --device-map mps, merge export.convert_hf_to_gguf.py (bf16) → llama-quantize Q4_0.Inherits the Gemma Terms of Use from the base model. This is an unofficial modification, not endorsed by Google.
google/gemma-3-1b-it