Gemma-4-31B-Heretic

Directional ablation release of google/gemma-4-31B-it using the Heretic Bayesian ablation framework.

Run Details

Base Model: google/gemma-4-31b-it (60 dense transformer layers, ~30.7B parameters).
Optimization: 80 trials using Optuna TPE. Refusal directions were ablated across adversarial security and roleplay probes while anchoring on math, coding, and reasoning splits to prevent capability loss.
Metrics:
- Baseline refusal rate: 86.25% (69/80 test prompts).
- Selected checkpoint (Trial 48): Refusals dropped to 41.25% (33/80 test prompts) with KL divergence of ~1.22 against the unablated base model.
- Low-KL alternative (Trial 50): 45.0% refusals with KL divergence of ~0.89.
Architecture Fix: Layer 5 in Gemma 31B uses mixed attention dimensions (head dimension 512, 4 key-value heads). The included config.json explicitly populates global_head_dim: 512 and num_global_key_value_heads: 4 inside text_config so downstream quantizers (e.g. ExLlamaV3) load the global projection tensors without shape mismatch errors.
Included Files: Master BF16 SafeTensors (13 shards, ~62 GB) and the full Optuna study log (optuna_study.jsonl).