Gemma-4-12B-Heretic-Abliterated-8Bit-MLX
This repository features an Apple Silicon native 8-bit MLX quantization of culturerevolt/gemma-4-12b-heretic-abliterated.
The base model is an abliterated, fully decensored variant of Google's unified multimodal gemma-4-12b-it architecture, stripped of categorical refusal alignments via norm-preserving directional ablation. This configuration is compiled explicitly for high-performance inference on Mac hardware using the MLX framework.
📊 Format & Hardware Recommendations
This file uses uniform group-wise quantization optimized for unified memory architectures. Every tensor is split into precise channel blocks to isolate and protect outlier weights naturally.
- Quantization Configuration: 8-bit weights compiled with a tight group size of 32.
- Target Environment: This represents the premium reference tier for Apple Silicon setups with 16GB or 24GB of unified memory. It preserves near-flawless parity with the original precision weights while maximizing inference throughput.
⚙️ Recommended Inference Settings
To ensure smooth generation, clean formatting, and proper instruction adherence in native Mac backends like LM Studio or the MLX command line tool, implement the following structures:
1. Multi-Modal Native Capabilities
Gemma-4 is built on a unified, encoder-free architecture. It processes visual patches and audio waveforms natively without requiring an external vision transformer module. Ensure your choice of frontend is configured to look for the companion Gemma-4-12B scale multimodal projector artifacts to fully enable media ingestion capabilities.
📜 Acknowledgements
- Google DeepMind for pioneering the unified Gemma-4 architecture.
- Philipp Emanuel Weidmann for developing the underlying Heretic abliteration framework.
- Massive thanks to the open-source local AI community for continuously pushing the boundaries of what is possible on local consumer hardware.
⚠️ Disclaimer & Boundary Limits
This model is completely unaligned. It will output text without filtering, judgment, or warning labels. By downloading this model, you accept full responsibility for the prompts fed to it and the text generated by it. Use responsibly within local sandbox development setups.
🎛️ Streamlined Jinja Chat Template
If you encounter interface parsing issues with heavy multi-turn configurations, use this clean, hyper-efficient template to ensure maximum token stability:
{%- for message in messages %}
<|turn|>{{ message['role'] }}
{{ message['content'] }}<|turn|>
{%- endfor %}
{%- if add_generation_prompt %}
<|turn|>assistant<|channel>thought <channel|>
{%- endif %}