Gemma-4-12B-it-Uncensored-Heretic-MXFP4-GGUF

MXFP4 GGUF quantization of llmfan46/gemma-4-12B-it-uncensored-heretic - an uncensored/heretic (abliterated) finetune of Google's Gemma 4 12B with vision support.

About MXFP4

MXFP4 (Microscaling FP4) is an open standard (OCP) 4-bit floating point format (E2M1) supported by NVIDIA, AMD, Microsoft, and Meta.

• Works on any GPU with MXFP4 support
• Open standard - not vendor locked
• Block-scaled format with shared scale factors

When to use MXFP4 vs other formats:

• MXFP4 - Open standard, broad hardware support
• NVFP4 - NVIDIA Blackwell native, best performance on RTX 50-series
• Q4_K_M - Best for pre-Blackwell GPUs and CPU inference

Files

Quantization Details

Model Description

This is an abliterated (uncensored/heretic) finetune of Google's Gemma 4 12B, a multimodal model with both text and vision capabilities. The original model was finetuned to remove safety alignment restrictions while maintaining the model's core capabilities.

Gemma 4 features a hybrid attention architecture with alternating sliding window and full attention layers, native vision encoding, and tool calling support.

Usage

llama.cpp CLI

# Text only
./llama-cli -m gemma4-12b-heretic-mxfp4.gguf -p "Hello" -n 100

# With vision (requires mmproj)
./llama-server -m gemma4-12b-heretic-mxfp4.gguf \
  --mmproj mmproj-gemma-4-12b-heretic-f16.gguf \
  --host 0.0.0.0 --port 8080 -ngl 99

huggingface-hub

from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="FreedomAISVR/Gemma-4-12B-it-Uncensored-Heretic-MXFP4-GGUF",
    filename="gemma4-12b-heretic-mxfp4.gguf"
)
mmproj_path = hf_hub_download(
    repo_id="FreedomAISVR/Gemma-4-12B-it-Uncensored-Heretic-MXFP4-GGUF",
    filename="mmproj-gemma-4-12b-heretic-f16.gguf"
)

Quantization Pipeline

1. Download source: llmfan46/gemma-4-12B-it-uncensored-heretic
2. Convert to F16 GGUF: convert_hf_to_gguf.py --outtype f16
3. Extract mmproj: convert_hf_to_gguf.py --mmproj --outtype f16
4. Quantize text: llama-quantize input-f16.gguf output-mxfp4.gguf MXFP4

Hardware Requirements

License

Apache 2.0 (inherited from base model)