gemma-4-12B-it-abliterated-mlx-4bit

MLX 4-bit conversion of OpenYourMind/gemma-4-12B-it-abliterated-uncensored for native Apple Silicon inference.

This is an uncensored (abliterated) variant of Google's Gemma 4 12B instruction-tuned model, converted to MLX 4-bit quantization for fast, private, on-device inference on M-series Macs.

Model details

Requirements

• Apple Silicon Mac (M1 or later)
• Unified memory: 16GB minimum, 24GB recommended
• mlx-vlm installed

pip install mlx-vlm

Usage

Text only

from mlx_vlm import load, generate

model, processor = load("IITheLordII/gemma-4-12B-it-abliterated-mlx-4bit")
output = generate(model, processor, prompt="Explain quantum computing simply.", max_tokens=512)
print(output)

Vision (image input)

from mlx_vlm import load, generate

model, processor = load("IITheLordII/gemma-4-12B-it-abliterated-mlx-4bit")
output = generate(
    model,
    processor,
    prompt="Describe what you see in this image.",
    image="path/to/image.jpg",
    max_tokens=256
)
print(output)

CLI

mlx_vlm.generate \
  --model IITheLordII/gemma-4-12B-it-abliterated-mlx-4bit \
  --prompt "Describe this image." \
  --image path/to/image.jpg \
  --max-tokens 256

LM Studio

Search IITheLordII/gemma-4-12B-it-abliterated-mlx-4bit in the LM Studio model browser (MLX filter enabled).

Ollama

ollama run hf.co/IITheLordII/gemma-4-12B-it-abliterated-mlx-4bit

Conversion details

Converted on Apple Silicon M4 Pro (24GB) using:

python3 -m mlx_vlm convert \
  --hf-path OpenYourMind/gemma-4-12B-it-abliterated-uncensored \
  --mlx-path ./gemma4-12b-abliterated-mlx-4bit \
  -q --q-bits 4 --q-group-size 64

Note: mlx-vlm is required (not mlx-lm) to preserve the vision tower. Using mlx-lm would silently drop the vision and audio embedders.

Performance (M4 Pro 24GB)

Disclaimer

This model has its refusal mechanisms removed via abliteration. It may generate sensitive or controversial content. Use responsibly and in accordance with applicable laws and the Gemma license.