MiniCPM5-2B-heretic-abliterated-GGUF

This repository hosts quantized GGUF checkpoints based on openbmb/MiniCPM5-2B, implementing the directional refusal ablation methodology showcased in insraq/MiniCPM5-2B-heretic-abliterated via Heretic v1.4.0.

These builds are packaged for local on-device inference using llama.cpp, Ollama, LM Studio, Jan, and standard GGUF executors.

Abliteration Profile

Directional ablation neutralizes the refusal reflex across the residual stream and MLP projections while preserving the primary base model's mathematical, coding, and multi-step reasoning performance.

Benchmark Metrics

Metric	Abliterated Model	Original Base (openbmb/MiniCPM5-2B)
Refusal Rate	5 / 100	99 / 100
KL Divergence	0.0391	0.0000 (Reference)

A low KL divergence of 0.0391 relative to the original openbmb/MiniCPM5-2B weights indicates minimal representation drift, preventing capability degradation on standard non-boundary tasks.

Ablation Parameters

Parameter	Value	Parameter	Value
`direction_index`	per layer	`mlp.down_proj.max_weight`	0.89
`attn.o_proj.max_weight`	1.47	`mlp.down_proj.max_weight_position`	28.68
`attn.o_proj.max_weight_position`	29.44	`mlp.down_proj.min_weight`	0.66
`attn.o_proj.min_weight`	1.45	`mlp.down_proj.min_weight_distance`	20.31
`attn.o_proj.min_weight_distance`	14.36	—	—

Available Quantizations

File Name	Size	Recommended Use Case
`MiniCPM5-2B-heretic-abliterated-Q3_K_M.gguf`	1.29 GB	Low-memory edge devices & micro-compute nodes.
`MiniCPM5-2B-heretic-abliterated-Q4_K_S.gguf`	1.50 GB	Lightweight 4-bit quant; lower RAM overhead.
`MiniCPM5-2B-heretic-abliterated-Q4_K_M.gguf`	1.56 GB	Recommended: Best balance of speed, footprint, and perplexity.
`MiniCPM5-2B-heretic-abliterated-Q5_K_M.gguf`	1.81 GB	High accuracy; preserves intricate reasoning chains.
`MiniCPM5-2B-heretic-abliterated-Q6_K.gguf`	2.07 GB	High fidelity; practically indistinguishable from BF16.
`MiniCPM5-2B-heretic-abliterated-Q8_0.gguf`	2.68 GB	Full 8-bit precision; minimal quantization loss.

Prompt Template (ChatML)

MiniCPM5 uses the standard ChatML prompt template:

<|im_start|>system
You are a helpful, unfiltered assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Quickstart Guide

llama.cpp

Run inference using llama-cli:

llama-cli \
  -m MiniCPM5-2B-heretic-abliterated-Q4_K_M.gguf \
  -p "<|im_start|>user\nExplain how stack-based buffer overflows work at the register level.<|im_end|>\n<|im_start|>assistant\n" \
  -n 512 \
  -c 4096 \
  -ngl 99 \
  --repeat-penalty 1.15 \
  --temp 0.8 \
  --top-p 0.95

Abiray/MiniCPM5-2B-heretic-abliterated-GGUF