Muse-Glimmer-30B-heretic-GGUF

GGUF quantizations of richardyoung/Muse-Glimmer-30B-heretic, a reproducible Heretic abliteration of Meta's meta-models/Muse-Glimmer-30B, a 29.6B dense vision-language model with controllable reasoning and tool use. Includes the vision projector (mmproj-Muse-Glimmer-30B-heretic-F16.gguf), so image input works in llama.cpp and Ollama.

Ollama: richardyoung/muse-glimmer-30b-heretic · Full weights + reproducibility files: richardyoung/Muse-Glimmer-30B-heretic

Abliteration Results

The very low KL divergence (0.063, far below the 0.5 “damage” threshold) means the model retains essentially all of its original capabilities and coherence.

Note on refusals: this build still refuses 52 of 100 harmful prompts, more than our other Heretic releases. That is deliberate: the trials that refused less also degraded the model's normal behavior (KL divergence above 0.10), and we chose to keep the original model's capabilities intact rather than trade them for fewer refusals.

How refusals were counted. Muse Glimmer always writes a reasoning block (to=self) before its answer. This run skips that block with an empty, closed reasoning block (the same approach Heretic uses for gpt-oss) and scores the model's actual answer. Scoring the first 100 tokens of the reasoning instead finds far fewer refusal phrases (as few as 59/100 for the unmodified model), so refusal counts from runs that did that are not directly comparable with these.

Heretic run: 200 trials, trial 116 chosen (fewest refusals with KL ≤ 0.10). Settings, seed and package versions are in reproduce/.

Quick Start

With Ollama (0.32.8 or newer)

ollama run richardyoung/muse-glimmer-30b-heretic

Use the Ollama library tags above: they set Muse Glimmer's chat renderer and parser (glimmer), which handle its reasoning and tool-call format. A plain ollama run hf.co/... pull of these GGUFs gets a raw prompt template instead, so chat formatting and thinking won't work properly.

Images: drag a file into the chat or put its path in the prompt (What does this chart show? ./chart.png). Reasoning is on by default (strength high); in the API set "think": "low" for faster answers, or "think": false to skip it.

With llama.cpp

hf download richardyoung/Muse-Glimmer-30B-heretic-GGUF \
    --include "*Q4_K_M*" "mmproj*" --local-dir ./models

# image + text
llama-mtmd-cli -m ./models/Muse-Glimmer-30B-heretic-Q4_K_M.gguf \
    --mmproj ./models/mmproj-Muse-Glimmer-30B-heretic-F16.gguf \
    --image document.png -p "Transcribe all the text in this image." --jinja -ngl 99

# OpenAI-compatible server
llama-server -m ./models/Muse-Glimmer-30B-heretic-Q4_K_M.gguf \
    --mmproj ./models/mmproj-Muse-Glimmer-30B-heretic-F16.gguf --jinja -ngl 99 -c 16384

Use a recent llama.cpp build (one that includes the muse-glimmer architecture). Put Reasoning strength: low (or medium / high / xhigh) in the system prompt to control how long it thinks.

With Python (llama-cpp-python)

Needs a llama-cpp-python release whose bundled llama.cpp includes Muse Glimmer support.

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="richardyoung/Muse-Glimmer-30B-heretic-GGUF",
    filename="*Q4_K_M*",
    n_gpu_layers=-1,
    n_ctx=16384,
)
output = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain abliteration in simple terms."}]
)
print(output["choices"][0]["message"]["content"])

Recommended sampling (from Meta's card): temperature 1.0, top_p 0.95, top_k 64.

Available Quantizations

All quants except Q8_0 use an importance matrix (computed from the Q8_0 on bartowski's calibration_datav3).

Tested before release: Q4_K_M + mmproj in llama.cpp (llama-mtmd-cli) and Ollama 0.35 transcribed a test invoice exactly and identified the shapes and colors in a test image; chat, reasoning and stop behavior were checked in Ollama.

What is Abliteration?

Abliteration removes the "refusal direction" from a model's residual stream, a surgical weight edit that disables refusals without retraining. See Refusal in Language Models Is Mediated by a Single Direction. Heretic automates it: it searches ablation parameters with an optimizer (200 trials here) and keeps the trials that best trade off fewer refusals against KL divergence from the original model. Only 27 matrices changed (18 attention output and 9 MLP down projections in the language model); the vision encoder and projector are bit-identical to Meta's.

Intended Use

Research, creative writing, education on alignment techniques, and unrestricted local inference. This model will comply with many requests the original refuses. Use responsibly and in accordance with applicable laws. The original model ships with Meta's Muse Glimmer Usage Policy; the weights are licensed Apache-2.0, and you are responsible for making sure your use complies with the licence, that policy, and applicable laws.

Other Models by richardyoung

• New Heretic models (reproducible, listed in the Heretic Grimoire): Muse-Glimmer-30B | LFM2.5-1.2B | Granite-4.2-3B | Granite-4.2-8B | Olmo-3-7B-Think | Phi-4-mini | DeepSeek-R1-0528-Qwen3-8B | Agents-A1-4B
• Abliterated/Uncensored models: Qwen2.5-7B | Qwen3-14B | DeepSeek-R1-32B | Qwen3-8B
• OCR & Vision: MiniCPM-V-4.6 GGUF | dots.mocr GGUF | Unlimited-OCR GGUF | olmOCR GGUF
• MLX quantizations (Apple Silicon): Kimi-K2 series | olmOCR MLX
• Healthcare/Medical: Synthea 575K patients dataset | CardioEmbed
• Research: LLM Instruction-Following Evaluation (arxiv:2510.18892)

---

Built & maintained by Richard Young · DeepNeuro