Muse-Glimmer-30B-Heretic-Uncensored-GGUF

Abstract

This repository disseminates a comprehensive suite of 27 GGUF (GPT-Generated Unified Format) quantizations of Muse-Glimmer-30B-Heretic-Uncensored, an abliterated derivative of Meta's Muse-Glimmer-30B produced with Heretic v1.4.0 by darkc0de. The release spans a memory footprint of 6.53 GB (IQ1_S) to 55.73 GB (F16/BF16), thereby accommodating deployment across heterogeneous hardware, from CPU-only systems to 32 GB VRAM accelerators. All sub-4-bit quantizations were calibrated with an importance matrix (imatrix) to mitigate quality degradation. The fidelity of the Q4_K_S variant relative to the F16 reference is quantified via token-level perplexity on a fixed evaluation set. The model exhibits zero refusals on adult/NSFW content; this property was empirically verified.

1. Model Overview

2. Quantization Rationale

The quantization selection is derived from the model's architectural properties and the memory constraints of representative target hardware. The model employs grouped-query attention with 2 KV heads, which renders the KV cache exceptionally compact:

• 52 layers × 2 KV heads × 128 head_dim = 52 KiB/token at FP16
• Q8_0 KV cache: 26 KiB/token → 64K context ≈ 1.7 GB
• Q4_0 KV cache: 13 KiB/token → 64K context ≈ 0.85 GB

2.1 Selection Guidance

3. Artifact Inventory

The repository contains 27 GGUF files. Reported sizes correspond to the stored artifacts:

4. Methodology

All artifacts were derived from the original safetensors using llama.cpp master (commit 030ebb5). Support for the muse_glimmer architecture is absent from release binaries prior to b10344; consequently, a master-branch build was required.

4.1 Conversion

python convert_hf_to_gguf.py darkc0de/Muse-Glimmer-30B-heretic \
  --outfile Muse-Glimmer-30B-Heretic-F16.gguf

4.2 Imatrix Calibration

Importance-matrix calibration was applied to all sub-4-bit quantizations. The calibration corpus comprised 128 chunks drawn from wiki.raw:

llama-imatrix -m Muse-Glimmer-30B-Heretic-F16.gguf \
  -f wiki.raw -o imatrix.dat -c 131072

4.3 Quantization

The --imatrix flag must precede the output specification; the parser does not recognize it in trailing position.

llama-quantize --imatrix imatrix.dat \
  Muse-Glimmer-30B-Heretic-F16.gguf \
  Muse-Glimmer-30B-Heretic-IQ3_XS.gguf IQ3_XS

5. Fidelity Benchmark

Consistent with the repository's scope, this page does not report aggregate capability scores (e.g., MMLU); such evaluations are documented in the upstream card. The benchmark herein quantifies quantization-induced degradation: the fidelity of the Q4_K_S variant relative to the F16 reference, measured as token-level perplexity over a fixed evaluation set.

Experimental setup. wikitext-2 test split; 32 chunks; n_ctx=2048; batch 2048; 16 CPU threads (the build machine was not equipped with a GPU). Both variants were evaluated on an identical token set such that systematic errors cancel.

Interpretation. A relative increase of +2.47% in perplexity is at or below the typical expectation for a Q4_K_S-class quantization of a 30B model, indicating successful quantization with minimal fidelity loss. Sub-4-bit quantizations (IQ/TQ series) incur greater fidelity loss in exchange for substantially reduced memory footprints; imatrix calibration recovers a meaningful portion of this gap.

6. Inference Performance

6.1 GPU (NVIDIA RTX 4080 16 GB; CUDA 13.3; llama.cpp b10355)

The model was fully offloaded (-ngl 99) with 16 CPU threads for batch processing. Q4_K_S occupies 15.01 GiB, within the 16 GB VRAM budget.

6.2 CPU-only (reference build, 16 threads)

6.3 Operational Notes

• Generation throughput of ~39 tokens/s on an RTX 4080 is sufficient for real-time agentic interaction.
• Prompt ingestion of ~2,300 tokens/s renders long-document processing non-bottlenecked.
• The 15.01 GiB model leaves ~1.4 GB VRAM headroom on a 16 GB card; with --cache-type-k/v q8_0, a 64K context adds ~1.7 GB, remaining within budget.

# GPU inference (llama.cpp b10355+, CUDA build)
llama-cli -m Muse-Glimmer-30B-Heretic-Q4_K_S.gguf \
  -ngl 99 -c 65536 --cache-type-k q8_0 --cache-type-v q8_0

For agentic deployments, the upstream card's recommended sampling configuration applies (temperature = 1.0, top_p = 0.95, top_k = 64; Reasoning strength: high for complex tasks).

7. Limitations and Responsible Use

• Abliterated model. The model exhibits reduced refusal behavior by design and may generate content that is inappropriate, offensive, or unsafe for many applications. Deployment should incorporate appropriate guardrails and human oversight, particularly in agentic or real-world-action contexts.
• Age restriction. Not intended for use by individuals under 18 years of age. Deployers operating in environments accessible to minors bear responsibility for regulatory compliance.
• Scope of evaluation. This repository provides a quantized artifact and its fidelity benchmark; it does not constitute a safety or capability evaluation of the model.
• Quantization edge cases. Quantized inference may deviate from full precision in edge cases (see Section 5 for aggregate fidelity).
• Multimodal note. Benchmarks were conducted in text-only mode; the perception encoder was not exercised.

8. License and Attribution

This work is a derivative of:

1. meta-models/Muse-Glimmer-30B — © Meta Superintelligence Lab, released under Apache 2.0.
2. darkc0de/Muse-Glimmer-30B-heretic — abliterated derivative produced with Heretic v1.4.0, retaining the Apache 2.0 license.

The GGUF conversion and quantization in this repository inherit the Apache 2.0 license. See LICENSE for the full text.

9. Citation

@misc{meta2026museglimmer,
  author = {Meta Superintelligence Lab},
  title = {Muse Glimmer: A 30B Multimodal Agentic Model for Local Deployment},
  year = {2026},
  url = {https://huggingface.co/meta-models/Muse-Glimmer-30B}
}

@misc{darkc0de2026heretic,
  author = {darkc0de},
  title = {Muse-Glimmer-30B-heretic},
  year = {2026},
  url = {https://huggingface.co/darkc0de/Muse-Glimmer-30B-heretic}
}

@misc{observerx2026gguf,
  author = {0bserverx},
  title = {Muse-Glimmer-30B-Heretic-Uncensored-GGUF},
  year = {2026},
  url = {https://huggingface.co/0bserverx/Muse-Glimmer-30B-Heretic-Uncensored-GGUF}
}

Research use and responsibility

This repository is intended for legitimate research and controlled evaluation, including interpretability, alignment and refusal-behavior analysis, red-team testing, and robustness work. It is not a ready-made production safety layer. If you deploy the model or expose it to other users, you are responsible for adding suitable access controls, moderation, monitoring, and abuse prevention.

Use of these files is subject to the Apache License 2.0 and all applicable laws. You are responsible for how you operate the model and for outputs produced in your environment. To the extent permitted by law, the repository maintainers and upstream authors accept no liability for misuse or resulting harm. Generated outputs are not statements or endorsements by the maintainers, upstream creators, or their organizations.