Muse-Glimmer-30B, surgical: 6.34% of the decoder removed, healed, Q8_0

Research artifact, not a general-purpose release.

• What was changed: FFN sublayers of layers 4, 7, 9 and 48 removed (6.34% of the decoder), healed by local distillation, quantised to Q8_0.
• Why it exists: the larger-surgery sibling of the Q4_K artifact from the same preregistered study; Q8_0 keeps the fidelity headroom that lets it pass.
• Parent model: meta-models/Muse-Glimmer-30B
• Useful for normal inference? passes the fidelity bar (98.34% / 0.03883) with the same benchmark-annex caveat as its sibling.
• Experiment: Model Surgery and Scale Recovery
• Evidence: the surgery study (dataset), SURGERY.json and SHA256SUMS in this repo
• Runner compatibility: any Runner release; standard tensor shapes.

A 27.58 GB GGUF of Meta's Muse-Glimmer-30B with the FFN sublayers of layers 4, 7, 9 and 48 structurally removed (weights exactly zero, standard tensor shape), the damage repaired by local distillation against the frozen parent on a decontaminated corpus, and the result quantized to Q8_0.

It passes a strict fidelity bar against the unmodified BF16 parent on a held-out split: **margin-qualified top-1 agreement 98.34% (bar:

=97%) and mean KLD 0.03883 (bar: <=0.05)**.

This is the larger-surgery sibling of Muse-Glimmer-30B-Surgical-Q4_K-GGUF (4.75% removed, 14.61 GB): more of the decoder is gone here, and Q8_0 keeps the fidelity headroom that makes it pass.

Serve with xyntetik-runner — the engine that trains, merges, and quantizes the GGUF it serves

This GGUF is built to be served with xyntetik-runner, a single-binary, dependency-free C11 inference engine (CPU / CUDA / Metal, OpenAI-compatible server). For this artifact the relevant differentiator is the training side:

Runner trains, merges, and quantizes GGUFs deterministically: the same data and seed reproduce the same adapter file byte for byte, and every written artifact carries a provenance record.

One binary covers the loop: --train (AdamW LoRA on the served GGUF, deterministic by default, gated in CI by adapter sha256), --lora to serve the adapter beside the frozen base, --merge-lora to fold it in with per-tensor requantization and a merge.json provenance record, and --score for teacher-forced held-out evals. Adapters interoperate both ways: a Runner-trained adapter scores identically (1.000 on its held-out eval) served by stock llama.cpp, and community F16 adapters load straight back — measured, not assumed.

Measured — the finding every fine-tuner should know before merging (held-out eval where the fine-tune scores 1.00 and the base model 0.69; verified in two independent runtimes):

A 4-bit merge silently deletes the adaptation while perplexity still looks fine. Merge to Q8_0 or F16, or keep serving base + adapter.

This artifact is that toolchain eating its own cooking. The surgery was applied by byte-level substitution into the parent GGUF, and the Q8_0 file was written by Runner's own quantizer. Quantization noise and surgery error were measured to be additive and independent across five configurations (0.965 to 0.996 times the naive sum), so this Q8_0's cost was predictable before it was built: +0.00026 KLD over the BF16 surgery.

Quickstart

# the same binary that wrote this artifact:
runner -m model.gguf --train data.jsonl --train-out adapter.gguf
# same data + seed -> byte-identical adapter, build provenance in adapter.gguf.train.json
runner -m model.gguf --lora adapter.gguf --merge-lora merged --quant q8_0
# provenance lands in merged.gguf.merge.json; do not merge to 4-bit (table above)

The exact boundary of the determinism claim, including what is deliberately not promised: docs/determinism-scope.md.

Run it

This file is 27.58 GB, so it wants a machine with roughly 30 GB of usable memory or more.

# Apple Silicon Mac
curl -L https://github.com/Joakimpalm-Zen/xyntetik-runner/releases/latest/download/runner-macos-arm64.tar.gz | tar xz
curl -L -o Muse-Glimmer-30B-surgical-6p34-Q8_0.gguf \
  https://huggingface.co/Joakimpalm-Zen/Muse-Glimmer-30B-Surgical-Q8_0-GGUF/resolve/main/Muse-Glimmer-30B-surgical-6p34-Q8_0.gguf
./runner-macos-arm64 -m Muse-Glimmer-30B-surgical-6p34-Q8_0.gguf -i

Linux and Windows builds are on the same release page. Any GGUF runtime with muse-glimmer support should load it: every tensor is an ordinary GGUF type in the standard shape; 12 FFN tensors happen to be exact zeros, which is the surgery.

Measured fidelity vs the BF16 parent

Per-domain behavior stays within a few percent of the parent in every measured domain; healing distils toward the parent, so this is a smaller model that behaves like the original rather than a different model that happens to score well.

Paired benchmark check (2026-08-29): the fidelity bar does not transfer

The fidelity table above is a real, strict measurement. This section reports what it does not buy. One preregistered measurement night translated the house bar into paired public-benchmark deltas versus the frozen parent: lm-evaluation-harness 0.4.9, stock task configs, both models scored through the identical loglikelihood path on identical items, per-item pairing, exact binomial confidence intervals on the discordant pairs, margins fixed before sampling.

The flip counts say the deficit is directional, not noise: on the 3.83% of items where the two models disagree, the surgical model loses roughly 2:1 (269 parent-right-variant-wrong against 116 the reverse).

Stated plainly:

• This model is measurably worse than its parent on HellaSwag. The deficit is about 1.5 points and the confidence interval lies entirely beyond the preregistered ±0.5 equivalence margin. Do not read the fidelity table above as benchmark equivalence: the two criteria disagree, and the paired test is the stricter one.
• The deficit was measured at the BF16 surgery stage. The shipped file adds Q8_0 quantization on top, which the fidelity table shows to be nearly transparent (quant-alone KLD 0.00040), so the BF16 number is the honest expectation for this file. A direct benchmark row for the quantized file is blocked because 27.58 GB exceeds the 24 GB measurement GPU used for the annex.
• Across the two surgery levels the KLD-to-benchmark mapping is monotone and super-linear (30.0 to 39.5 points lost per unit of KLD, from 4.75% to 6.34% removed): deeper surgery costs more benchmark accuracy per unit of measured divergence, so fidelity numbers under-predict benchmark harm as surgery grows. This is the larger surgery; it pays the larger price.

The parent-fidelity table remains this card's primary claim: these weights track the parent's token distribution closely, and that is what they are for. The benchmark row exists so nobody has to take distribution fidelity as a proxy for task equivalence. We measured that it is not one.

How it was made, in six lines

1. Depth-sensitivity mapping found the FFN sublayers whose removal is cheapest, chosen depth-separated (measured to matter).
2. The four FFNs were removed; un-healed, that fails the bar (KLD 0.18772).
3. One FFN per cut (the first surviving layer below it) was retrained against the frozen parent on a corpus scaled to 14x unique tokens.
4. The training data was decontaminated at the 64-token WINDOW level, not the document level: a document-disjoint corpus still carried 11.5% of the eval split's windows via near-duplicates, and those sources were dropped rather than the filter weakened. Zero overlapping windows in the final audit.
5. Quantization noise and surgery error are measured independent (additive within 3.5% across five configurations), making quant costs predictable before a file is built.
6. The GGUF is byte-level substitution of the 24 modified tensors into the parent GGUF, quantized by xyntetik-runner's own quantizer; the BF16 path was validated exactly (KLD 0.00000 against the source checkpoint).

Notes and limits

• The 12 removed-FFN tensors are stored as exact zeros in standard shape, so any runtime that runs the parent GGUF can run this file with no loader changes. The zeros still occupy their quantized bytes; true tensor removal would shave a further ~6%.
• Tested in xyntetik-runner (native muse-glimmer support). Other GGUF runtimes should work if they run the parent at Q8_0; expected, not verified here.
• The fidelity bar is parent-agreement on a held-out split (43,855 audited windows), not a capability benchmark. The full method write-up, including the depth-absorbability law and the quantization-independence result: Muse-Glimmer-30B-surgery-report.

Base model: Muse-Glimmer-30B by Meta, Apache-2.0. This derivative keeps the license and changes 24 tensors (12 zeroed, 12 replaced with healed weights) as described.

Decontamination re-score (2026-08-29)

A window-level audit found that 0.65% of the eval split's windows had leaked into the healing corpus through near-duplicate sources; all healed artifacts were re-scored on the decontaminated subset.

This artifact passes on clean data with comfortable headroom: KLD 0.03901 (bar 0.05), margin-qualified top-1 98.34% (bar 97%), roughly 15 times the measurement noise floor from the bar. The published full-split numbers reproduce from the file on disk to five decimals.

The isolated leak effect on this artifact was +0.00140 KLD, consistent with its two siblings. The measurement method and the control decomposition that separated it from an opposing domain-mix shift will accompany the method write-up's next revision.

Integrity

sha256(Muse-Glimmer-30B-surgical-6p34-Q8_0.gguf) =
d7bb5c6f3b7170160322c684f338c0db630ea24ab09b896a6c97c8c415f58d4d

SURGERY.json in this repo records the byte-substitution provenance: the parent revision, which tensors were zeroed and which were replaced with healed weights.