Muse-Glimmer-30B, 6.34% of the decoder removed, Q4_K with distillation-recovered scales (GGUF)

Research artifact, not a general-purpose release.

• What was changed: the healed 6.34% model at Q4_K with every block's fp16 scales retrained by distillation against the BF16 parent; integer codes unchanged.
• Why it exists: to close the gate the surgery study left open: the 6.34% model could not ship at plain Q4_K (0.05815, fail); with recovered scales it reads 0.04949 / 97.90%, inside the bar.
• Parent model: Muse-Glimmer-30B-Surgical-Q8_0-GGUF, itself from meta-models/Muse-Glimmer-30B
• Useful for normal inference? as usable as the Q8_0 sibling at half the bytes, with the same benchmark-annex caveat.
• Experiment: Model Surgery and Scale Recovery
• Evidence: scale-recovery evidence (dataset: preregistration, four-arm scores, logs, code), also copied under evidence/ here
• Runner compatibility: any Runner release; Q4_K is GPU-resident; any runtime that runs the parent runs this file.

The healed 6.34% surgical model — FFN sublayers of layers 4, 7, 9 and 48 removed, one FFN retrained per cut, the same weights as the published Q8_0 release — at Q4_K, 14.61 GiB, with every quantised block's scales retrained against the frozen BF16 parent. Mean KLD 0.04949, margin-qualified top-1 97.90% on the study's 45,056-position held-out split — inside the house bar.

This closes the gate the surgery study left open. Its dense-ladder arithmetic (L6) found the 6.34% model could not ship at Q4_K: BF16 surgery cost 0.03857 plus the Q4_K quantisation term 0.01930 lands near 0.056, and three levers on the surgery term left the gap at +0.00561. Recovery attacks the quantisation term instead — and the two turn out not to be independent once the scales are trained.

Serve with xyntetik-runner

runner -m Muse-Glimmer-30B-surgical-6p34-Q4_K-recovered.gguf --serve --port 8080

Q4_K is GPU-resident in Runner. The 12 removed FFN tensors are stored as exact zeros in standard shape, as in the Q8_0 release, so any runtime that runs the parent GGUF runs this file with no loader changes.

Quickstart

runner -m Muse-Glimmer-30B-surgical-6p34-Q4_K-recovered.gguf -p "The lighthouse keeper" -n 64

What was changed, exactly

Two layers of change, each recorded in its own sidecar:

• Surgery (SURGERY.json, from the study): FFN sublayers at layers 4, 7, 9 and 48 zeroed; the first surviving FFN downstream of each cut (layers 5, 8, 10 and 49) retrained against the frozen parent on the run-9 audited corpus (14.4× dose). These are the weights of Muse-Glimmer-30B-Surgical-Q8_0-GGUF (run9_d14x_bf16.gguf).
• Quantisation: Runner's own quantiser, --quant q4_k, every quantised tensor Q4_K (418 tensors), F32 elsewhere. Unmodified, this file scores 0.05615 / 97.41% — FAIL.
• Recovery (RECOVERY.json): the fp16 d and dmin scale fields of every block in 416 Q4_K tensors (196,591,616 fields) retrained; every integer code, every F32 tensor, the embedding and output-head scales, all metadata and offsets unchanged. Byte length identical to the unrecovered file. Verified: every patched tensor re-decodes to the trained weights, every other tensor is byte-identical. Largest change to any decoded weight: 0.0071.

Training: 400 steps × one 2048-token window of corpus_run9 (0.82 M tokens, the study's window-audited training corpus, nothing else; 20 of its 474 sequences held aside as the kill-check dev proxy, the study's held-out split never read during training), lr 3e-4 cosine, top-64 forward KL to the parent through the study's own bf16 streamer and softcapped logit function, only scales trainable.

Why scales, and what the control found

In a fixed GGUF the per-block scale is the remaining continuous degree of freedom, linear in the decoded weight for every block format; it can be trained end to end against the parent's distribution without reimplementing any codec. The study's ladder had attacked only the surgery term (healing dose, seeds, rung-2 adapters, wider repair windows). This is the first lever on the other term.

The preregistered control — the same recovery on the parent's Q4_K, no surgery — tested whether the study's independence law (quantisation and surgery costs additive, five prior points at 0.965–0.996×) survives recovery. It does not:

Recovery removed 21% of the quant term on the parent and 38% on the surgical file: scales trained against the parent took back part of the surgery error too. Recovered quantisation and surgery are the first sub-additive pair in the study's compounding taxonomy. Preregistered as the refutation outcome (supported-preregistered); the mechanism is an interpretation (supported-exploratory). It moves the ladder's gate from BF16 surgery ≤ 0.0307 to ≤ ~0.039 at recovered Q4_K, which predicts the rung-2 6.34% model at ≈ 0.0474 — a test written down before it is run.

Measured envelope

Study scorer (run6_x1_score.py), 11 held-out sequences / 45,056 positions, frozen BF16 parent, never trained on:

Runner fidelity (scripts/kld-compare-raw.py, both models on CPU, 500 positions of the detokenised held-out split, tie band 0.5 nats):

Both instruments agree on the headline: the unrecovered file fails the KLD term, this file passes it. The 500-position runner pair's paired interval includes zero; the 45,056-position study scorer resolves the same pair at 9.4× the study's ±0.00071 noise floor.

Benchmark annex (separate quantity): HellaSwag validation, fixed random 1,000-item subsample, parent and candidate through one instrument, paired.

The study's published full-n figure for this surgery at BF16 is −1.52 [−1.87, −1.15] on acc_norm; quantisation plus recovery added no measurable task loss beyond the surgery's own within this subsample's interval.

Limits, read before quoting

• The margin under the bar is 0.00051 — 0.7× the study's ±0.00071 noise floor. This file passes because it was measured, not comfortably; a different training seed could plausibly land either side. The same qualification the study attached to its 4.75% Q4_K release (#30).
• Parent-agreement is not a capability benchmark. The annex shows the surgery's task cost persists: fidelity and benchmark equivalence are different quantities.
• "No quality loss" is not claimed and is not supported by this evidence.
• Two fidelity instruments, two protocols. Study-scorer KL is full-vocabulary variant‖parent; the runner harness is variant‖reference over the top-20 union. Not interchangeable.
• Size is not reduced by the surgery. The zeroed FFN tensors are still stored (0.84 GiB at Q4_K, verified still exact zeros after recovery); dropping them is a runtime-side edit not applied here.
• Selection data. corpus_run9 trained the scales; the held-out split was opened once, by the scorer, for the numbers above. Benchmark items touched none of the study's data.
• One model, one corpus, one shared 24 GB slice.

Reproduce

# 0. Starting point: the study's healed BF16 (weights of the published Q8_0).
# 1. Quantise with Runner's own quantiser.
runner -m run9_d14x_bf16.gguf --quantize healed_6p34_run9_q4_k --quant q4_k
# 2. Decompose (codes frozen, scales trainable; bit-exact by construction).
python build_unit_cache.py --gguf healed_6p34_run9_q4_k.gguf --output cache/
# 3. Teacher targets from the frozen parent through the study's streamer.
python muse_teacher.py --output teacher-run9
# 4. Train the scales.
python train_muse_scales.py --gguf healed_6p34_run9_q4_k.gguf --cache cache/ \
    --teacher teacher-run9 --steps 400 --seq-tokens 2048 --lr 3e-4 --output run/
# 5. Write back with verification.
python write_scales.py --arch muse --source healed_6p34_run9_q4_k.gguf --cache cache/ \
    --checkpoint run/checkpoint-00400.pt --output Muse-Glimmer-30B-surgical-6p34-Q4_K-recovered.gguf
# 6. Score on the study's held-out split.
python scripts/run6_x1_score.py --variants variants.json --out score.json

Code in code/; the preregistration (hypothesis, lean, criteria, kill rule, and the control) with its commit timestamps in evidence/PREREG_MUSE_SR1.md.

Provenance

• Parent: meta-models/Muse-Glimmer-30B, revision a4e59da52a7bc87ae7251dd5545c0dd437c44b68, Apache-2.0.
• Surgical weights: the Muse-Glimmer surgery study's run9_d14x_bf16.gguf (healed FFN[4,7,9,48], 14.4× dose), published as Joakimpalm-Zen/Muse-Glimmer-30B-Surgical-Q8_0-GGUF; SURGERY.json included.
• Unrecovered Q4_K: SHA-256 7a780a3e21b34341…, 15,686,266,304 bytes.
• This file: SHA-256 12a429483b18aefc…, 15,686,266,304 bytes.
• Training checkpoint: SHA-256 9ed79cd4d6667a84…, step 400.
• Training data: corpus_run9 only (window-audited); the first 2048 tokens of 400 of its 454 training sequences, each seen once, seed 20260913; hashes in RECOVERY.json.