Muse-Glimmer-30B, 6.34% of the decoder removed, Q4_K with distillation-recovered scales (GGUF)

Research artifact, not a general-purpose release.

• What was changed: the healed 6.34% model at Q4_K with every block's fp16 scales retrained by distillation against the BF16 parent; integer codes unchanged.
• Why it exists: to close the gate the surgery study left open: the 6.34% model could not ship at plain Q4_K (0.05815, fail); with recovered scales it reads 0.04949 / 97.90%, inside the bar.
• Parent model: Muse-Glimmer-30B-Surgical-Q8_0-GGUF, itself from meta-models/Muse-Glimmer-30B
• Useful for normal inference? as usable as the Q8_0 sibling at half the bytes, with the same benchmark-annex caveat.
• Experiment: Model Surgery and Scale Recovery
• Evidence: scale-recovery evidence (dataset: preregistration, four-arm scores, logs, code), also copied under evidence/ here
• Runner compatibility: any Runner release; Q4_K is GPU-resident; any runtime that runs the parent runs this file. --kv k8v4 and --kv fp4 from Runner v0.5.6, measured on this file below.

• Validation status (Runner support tiers): each file's row in the Runner roster says which tier it is in, what is re-checked on a release and the date and version of the last run.
  • Muse-Glimmer-30B-surgical-6p34-Q4_K-recovered.gguf: roster row

Want this for another model? The tools behind these files are open: requantization, expert pruning and sublayer removal in Xyntetik Runner, and the method for this file is on this card or in its evidence dataset.

Request a model · Report your result · Community results · All files

If this file is useful to you, a contribution funds the hardware time behind the next one.

The healed 6.34% surgical model - FFN sublayers of layers 4, 7, 9 and 48 removed, one FFN retrained per cut, the same weights as the published Q8_0 release - at Q4_K, 14.61 GiB, with every quantised block's scales retrained against the frozen BF16 parent. Mean KLD 0.04949, margin-qualified top-1 97.90% on the study's 45,056-position held-out split, inside the house bar.

This closes the gate the surgery study left open. Its dense-ladder arithmetic (L6) found the 6.34% model could not ship at Q4_K: BF16 surgery cost 0.03857 plus the Q4_K quantisation term 0.01930 lands near 0.056, and three levers on the surgery term left the gap at +0.00561. Recovery attacks the quantisation term instead, and the two turn out not to be independent once the scales are trained.

Serve with xyntetik-runner

runner -m Muse-Glimmer-30B-surgical-6p34-Q4_K-recovered.gguf --serve --port 8080

Q4_K is GPU-resident in Runner. The 12 removed FFN tensors are stored as exact zeros in standard shape, as in the Q8_0 release, so any runtime that runs the parent GGUF runs this file with no loader changes.

Quickstart

runner -m Muse-Glimmer-30B-surgical-6p34-Q4_K-recovered.gguf -p "The lighthouse keeper" -n 64

What was changed, exactly

Two layers of change, each recorded in its own sidecar:

• Surgery (SURGERY.json, from the study): FFN sublayers at layers 4, 7, 9 and 48 zeroed; the first surviving FFN downstream of each cut (layers 5, 8, 10 and 49) retrained against the frozen parent on the run-9 audited corpus (14.4× dose). These are the weights of Muse-Glimmer-30B-Surgical-Q8_0-GGUF (run9_d14x_bf16.gguf).
• Quantisation: Runner's own quantiser, --quant q4_k, every quantised tensor Q4_K (418 tensors), F32 elsewhere. Unmodified, this file scores 0.05615 / 97.41%: FAIL.
• Recovery (RECOVERY.json): the fp16 d and dmin scale fields of every block in 416 Q4_K tensors (196,591,616 fields) retrained; every integer code, every F32 tensor, the embedding and output-head scales, all metadata and offsets unchanged. Byte length identical to the unrecovered file. Verified: every patched tensor re-decodes to the trained weights, every other tensor is byte-identical. Largest change to any decoded weight: 0.0071.

Training: 400 steps × one 2048-token window of corpus_run9 (0.82 M tokens, the study's window-audited training corpus, nothing else; 20 of its 474 sequences held aside as the kill-check dev proxy, the study's held-out split never read during training), lr 3e-4 cosine, top-64 forward KL to the parent through the study's own bf16 streamer and softcapped logit function, only scales trainable.

Why scales, and what the control found

In a fixed GGUF the per-block scale is the remaining continuous degree of freedom, linear in the decoded weight for every block format; it can be trained end to end against the parent's distribution without reimplementing any codec. The study's ladder had attacked only the surgery term (healing dose, seeds, rung-2 adapters, wider repair windows). This is the first lever on the other term.

The preregistered control - the same recovery on the parent's Q4_K, no surgery - tested whether the study's independence law (quantisation and surgery costs additive, five prior points at 0.965–0.996×) survives recovery. It does not:

Recovery removed 21% of the quant term on the parent and 38% on the surgical file: scales trained against the parent took back part of the surgery error too. Recovered quantisation and surgery are the first sub-additive pair in the study's compounding taxonomy. Preregistered as the refutation outcome (supported-preregistered); the mechanism is an interpretation (supported-exploratory). It moves the ladder's gate from BF16 surgery ≤ 0.0307 to ≤ ~0.039 at recovered Q4_K, which predicts the rung-2 6.34% model at ≈ 0.0474: a test written down before it is run.

Measured envelope

Study scorer (run6_x1_score.py), 11 held-out sequences / 45,056 positions, frozen BF16 parent, never trained on:

Runner fidelity (scripts/kld-compare-raw.py, both models on CPU, 500 positions of the detokenised held-out split, tie band 0.5 nats):

Both instruments agree on the headline: the unrecovered file fails the KLD term, this file passes it. The 500-position runner pair's paired interval includes zero; the 45,056-position study scorer resolves the same pair at 9.4× the study's ±0.00071 noise floor.

Benchmark annex (separate quantity): HellaSwag validation, fixed random 1,000-item subsample, parent and candidate through one instrument, paired.

The study's published full-n figure for this surgery at BF16 is −1.52 [−1.87, −1.15] on acc_norm; quantisation plus recovery added no measurable task loss beyond the surgery's own within this subsample's interval.

KV cache on a GPU (added 2026-09-22)

Measured on this file: 500 held-out positions, tie band 0.5, one 24 GB MIG slice of a datacenter GPU through --gpu auto, with Xyntetik Runner 0.5.5 at the kv-split tip (the --kv k8v4 and --kv fp4 flags ship in v0.5.6). A 24 GB slice holds one copy of this model and not two, so the reference is a CPU --kv f16 server and the first line prices the backend on its own.

k8v4 costs 16 to 29x the q8 cache's KLD (0.002 to 0.004 absolute) at 41% of the f16 bytes. The backend contributes 2.0e-07, four orders below either cache format, and the CPU's own reassociation floor at prompt batch 1 against batch 64 is exactly 0.0 on the same 500 positions, so these numbers price the cache format and not the port.

Scope: one model, one quant, one slice, greedy positions from one held-out split. Models with an attention K bias behave differently under fp4 and are a separate row class.

Reproduce: scripts/kld-compare-raw.py with --args-a "--kv k8v4" and KLD_GPU=auto on side A against --args-b "--kv f16" on a CPU side B, same GGUF on both sides. A second build of the runner on a different toolchain agrees with the binary these rows were taken with to 3.3e-07 mean KLD at 100% top-1 agreement on the same model and corpus. Records: the scale-recovery evidence dataset, evidence/kv-cache-2026-09-22/.

Limits, read before quoting

• The margin under the bar is 0.00051, 0.7× the study's ±0.00071 noise floor. This file passes because it was measured, not comfortably; a different training seed could plausibly land either side. The same qualification the study attached to its 4.75% Q4_K release (#30).
• Parent-agreement is not a capability benchmark. The annex shows the surgery's task cost persists: fidelity and benchmark equivalence are different quantities.
• "No quality loss" is not claimed and is not supported by this evidence.
• Two fidelity instruments, two protocols. Study-scorer KL is full-vocabulary variant‖parent; the runner harness is variant‖reference over the top-20 union. Not interchangeable.
• Size is not reduced by the surgery. The zeroed FFN tensors are still stored (0.84 GiB at Q4_K, verified still exact zeros after recovery); dropping them is a runtime-side edit not applied here.
• Selection data. corpus_run9 trained the scales; the held-out split was opened once, by the scorer, for the numbers above. Benchmark items touched none of the study's data.
• One model, one corpus, one shared 24 GB slice.

Reproduce

# 0. Starting point: the study's healed BF16 (weights of the published Q8_0).
# 1. Quantise with Runner's own quantiser.
runner -m run9_d14x_bf16.gguf --quantize healed_6p34_run9_q4_k --quant q4_k
# 2. Decompose (codes frozen, scales trainable; bit-exact by construction).
python build_unit_cache.py --gguf healed_6p34_run9_q4_k.gguf --output cache/
# 3. Teacher targets from the frozen parent through the study's streamer.
python muse_teacher.py --output teacher-run9
# 4. Train the scales.
python train_muse_scales.py --gguf healed_6p34_run9_q4_k.gguf --cache cache/ \
    --teacher teacher-run9 --steps 400 --seq-tokens 2048 --lr 3e-4 --output run/
# 5. Write back with verification.
python write_scales.py --arch muse --source healed_6p34_run9_q4_k.gguf --cache cache/ \
    --checkpoint run/checkpoint-00400.pt --output Muse-Glimmer-30B-surgical-6p34-Q4_K-recovered.gguf
# 6. Score on the study's held-out split.
python scripts/run6_x1_score.py --variants variants.json --out score.json

Code in code/; the preregistration (hypothesis, lean, criteria, kill rule, and the control) with its commit timestamps in evidence/PREREG_MUSE_SR1.md.

Provenance

• Parent: meta-models/Muse-Glimmer-30B, revision a4e59da52a7bc87ae7251dd5545c0dd437c44b68, Apache-2.0.
• Surgical weights: the Muse-Glimmer surgery study's run9_d14x_bf16.gguf (healed FFN[4,7,9,48], 14.4× dose), published as Joakimpalm-Zen/Muse-Glimmer-30B-Surgical-Q8_0-GGUF; SURGERY.json included.
• Unrecovered Q4_K: SHA-256 7a780a3e21b34341…, 15,686,266,304 bytes.
• This file: SHA-256 12a429483b18aefc…, 15,686,266,304 bytes.
• Training checkpoint: SHA-256 9ed79cd4d6667a84…, step 400.
• Training data: corpus_run9 only (window-audited); the first 2048 tokens of 400 of its 454 training sequences, each seen once, seed 20260913; hashes in RECOVERY.json.

About Xyntetik

This repository is published by Xyntetik, the Zenova AB program for running capable language models on hardware you own and proving what they did. Its released engine is Xyntetik Runner: a free, single-binary GGUF engine in C for CPU, CUDA and Metal that serves, scores and trains through the quantized file it deploys and records every run as a receipt. The measurements behind the program's claims are collected at xyntetik.com/evidence. The program's organization on Hugging Face is Xyntetik.