MERNIK Gnom — learned damage ranker for quantization allocation

Gnom is the allocator's learned ranker from the MERNIK project: it predicts, per tensor group of a language model, how much damage (KL-divergence vs the BF16 reference) dropping that group by one quant tier will cause. The quantizer then spends its byte budget protecting the predicted damage leaders.

Earlier Gnom nets shipped as .npz blobs with the feature contract living in code. These artifacts are self-describing .safetensors: weights + feature contract + per-family normalization + measured scores live in one file.

Small by design: the deploy artifact is 75 MLPs × 2,209 parameters = 166,675 params total, 1.35 MB on disk.

The experimental v5.3 candidate is even smaller: 75 MLPs × 337 parameters = 25,275 params total, 258 KB on disk. Both artifacts contain the same 500 tensors (5 families × 5 folds × 3 seeds + mu/sd); v5.3 is smaller because its hidden layers are 8/4 instead of 48/24, while its input is wider (36 columns vs 20). Its higher family-mean OOF (+0.542 vs +0.527) comes from the v5.3 input features (layer/tensor-relative + base_bpw/ drop_bpw), not from extra parameters.

Files

Feature schemas

code/build_features.py supports --schema v51, v52, and v53: v51 = 30 columns, v52 = 38 (adds MoE dispersion), v53 = 34 (adds layer/tensor-relative features). The shipped v5.1-v5.3 feature files use the archived MoE ratio formula; tinynet.featurize(..., moe_ffn_mode="pyc") exposes the later bytecode variant for comparison. The v5.2/v5.3 A/B was below MDE and strict LOMO was mixed, so the deploy artifact stays v5; v5.3 is published as an experimental candidate below.

v5.3 candidate (experimental)

gnom_kfold_v53_2026-11-12.safetensors is the layer/tensor-relative v5.3 scheme: 34 raw features + base_bpw/drop_bpw (36 columns), 5 grouped folds × 3 seeds per family, dims 36,8,4,1. Family-mean OOF Spearman +0.542 (v5 deploy +0.527); per family: lfm3 +0.706, ling_moe +0.129, rl17 +0.787, rl4b +0.630, rl9b +0.460. Strict LOMO is mixed and the gain is below MDE, so this is a candidate, not a replacement for the v5 deploy. Feature order is stored in the artifact metadata; scoring needs 36-col rows, not the 20-col v5 contract.

Usage

from load_gnom import load, predict
st, nets, meta = load("gnom_kfold_v5_2026-10-31.safetensors")

X = build_unit_rows(...)          # 20-col contract, see load_gnom.py docstring
damage = predict(st, nets, "rl17", X)   # higher = more damaging

Family selection: rl17 = Spark-X2.5-1.7B, rl4b = Spark-X2.5-4B, lfm3 = LFM3-1.7B, rl9b = Ornith-1.0-9B, ling_moe = Ling-3.0-tiny.

Self-test (reproduces the stored exam numbers from the artifact + pool):

python load_gnom.py gnom_kfold_v5_2026-10-31.safetensors

The self-test reads the lab paths stored in the pool file, so it expects a MERNIK checkout with gnom/features/ and the labels present (see Provenance). The load/predict API itself has no path dependencies.

Two exams, two numbers

Clean 5-family Fracture pool, n=1165 units; quantizer pins (exact-zero damage: F32 1D/norms/ssm/rope tensors) filtered out — no signal there.

• Within-family (v5, deploy): 5-fold grouped folds over unit tags (all tiers of a unit share a fold — no same-unit leakage), 3 seeds per fold. Every unit is scored by nets that never saw it. This is the number to expect when quantizing one of these five families.
• LOMO (v4, floor): each family's net trained strictly without that family. Use as the conservative expectation for a genuinely new architecture.

The exams differ in folds and preprocessing, so the columns are two views, not a controlled delta (rl4b's LOMO is slightly above its in-family OOF for that reason).

v5 is not a marginal retrain — it is the first Gnom generation that beats the old production tables on a real model. Agreement with measured per-group damage, same groups and labels:

The Ling row is the important one: the old table had no rank signal at all on the MoE (−0.022); v5 does. The ensemble column is optimistic (folds share neighbours through the fold-averaged table); the OOF column is what to expect on unseen units of the same family.

First measured deployment — Ling-3.0-tiny (2026-10-31)

The v5 deploy tables were measured on the weakest family on purpose: the MoE. Per-group raw predictions (netpred/netpred_ling_v5_2026-10-31.json) were fed to both allocators on the same BF16 base and imatrix; protocol unchanged (ref_lingq8_wiki.dat, wiki 297×1024, -c 1024 -b 128 --seed 7).

Held-out agreement with measured group damage on Ling: +0.419 ensemble (honest OOF +0.134) vs −0.022 for the previous production table — the old table carried no Ling signal at all, so any non-zero ranking here is new.

At 4500 the v5 water-fill dominates the previous build on every column; at 5100 it buys the mean/median (−43% / −47% vs NETPURE) and pays the tail (max +8.7%). The greedy NETPURE builds still hold the lowest max KLD. Operator generation check was positive (informal, not a battery); HE was not run — the capability column sat on the floor at 4500 and the open question was fidelity per byte.

Against the vendor Q4_K_M, the heavier file loses. The 4500 arm is 84 MiB lighter than stock Q4_K_M (4516 vs 4600 MiB) and beats it on every column: PPL −2.0%, mean KLD −37.9%, median −41.6%, max KLD −34.6%, RMS Δp 7.51% vs 9.41%, same-top-p 88.65% vs 85.80%. The 5100 arm costs +516 MiB but widens the gap: mean −55.5%, median −59.4%, max −31.3%, PPL −1.5%, same-top-p 90.43%. Stock Q4_K_M is the no-imatrix, no-allocator control — the win is fidelity per byte and per capability, not a bigger file.

Ornith table netpred/netpred_ornith_v5_2026-10-31.json is generated and dry-run checked, not yet measured in a build.

Scope and known limits

• Dense decoder families (Spark 1.7B/4B, LFM3 hybrid): strong — 0.60–0.74.
• Ornith 9B: moderate in-family (0.45), negative strictly cross-family.
• MoE (Ling-3.0-tiny): weak even in-family (0.16). The 6-type one-hot covers dense tissue names only; expert routing/MLA structure is not represented in these features. Treat MoE support as open research, not a product claim.
• Single-seed numbers scatter (see metadata per_seed_oof); always use the ensemble. Never quote a single seed alone.

Artifact schema (v5)

• tensors: <family>/f<fold>/mu, <family>/f<fold>/sd (20,) — per-fold standardization; <family>/f<fold>/s<seed>/W1..b3 — mish MLP (20→48→24→1)
• v4 layout (kept for the transfer exam): <family>/mu, <family>/sd, <family>/s<seed>/W1..b3
• metadata nets: per-family dims, folds, seeds, fold_seed, oof_spearman, per_seed_oof, per_fold, n_units (group), n_rows
• metadata feature_order: the 20 input columns (18 raw + base_bpw + drop_bpw)
• metadata pool_sha: sha256[:16] of the training pool file
• activation mish, target damage_kld, epochs 1500, seeds {7,8,9}

Roundtrip is verified: reload → per-fold (x−mu)/sd → average fold × seed mish forwards reproduces each stored score to 1e-9.

Provenance

• labels: wepiqx/MERNIK-Fracture (KLD damage per tensor group; direct measurement, not PPL proxy; the *_rankonly_* legs are what training consumes)
• features: features/*.npz in the same dataset (18 raw per-tensor columns + imatrix scalars; the 20-col contract adds base_bpw / drop_bpw)
• training code: code/train_gnom.py + code/tinynet.py (this repo); featurizer code/build_features.py needs a MERNIK checkout (gguf/imatrix readers)
• packer: code/pack_gnom.py (kept in sync with scripts/pack_gnom.py in wepiqx/MERNIK)
• built: 2026-10-31, MERNIK lab