NeoHorse-1-9B — MERNIK Quantization

MERNIK ("the one who measures") quants of TokenRhythm/NeoHorse-1-9B: imatrix-driven priority-queue allocation, judged by all three columns in synergy — PPL (canary), KLD-vs-ref (rank), scored tasks (verdict) — never by one alone. That synergy is the method; any single metric crowns its own artifacts.

Note: the Q8 / F16 / Q4 / Q2 tags in file names are for your convenience (build character hints) — pick by size, verify by the table below. All files here are MERNIK quants.

🧪 MERNIK protocol. This card leads with the verdict column; every build below carries all three — PPL (canary), KLD-vs-ref (rank), scored tasks (verdict) — published win or lose. Method + protocol: wepiqx/MERNIK.

Results (all builds, all columns)

HE = HumanEval: 164 hand-written Python tasks, model writes code, hidden tests run it. pass@1 = solved on the first try — strict instruction-following plus real code, no partial credit.

Official BF16 posts 98.17% (context length undisclosed in the vendor report — our runs below are -c 8192, so treat the gap as approximate). llama-server --jinja, temp 1.0 / top_p 0.95 / top_k 20 / presence 0.0, max_tokens 2048, fixed seeds. PPL/KLD: llama-perplexity -ngl 99 -c 1024 -n 64 -b 512 --seed 7, wikitext-2-raw, KLD vs Q8-proxy (no BF16 runs).

The budget law: SMAPE owns small budgets (+3pp at 5100), MSE owns big ones (+3.7pp at 6500). Same PPL both times, opposite code. Full analysis: wepiqx/MERNIK.

⚡ Fast ring: GPQA-recognition (non-standard). Same builds scored by first-token letter likelihood (no generation, ~3 min): Neo-MSE-6500 51.5%, Neo-SMAPE-5100 48.0%. Rank agrees with HE here (MSE > SMAPE) but punishes thinkers — see MERNIK for the method and its limits.

🗺️ Capability map (field-tested, no mercy). 6500 writes full working games from scratch. 5100-SMAPE writes a complete working snake, no stuttering. 3650-Q2 writes correct syntax and surprisingly good Russian, but a whole game file is beyond it — short functions yes, projects no. Every weight has its jobs; the table above tells which.

🔮 Possible next: smape_frag. SMAPE-5100 already matches stock Q6_K (+1 task) at −2.2 GB. The fragility-modulated version could take the outright crown — but its fragility table costs serious compute. Want the table flipped? Say so in the discussions — community demand decides.

What is FRAG? Plain SMAPE aims the budget by theory (relative error). FRAG first measures every tensor group: each one is briefly dropped a tier and the real damage is recorded — hundreds of test builds. Groups that break easily ("fragile") get boosted, tough ones ("oak") get discounted. Theory aims, measurement corrects. That's why it wins — and why it costs.

⚠️ Sampling note. The vendor-recommended presence_penalty 1.5 breaks thinking models on this harness (HTTP 500 peg-native format — the penalty corrupts the thought template and the model stalls). All runs here use presence_penalty 0.0, verified clean across 3×164 tasks.

⚠️ Context note. These runs used -c 8192 / max_tokens 2048, which is NOT enough for a thinking model: long chains of thought trip on the generation ceiling and the quant can't fully open up. At 16384+ the gap above should widen — this table is the floor, not the ceiling. Re-runs at larger budgets are planned and will be logged here.

Usage

llama-server -m NeoHorse-1-9B-MERNIK-6500-Q8.gguf \
  -c 16384 --jinja -ngl 99 \
  --temp 0.6 --top-p 0.95 --top-k 20

📝 Sampling for code: temp 0.6, top_k 20, top_p 0.95, min_p 0 — verified in the field (a full working snake game, Russian UI, almost no thinking needed). Benchmarks run at temp 1.0 per vendor protocol; real work wants 0.6.

🧪 Long context — help test. The main build keeps attention at Q8 almost everywhere (45×Q8 + 5×Q6 of 56); the TD-SMAPE file above holds a 949 MiB F16 attention core (qkv layers 20–30 + 3) — built for distance. Neither PPL nor KLD sees past short chunks, so we have not measured this. If you run needle-in-haystack or long-doc tasks, report your context length and score — real user numbers go on this card.

Quantization and tuning by wepiqx. Built with llama.cpp.

⏳ 40+ GPU-hours on a GTX 1070. Every number on this card was measured, not estimated — eleven HumanEval batteries plus a full PPL/KLD grid on an 8 GB card from 2016. If these quants serve you, a like or a test report is the fuel for the next round.