SenseNova-Loop-9B-Preview — 5300 SSM-F32

If these quants are useful: I can build more sizes and am looking into vision support in the same llama.cpp fork, so the model can work end-to-end from one runtime. Feedback decides whether that is worth doing.

A text-only GGUF quantization of sensenova/SenseNova-Loop-9B-Preview, built with the MERNIK allocator and an explicit F32 pin on all 48 SSM ssm_alpha / ssm_beta gates.

This is not a stock llama.cpp file. SenseNova-Loop uses the sensenova_loop architecture (looped decoder + mHC residual routing), which requires a patched llama.cpp. Stock builds fail with:

unknown model architecture: 'sensenova_loop'

Do not strip or re-quantize the F32 pins. These pins are not cosmetic. In our testing, removing these F32 protections — for example, re-quantizing the SSM gates to Q4_K — produces looping, repetitive degeneration. The model turns into garbage. The F32 set is the difference between a usable file and a broken one. Never re-quantize this GGUF with a generic Q4_K_M recipe.

Recommended file

At a glance

Why the SSM gates are F32

The model has 24 SSM layers, each with ssm_alpha.weight and ssm_beta.weight, shape [4096, 32]. These 48 tensors are small — about 0.5 MiB each at F32 — but they sit on the recurrent path.

llama.cpp does not pin SSM gates at all. In the generic path they are quantized like ordinary 2-D weights; only ssm_conv1d is special-cased to F32. In a stock Q4_K_M recipe all 48 gates therefore become Q4_K, and the model quality collapses hard — it starts looping and degenerating.

This build forces all 48 gates back to F32 with explicit tensor-type rules. The extra cost is small (~20.6 MiB versus Q4_K), and the mHC routers are already F32 for the same reason: routing and recurrent-gate precision matter more than their byte count suggests.

Gnom-guided allocation

The 5300 MiB budget was not a uniform Q4 mix. The per-tensor allocation was guided by Gnom v5, the damage/importance predictor used for this build:

• Gnom model: https://huggingface.co/wepiqx/MERNIK-gnom
• Gnom predicts where quantization damage is likely to hurt; the allocator then spends the byte budget unevenly instead of treating every tensor alike.
• The SSM gates and mHC routers are then pinned back to F32, outside that predicted budget, because their precision is structurally important.

Quantization details

328 of 538 tensor entries are F32 — more than half the model by tensor count (61%). They are mostly small norms, mHC routers and SSM gates: together only 40.1 MiB, under 1% of the file. The count is large; the byte cost is deliberately small.

All 48/48 ssm_alpha / ssm_beta tensors are F32 in the final file. There are no other non-F32 2-D tensors below 8 MiB F32.

Verification data, including SHA-256 and the full type table, is in sensenova_verification.json.

Build the runtime

This model needs a llama.cpp build with sensenova_loop support. Use our MTP-enabled fork of the GuideboardLabs fork:

git clone https://github.com/wepiqx/llama.cpp-sensenova-loop
cd llama.cpp-sensenova-loop
git checkout 61ee5470fe3ad56f3153fb36fb4e88ba44ce83d5

The upstream GuideboardLabs fork at fad174e can run normal generation, but does not support MTP — see below.

LLAMA_BUILD_SERVER=ON is required in this fork even if you only want llama-cli: the CLI target is registered together with the server target.

NVIDIA CUDA

cmake -B build \
  -DGGML_CUDA=ON \
  -DGGML_VULKAN=OFF \
  -DLLAMA_BUILD_SERVER=ON \
  -DLLAMA_CURL=OFF \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-cli -j"$(nproc)"

Vulkan (AMD / Intel / NVIDIA)

cmake -B build \
  -DGGML_CUDA=OFF \
  -DGGML_VULKAN=ON \
  -DLLAMA_BUILD_SERVER=ON \
  -DLLAMA_CURL=OFF \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-cli -j"$(nproc)"

CPU only

cmake -B build \
  -DGGML_CUDA=OFF \
  -DGGML_VULKAN=OFF \
  -DLLAMA_BUILD_SERVER=ON \
  -DLLAMA_CURL=OFF \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-cli -j"$(nproc)"

Apple Metal (macOS)

cmake -B build \
  -DGGML_METAL=ON \
  -DGGML_CUDA=OFF \
  -DGGML_VULKAN=OFF \
  -DLLAMA_BUILD_SERVER=ON \
  -DLLAMA_CURL=OFF \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-cli -j"$(sysctl -n hw.ncpu)"

The binary is written to build/bin/llama-cli. Use -ngl 99 to offload all layers; lower it if the model does not fit in VRAM.

Usage

With the runtime built, run:

llama-cli \
  -m SenseNova-Loop-9B-Preview-MERNIK-5300-SSMF32.gguf \
  -ngl 99 -c 1024 -st \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 \
  -p "What is the capital of France? Answer in one short sentence."

Recommended sampling — do not lower temperature

--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00

Use this one setting. This model is very sensitive to sampling; lowering the temperature is not recommended. Do not try to "stabilize" it by dropping to temp 0.2–0.6 — use the fixed setting above.

MTP

The MTP weights are included in this GGUF, but upstream llama.cpp does not have working MTP support for sensenova_loop. The clean GuideboardLabs build loads the file for normal generation, but --spec-type draft-mtp segfaults.

Use our MTP-enabled fork:

• https://github.com/wepiqx/llama.cpp-sensenova-loop
• commit 61ee5470fe3ad56f3153fb36fb4e88ba44ce83d5
• build instructions: SENSENOVA-MTP.md in that repo

Then enable MTP with:

--spec-type draft-mtp --spec-draft-n-max 2

Without --spec-type, the same file runs as a normal text model on either fork.

Compatibility

• Stock llama.cpp: not supported; it does not know sensenova_loop.
• Stock GuideboardLabs sensenova-loop fork: normal generation is verified on the clean fad174e build — the file loads and generates. MTP on that clean build is not enabled; it needs the MTP memory patch. Base inference does not.
• LM Studio / Ollama: not supported until they ship SenseNova-Loop.
• Patched llama.cpp: use a build whose sensenova_loop implementation matches the tensor layout of this file. MTP-enabled fork: https://github.com/wepiqx/llama.cpp-sensenova-loop at commit 61ee5470fe3ad56f3153fb36fb4e88ba44ce83d5 (built on GuideboardLabs fad174e). This build includes the MTP block and the runtime memory path needed to use it.
• AtomicChat quants are a different export: their Q4_K_M has 523 tensors, no MTP block and a different layout. Our file has 538 tensors and includes MTP. The tensor counts and runtimes are not interchangeable — do not compare them as if they were the same GGUF.
• The GGUF is text-only. The vision tower is not included.

Status

This is a local research build, not a certified benchmark. The main qualitative observation is that the SSM-F32 build is stable on long code/agentic generation where the generic Q4_K_M recipe loops, including under MTP.

Measurement protocol

PPL and KLD are measured with warmup enabled, context 1024, batch 128, and the full wiki.test.raw corpus — no --chunks cap. KLD uses the protocol's Q8 reference logits.

Perplexity (canary only — not a verdict)

These are full-corpus canary readings, not a quality ranking. PPL is recorded for sanity and reproducibility only; the fidelity verdict is KLD. The Q8 reference used for both arms has base PPL 18.6361 ± 0.1784.

KL divergence (fidelity verdict)

Both arms use the same Q8 reference logits on the full 290-chunk corpus.

The SSM-F32 build is better on mean (−26.5%) and median (−33.4%) KLD, with a slightly lower max and better token-probability agreement. The two arms are close on the tail; this is a fidelity result, not a capability claim.

Credits

• Base model: SenseNova, Apache License 2.0.
• Quantization and allocation: wepiqx.
• Gnom v5 importance model: https://huggingface.co/wepiqx/MERNIK-gnom
• Architecture support: llama.cpp SenseNova-Loop fork (GuideboardLabs), with local MTP export/runtime work.

License

Apache License 2.0. See the base model card and the upstream license for the original weights.