Text-only GGUF quants of sensenova/SenseNova-Loop-9B,
the release that supersedes SenseNova-Loop-9B-Preview. Weights, training and the model itself are
SenseNova / OpenSenseNova's work. This repository only converts and quantizes.
Stock llama.cpp cannot load these. The architecture is sensenova_loop, implemented in the runtime
fork https://github.com/GuideboardLabs/llama.cpp-sensenova-loop (branch sensenova-loop, two commits,
patches included). Vision and the MTP head are not part of the export; this is a text decoder.
| file | quant | size |
|---|---|---|
SenseNova-Loop-9B-Q8_0.gguf |
Q8_0 | 9,540,092,256 B |
SenseNova-Loop-9B-Q4_0.gguf |
Q4_0 | 5,325,947,232 B |
SenseNova-Loop-9B-Q4_K_M.gguf |
Q4_K_M | 5,628,723,552 B |
SenseNova-Loop-9B-Q3_K_M.gguf |
Q3_K_M | 4,642,406,752 B |
Q8_0 does not fit an 8 GB card; it is here for the fidelity check and for machines with more VRAM. Every size above is measured from the file, not estimated.
Convolution kernels, norms and the mHC router weights stay FP32; everything else is quantized. The
router weights have to be FP32: ggml_mhc_route asserts GGML_TYPE_F32, and an f16 outtype that leaves
them F16 fails at graph reserve.
The looped block is stored once. The runtime aliases the second visit of those layers onto the first, which is numerically identical and 36 percent smaller than storing the block twice.
Same architecture as the Preview, verified tensor for tensor: 32 stored decoder blocks, 24 of them
executed twice (56 executions per forward pass), two mHC residual streams with a router on every
sublayer inside the repeated range, hybrid linear/full attention on a 3:1 pattern. Exactly three
differences, all in config.json:
| key | Preview | Release |
|---|---|---|
max_position_embeddings |
262144 | 524288 |
rope_parameters.rope_type |
default |
yarn, factor 2.0, original 262144 |
mtp_num_hidden_layers |
1 | 0 (all 16 mtp.* tensors gone) |
All four YaRN keys are carried in the GGUF (rope.scaling.type=yarn, factor=2.0,
original_context_length=262144, context_length=524288) and are exercised by the runtime.
The release checkpoint also carries one extra tensor, model.language_model.loop_gate.logits
(F32 [4096]), which the shipped modeling_sensenova_loop.py — md5-identical to the Preview's copy —
never references, while loop_settings.eval_count is still pinned at 2. It is an unused parameter,
most likely the seed of the "Dynamic Loop" work the model card describes as future, and the converter
drops it.
GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1 ./build/bin/llama-server \
-m SenseNova-Loop-9B-Q4_0.gguf -ngl 99 -fa on -c 16384 --port 8080
The environment variable matters on small-BAR cards: without it the recurrent state buffers land in host-visible memory. It only takes effect once the model fits in VRAM.
Sampling from the shipped generation_config.json (temperature 1.0, top_p 0.95) with thinking left
on is how we run it.
RX 5700 XT (NAVI10), Vulkan, GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1, 6 threads:
| quant | decode | prefill |
|---|---|---|
| Q4_0 | 31.3 t/s | 85.8 t/s (22-token prompt) |
Two checks against the reference transformers implementation (modeling_sensenova_loop.py, bf16,
CPU):
[760, 6511, 314, 9338, 369] for
The capital of France is). Paris, ., \n, Paris,
is), and the log-probability ordering agrees at every step; the top-1 logprob differs by 0.08
nats at the widest point. That is the check that exercises YaRN: if the runtime ignored
rope.scaling.type=yarn, the first tokens would already diverge.\n at -1.88 and The at
-2.07, a 0.19-nat near-tie. Q4_0 puts The first. That is 4-bit noise on a near-tie, not a rope
mismatch, which is why the Q8_0 row carries the fidelity claim.Context was exercised to 16K tokens here, and to 512 positions in the fidelity check; the release advertises 524288 with YaRN, which we have not validated.