GGUF builds of satellitedown/SenseNova-Loop-9B-abliterated: SenseNova-Loop-9B with its refusals removed, chosen by whether it carries out harmful requests rather than by whether it refuses them. Text only; the vision tower is not included.
| File | Size | Same top token as full precision | Mean KL | M4 MacBook Air, 16 GB |
|---|---|---|---|---|
SenseNova-Loop-9B-abliterated-Q4_K_M.gguf |
5.64 GB | 94.1% | 0.0333 | 11.11 tokens/s |
SenseNova-Loop-9B-abliterated-Q6_K.gguf |
7.38 GB | 98.3% | 0.0036 | 8.72 tokens/s |
Accuracy is measured against the full-precision abliterated model on 661 token positions. Speed is plain generation in short runs. The Air has no fan: under continuous work the 4-bit file dropped to about 9 tokens/s within minutes and to about 7 after twenty-five.
On a 16 GB Mac use the 4-bit file. With the 6-bit file loaded at a 32K context, the test machine had under 0.1 GB of memory free and was using swap.
SenseNova-Loop-9B-abliterated-drafter-2-Q4_K_M.gguf (0.92 GB) is a second, small model for the 4-bit file. It reads the main model's hidden states and proposes the next seven tokens; the main model checks them in one pass, keeps the ones it would have written itself, and adds its own next token. One pass over eight tokens costs about 1.5 times a pass over one, so every accepted token is nearly free. Every token of the answer is still the main model's choice: at temperature 0, all 16 test answers were token for token the same with and without it.
On an M4 MacBook Air (16 GB), short runs, temperature 0, thinking off:
| The model is writing | Without drafting | With the drafter |
|---|---|---|
| New code | 11.08 tokens/s | 31.6 tokens/s |
| An explanation in prose | 11.02 tokens/s | 12.2 tokens/s |
| A file it has read, with a small change | 11.01 tokens/s | 65.7 tokens/s |
| A file written through a tool call | 10.62 tokens/s | 59.5 tokens/s |
The last two rows also use drafts taken from the prompt (ngram-simple), which need no drafter. Over ten coding-agent tasks with thinking on, the model wrote 29.0 tokens/s with the command below, against 9.1 with prompt drafts only and the server's default sampling, and passed 10/10 every time.
The drafter is a DFlash block drafter (6 layers, 1.3B parameters) that started from z-lab/Qwen3.5-9B-DFlash and was trained on 11,942 answers and 5,406 coding-agent turns written by this model's 4-bit file, with the hidden states taken from the 4-bit file through llama.cpp itself. Its output layer covers the 65,536 tokens this model writes most, a quarter of the vocabulary: a token outside them ends a draft and costs nothing else. It keeps 3.5 tokens per pass on held-out answers and 3.7 on held-out agent turns. It only fits this model and this quantization: with another file it still cannot change the answers, it just gets fewer tokens accepted.
SenseNova-Loop-9B-abliterated-drafter-Q4_K_M.gguf is the first drafter. Looper 0.2.0 downloads it by name, so it stays. The second keeps more tokens per pass (3.74 against 3.65 on held-out agent turns) and takes 7 ms less per round, because its output layer is a quarter of the size.
Mainline llama.cpp cannot load this architecture yet, so neither can the apps built on it, such as LM Studio and Ollama. Use the SenseNova Loop fork (https://github.com/GuideboardLabs/llama.cpp-sensenova-loop) at commit a7f77a788893ef2cb17c82b9ac0cb1279bcd0918 with the 18 patches in patches/ of this repo, which apply to it cleanly (git am):
git clone https://github.com/GuideboardLabs/llama.cpp-sensenova-loop llama.cpp-loop && cd llama.cpp-loop
git checkout a7f77a788893ef2cb17c82b9ac0cb1279bcd0918 # the fork's SenseNova Loop commit, where the patches apply
git am /path/to/patches/*.patch
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --target llama-server -j
./build/bin/llama-server -m SenseNova-Loop-9B-abliterated-Q4_K_M.gguf -md SenseNova-Loop-9B-abliterated-drafter-2-Q4_K_M.gguf -ngl 99 -c 24576 --jinja -fa on --parallel 1 --cache-ram 0 --ctx-checkpoints 3 --ctx-checkpoint-turns --prompt-tile 32 \
--spec-type ngram-simple,draft-dflash --spec-ngram-simple-size-m 31 --spec-draft-n-max 7 --spec-draft-n-ubatch 64 --spec-rollback 31 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0
What the patches do:
--spec-rollback 31 keeps them for 31 tokens, so a draft from the prompt is cut like a short one instead of being run again). About 0.5 GB less memory, and a file rewrite 12% faster on a warm machine.--ctx-checkpoint-turns it cuts only for a client that returns to those checkpoints; one that sends back what the model wrote, as an agent with reasoning on does, gets its request in one pass. The GPU kernel for more than 16 rows works in tiles of 32 and a pass pays for every tile it starts, so 40 new tokens cost what 64 do; with --prompt-tile 32, what a prompt leaves after whole tiles goes into a second pass through the few-row kernels when it is 16 tokens or fewer: 40 tokens in 531 ms instead of 705. Over the agent tasks a prompt token took 13.2 ms, against 14.4 before both.GGML_METAL_PROFILE=1), and llama-loop-taps, which writes the hidden states and the top logits a drafter is trained and measured on.--spec-type ngram-simple,draft-dflash drafts text the server has already seen from the prompt, up to 31 tokens at a time, and everything else with the drafter, 7 at a time. Both numbers are chosen for the Apple GPU kernels: 32 and 8 rows are what one pass handles without a second tile. The sampling values are the ones Qwen gives for coding with thinking on; the server's own defaults (temperature 0.8, top-k 40) are not this model's.
--cache-ram 0 --ctx-checkpoints 3 matters on small machines: with the defaults the server grew by 8.4 GB over 138 short requests on the test Mac and was killed. --spec-draft-n-ubatch 64 keeps the drafter's buffers at 139 MiB instead of 995. With the drafter the server holds about 0.7 GB more than without.
Converted from the BF16 weights, then llama-quantize with the two small linear-attention gate tensors of every layer (ssm_alpha, ssm_beta) kept at 16 bits, 13 MB in total.
What these files do with harmful requests, answering through llama.cpp, against the original at full precision:
| On 83 unseen harmful requests | Carries out the request | Dodges, answering a harmless question instead | Refuses, or gives no answer |
|---|---|---|---|
| Original, full precision | 0 | 3 | 80 |
| 4-bit, thinking on | 60 | 22 | 1 |
| 4-bit, thinking off | 50 | 32 | 1 |
| 6-bit, thinking on | 57 | 24 | 2 |
| 6-bit, thinking off | 44 | 38 | 1 |
| Full precision, thinking on | 58 | 25 | 0 |
Both files carry out more with thinking on, which is the default. Thinking off through the chat template is the weak mode in every build; the full-precision card says why and what to do instead. A request now and then never stops thinking (1 of the 83 with the 4-bit file, 1 with the 6-bit, within 4,000 tokens); those count as refused. Coding-agent tasks passed with the 4-bit file: 10/10 (the original's 4-bit build: 10/10). How the labels were produced, and the limits, are on the full-precision card.
Apache-2.0, the licence of the original. This is a modified version of sensenova/SenseNova-Loop-9B, and the drafter a modified version of z-lab/Qwen3.5-9B-DFlash, also Apache-2.0; see NOTICE. The patches are MIT, like llama.cpp.