GGUF builds of satellitedown/SenseNova-Loop-9B-abliterated: SenseNova-Loop-9B with its refusals removed, chosen by whether it carries out harmful requests rather than by whether it refuses them. Text only; the vision tower is not included.
| File | Size | Same top token as full precision | Mean KL | M4 MacBook Air, 16 GB |
|---|---|---|---|---|
SenseNova-Loop-9B-abliterated-Q4_K_M.gguf |
5.64 GB | 94.1% | 0.0333 | 11.11 tokens/s |
SenseNova-Loop-9B-abliterated-Q6_K.gguf |
7.38 GB | 98.3% | 0.0036 | 8.72 tokens/s |
Accuracy is measured against the full-precision abliterated model on 661 token positions. Speed is plain generation in short runs. The Air has no fan: under continuous work the 4-bit file dropped to about 9 tokens/s within minutes and to about 7 after twenty-five.
On a 16 GB Mac use the 4-bit file. With the 6-bit file loaded at a 32K context, the test machine had under 0.1 GB of memory free and was using swap.
SenseNova-Loop-9B-abliterated-drafter-Q4_K_M.gguf (0.77 GB) is a second, small model for the 4-bit file. It reads the main model's hidden states and proposes the next seven tokens; the main model checks them in one pass, keeps the ones it would have written itself, and adds its own next token. One pass over eight tokens costs about 1.5 times a pass over one, so every accepted token is nearly free. Every token of the answer is still the main model's choice: at temperature 0, 15 of 16 test answers were token for token the same with and without it, and the rest part at a token where the model's two best candidates were nearly tied (checking several tokens in one pass adds the same numbers in another order).
On an M4 MacBook Air (16 GB), short runs, temperature 0, thinking off:
| The model is writing | Without drafting | With the drafter |
|---|---|---|
| New code | 11.08 tokens/s | 29.56 tokens/s |
| An explanation in prose | 11.02 tokens/s | 11.41 tokens/s |
| A file it has read, with a small change | 11.01 tokens/s | 61.08 tokens/s |
| A file written through a tool call | 10.62 tokens/s | 54.58 tokens/s |
The last two rows also use drafts taken from the prompt (ngram-simple), which need no drafter. Over ten coding-agent tasks with thinking on, the model wrote 22.6 tokens/s with the command below, against 9.1 with prompt drafts only and the server's default sampling, and passed 10/10 both times.
The drafter is a DFlash block drafter (6 layers, 1.3B parameters) that started from z-lab/Qwen3.5-9B-DFlash and was trained on 5,966 answers and 2,325 coding-agent turns written by this model's 4-bit file, with the hidden states taken from the 4-bit file through llama.cpp itself. It keeps 3.5 tokens per pass on held-out answers and 3.7 on held-out agent turns. It only fits this model and this quantization: with another file it still cannot change the answers, it just gets fewer tokens accepted.
Mainline llama.cpp cannot load this architecture yet, so neither can the apps built on it, such as LM Studio and Ollama. Use the SenseNova Loop fork (https://github.com/GuideboardLabs/llama.cpp-sensenova-loop) at commit a7f77a788893ef2cb17c82b9ac0cb1279bcd0918 with the 10 patches in patches/ of this repo, which apply to it cleanly (git am):
git clone https://github.com/GuideboardLabs/llama.cpp-sensenova-loop llama.cpp-loop && cd llama.cpp-loop
git checkout a7f77a788893ef2cb17c82b9ac0cb1279bcd0918 # the fork's SenseNova Loop commit, where the patches apply
git am /path/to/patches/*.patch
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --target llama-server -j
./build/bin/llama-server -m SenseNova-Loop-9B-abliterated-Q4_K_M.gguf -md SenseNova-Loop-9B-abliterated-drafter-Q4_K_M.gguf -ngl 99 -c 24576 --jinja -fa on --parallel 1 --cache-ram 0 --ctx-checkpoints 3 \
--spec-type ngram-simple,draft-dflash --spec-ngram-simple-size-m 31 --spec-draft-n-max 7 --spec-draft-n-ubatch 64 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0
What the patches do:
GGML_METAL_PROFILE=1), and llama-loop-taps, which writes the hidden states a drafter is trained on.--spec-type ngram-simple,draft-dflash drafts text the server has already seen from the prompt, up to 31 tokens at a time, and everything else with the drafter, 7 at a time. Both numbers are chosen for the Apple GPU kernels: 32 and 8 rows are what one pass handles without a second tile. The sampling values are the ones Qwen gives for coding with thinking on; the server's own defaults (temperature 0.8, top-k 40) are not this model's.
--cache-ram 0 --ctx-checkpoints 3 matters on small machines: with the defaults the server grew by 8.4 GB over 138 short requests on the test Mac and was killed. --spec-draft-n-ubatch 64 keeps the drafter's buffers at 139 MiB instead of 995. With the drafter the server holds about 1.4 GB more than without.
Converted from the BF16 weights, then llama-quantize with the two small linear-attention gate tensors of every layer (ssm_alpha, ssm_beta) kept at 16 bits, 13 MB in total.
What these files do with harmful requests, answering through llama.cpp, against the original at full precision:
| On 83 unseen harmful requests | Carries out the request | Dodges, answering a harmless question instead | Refuses, or gives no answer |
|---|---|---|---|
| Original, full precision | 0 | 3 | 80 |
| 4-bit, thinking on | 60 | 22 | 1 |
| 4-bit, thinking off | 50 | 32 | 1 |
| 6-bit, thinking on | 57 | 24 | 2 |
| 6-bit, thinking off | 44 | 38 | 1 |
| Full precision, thinking on | 58 | 25 | 0 |
Both files carry out more with thinking on, which is the default. Thinking off through the chat template is the weak mode in every build; the full-precision card says why and what to do instead. A request now and then never stops thinking (1 of the 83 with the 4-bit file, 1 with the 6-bit, within 4,000 tokens); those count as refused. Coding-agent tasks passed with the 4-bit file: 10/10 (the original's 4-bit build: 10/10). How the labels were produced, and the limits, are on the full-precision card.
Apache-2.0, the licence of the original. This is a modified version of sensenova/SenseNova-Loop-9B, and the drafter a modified version of z-lab/Qwen3.5-9B-DFlash, also Apache-2.0; see NOTICE. The patches are MIT, like llama.cpp.
satellitedown/SenseNova-Loop-9B-abliterated