This is an unofficial, text-only MLX conversion of sensenova/SenseNova-Loop-9B, quantized to 8-bit weights. It is not an official SenseNova release.
SenseNova-Loop-9B uses a custom looped, dual-stream mHC architecture. Stock mlx-lm does not support this architecture. These weights require an MLX/oMLX runtime with a compatible SenseNova_loop implementation.
The customized implementation used in our tests is included as mlx_sensenova_loop.py for integration and reference. Merely downloading this repository into a stock mlx-lm installation is not sufficient.
The conversion contains text-generation weights only; vision inputs are not supported by this package.
mlx_lm.convertSENSENOVA_MHC_METAL=1 and SENSENOVA_MHC_MERGE_METAL=1The source repository's default generation configuration is temperature=1.0, top_p=0.95, with sampling enabled. In our Chinese long-form tests, that setting could produce irrelevant foreign words in all tested precisions, including unquantized BF16.
For formal Chinese long-form generation, we recommend:
{
"temperature": 0,
"top_p": 1
}
This recommendation targets language artifacts; it does not guarantee factual correctness. Reasoning may also be in English while the final answer is entirely Chinese.
Test environment: Apple Silicon Mac with 128 GB unified memory, oMLX 0.7.3, customized mlx-lm, MLX 0.32.4. Each result is the median of three OpenAI-compatible requests generating 64 tokens. Results depend on hardware, prompt, runtime, and workload.
| Variant | Generation speed |
|---|---|
| BF16 baseline | 8.50 token/s |
| 8-bit | 13.51 token/s |
| 4-bit | 20.90 token/s |
For the 4-bit variant, enabling compiled mHC router subgraphs increased speed from 16.44 to 20.90 token/s (about 27.1% faster). The compilation optimization passed 32-token output and logit-equivalence checks. That equivalence applies to compilation, not to quantization.
Evaluation covered 614 token positions across Chinese, English, Python, and math text. Lower perplexity is better; top-1 agreement is measured against BF16.
| Variant | Perplexity | Top-1 agreement with BF16 |
|---|---|---|
| BF16 | 2.7127 | reference |
| 8-bit | 2.6938 | 99.02% |
| 4-bit | 2.7875 | 93.81% |
Small differences in perplexity can arise from evaluation variation; the table should not be interpreted as proof that 8-bit improves the base model.
temperature=1.0, top_p=0.95)Three fixed seeds were run for each precision. All nine essays finished naturally. Every matched foreign word below was manually reviewed and was inappropriate in context.
| Variant | Foreign words across 3 essays | Foreign words per 1,000 Chinese characters |
|---|---|---|
| BF16 | 7 | 1.21 |
| 8-bit | 23 | 3.32 |
| 4-bit | 18 | 2.76 |
This small sample shows that the artifact exists in the base model too. Quantization may aggravate it, but the results do not show a monotonic relationship with quantization depth.
temperature=0, top_p=1)The 4-bit variant was tested on five different approximately 2,000-character Chinese tasks: two book reflections, one argumentative essay, one literary essay, and one popular-science article.
ChatGPT, AI, and REM were not counted as errors.Five tasks do not prove that artifacts are impossible under all prompts.
This repository includes model-mtp.safetensors, an optional MLX BF16 conversion of the official MTP draft weights. The main model remains 8-bit; the draft is deliberately kept in BF16 because it is a small file and quantizing it did not address the measured bottleneck.
MTP is disabled by default and requires SENSENOVA_MTP=1 with the customized runtime. The included mlx_mtp_speculative.py provides the tested single-request runner. The current oMLX generic MTP toggle does not recognize this custom architecture.
Use depth 2. Deeper drafts did not increase useful acceptance and became slower; depths above 3 are not recommended.
SENSENOVA_MTP=1 python mlx_mtp_speculative.py \
--model /path/to/SenseNova-Loop-9B-MLX-8bit \
--prompt "Your prompt" --depth 2
Add --baseline to run the same prompt without speculative decoding.
MTP speed was measured against the 4-bit target, where one code task improved by about 9.6% while another was slower. The draft behavior may differ with the 8-bit target and has not been separately benchmarked here. Treat MTP as an optional, workload-dependent experiment, not a general speed setting.
hf download NeosQ/SenseNova-Loop-9B-MLX-8bit \
--local-dir SenseNova-Loop-9B-MLX-8bit
Apache-2.0. Please also follow the base model's license and terms. All quantization and testing described here are unofficial community work.
model-00001-of-00002.safetensors 5.4 GBmodel-00002-of-00002.safetensors 4.2 GBmodel-mtp.safetensors 487 MBsensenova/SenseNova-Loop-9B