SenseNova-Loop-9B-Preview

English | 简体中文

📣 News

• [2026.10.30] We release SenseNova-Loop-9B-Preview, a looped multimodal model for long-context and agentic workflows.

🚀 SenseNova-Loop-9B-Preview

🎬 Showcase (Created by Raccoon Work)

Motion Design

PPT Design

Data Analysis

Overview

SenseNova-Loop-9B-Preview is a multimodal language model developed for long-context and agentic workflows. It accepts text and images and generates text for applications that combine document understanding, coding, and interaction with external tools.

The model combines a looped decoder with two-stream manifold-constrained hyper-connections (mHC). A contiguous block of decoder layers is executed twice with shared weights, while learned mappings manage information flow between two residual streams in the looped section. Together, these components provide a deeper execution path without duplicating the repeated block's parameters.

Documented training recipes combine long sequences, multimodal inputs, and supervised task trajectories, with a maximum training sequence length of 262,144 tokens. For the best model performance, we recommend using our documented deployment environment and configuration parameters.

Model highlights

• Shared looped block: 24 decoder layers are applied twice in sequence, producing 56 decoder-layer executions from 32 distinct layers.
• Two-stream mHC: learned mappings route and combine information across two residual streams in the looped section.
• Hybrid attention: the decoder interleaves linear and full attention in a repeating four-layer pattern.
• Long-context multimodal training: documented recipes include text, images, code, and tool trajectories at sequence lengths up to 256K tokens.

📊 Key Benchmarks

Model specifications

Model architecture

SenseNova-Loop-9B-Preview builds on a 32-layer multimodal decoder that combines shared-weight looped computation, hybrid attention, and two-stream manifold-constrained hyper-connections (mHC). The following sections describe how these components extend the computation path and manage information flow through the model.

Looped decoder

The decoder first processes the input through layers 0–3. The same block of layers 4–27 is then applied twice in sequence, after which layers 28–31 and the output head produce the model's output. If B represents the repeated block and h its input, the looped computation can be written as B(B(h)).

During inference, the model performs 56 decoder-layer computations per forward pass while storing only 32 sets of decoder-layer weights. The looped block reuses the same weights across both passes, with the second pass taking the first pass's output as input.

Hybrid attention

The decoder interleaves linear-attention and full-attention layers. Each four-layer group contains three linear-attention layers followed by one full-attention layer. The loop boundaries align with complete four-layer groups, so the attention pattern is preserved across both passes.

Two-stream mHC

Inside the looped section, mHC carries two residual streams around the decoder-layer computations. Learned mappings govern how a layer reads from the streams, mixes stream information, and incorporates the layer output into the residual state. Sinkhorn normalization constrains the stream-mixing map; the documented configuration uses 20 iterations.

The loop range, pass count, and mHC routing parameters form part of the checkpoint's execution configuration. A compatible inference implementation must load these parameters and reproduce the same execution order.

Inference acceleration

SenseNova-Loop-9B-Preview supports MTP for accelerated inference. Dspark and Dflash2 are under development, with training currently in progress.

Practical speedups depend on the workload, hardware, and serving configuration.

Models and resources

Getting started

Use Transformers to load the model and its custom implementation from a Hugging Face repository or a local model directory. The example below runs text generation on a single GPU.

Requirements

The checked inference environment uses Transformers 5.18.0 and PyTorch 2.11.0 with CUDA 13.0. The model directory must include the configuration, tokenizer, chat template, model code, and accessible weight shards.

First install a PyTorch build compatible with your CUDA environment, then install the remaining dependencies:

python -m pip install "transformers==5.18.0" accelerate safetensors

Text generation with Transformers

Set model_id to the Hugging Face repository ID or a local model directory, then run:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sensenova/SenseNova-Loop-9B-Preview"
device = "cuda:0"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map={"": device},
).eval()

messages = [{"role": "user", "content": "What is 13 times 7? Reply with the number only."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(device)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
        use_cache=True,
    )

response = tokenizer.decode(
    outputs[0, inputs.input_ids.shape[1]:],
    skip_special_tokens=True,
)
print(response)

The example uses greedy decoding and generates up to 256 new tokens. Set enable_thinking=True to enable thinking mode, or change device to select another GPU.

trust_remote_code=True loads the custom SenseNova Loop model implementation.

Training

Model training

To fine-tune SenseNova-Loop-9B-Preview with mHC on a single node with 8 GPUs, run the following commands in your configured training environment:

git clone https://github.com/OpenSenseNova/torchtitan.git
cd torchtitan
MODEL_PATH=/path/to/SenseNova-Loop-9B-Preview DATASET=/path/to/train.jsonl \
  bash examples/train/SenseNova-Loop-9B-Preview/mhc2_loop2_titan.sh

Replace /path/to/train.jsonl with your SFT dataset. See the training guide for configuration options, DPO, and MTP training.

Speculative Decoding Training

To train a DSpark or DFlash2 draft model for SenseNova-Loop-9B-Preview on a single node with 8 GPUs, use the training scripts provided in the SenseSpec repository.

First, clone the repository and navigate to the project directory:

git clone https://github.com/OpenSenseNova/SenseSpec.git
cd SenseSpec

DSpark Training

Run the following command to train the DSpark draft model:

bash examples/SenseNova-Loop-9B-Preview-dspark/run.sh

DFlash2 Training

Run the following command to train the DFlash2 draft model:

bash examples/SenseNova-Loop-9B-Preview-dflash2/run.sh

Deployment

LightLLM

Four-GPU example; replace the local model path:

docker pull ghcr.io/modeltc/lightllm:main

docker run --rm --gpus all --ipc=host -p 8000:8000 \
  -v /path/to/SenseNova-Loop-9B-Preview:/model \
  -e LOADWORKER=4 -e LIGHTLLM_TRITON_AUTOTUNE_LEVEL=1 \
  ghcr.io/modeltc/lightllm:main \
  python -m lightllm.server.api_server \
    --model_dir /model --model_name SenseNova-Loop-9B-Preview \
    --tp 4 --host 0.0.0.0 --port 8000

Example request:

curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "SenseNova-Loop-9B-Preview", "messages": [{"role": "user", "content": "Hello!"}], "max_tokens": 128}'

SGLang

Use the version from SGLang PR #47281, which includes the SenseNova Loop implementation, and replace the model path below with your local checkpoint directory:

git clone https://github.com/sgl-project/sglang.git
cd sglang
git fetch origin pull/47281/head
git checkout -b sensenova-loop FETCH_HEAD
python -m pip install -e ./python

sglang serve --model-path /path/to/SenseNova-Loop-9B-Preview

vLLM

Use the version from vLLM PR #64832, which includes the SenseNova Loop implementation, for inference:

git clone https://github.com/vllm-project/vllm.git
cd vllm
git fetch origin pull/64832/head
git checkout -b sensenova-loop FETCH_HEAD
python -m pip install -e .

vllm serve sensenova/SenseNova-Loop-9B-Preview \
  --served-model-name sensenova_loop \
  --tensor-parallel-size 4 \
  --dtype bfloat16

License and citation

License: Apache License 2.0.
Citation:

@misc{sensenova2026loop,
  title        = {{SenseNova-Loop-9B-Preview}},
  author       = {{SenseNova}},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview}
}