Block-wise FP8 weight quantization of sensenova/SenseNova-Loop-9B,
the 9B-parameter looped multimodal decoder (32 physical layers executed as 56 layer passes, two-stream mHC,
hybrid linear/full attention). The checkpoint layout matches the official Qwen FP8 releases
(quant_method: fp8, block-wise weight_scale_inv), so FP8-aware engines can load it without renames or extra files.
e4m3fn, one BF16 scale per 128x128 block (<name>.weight_scale_inv, shape [ceil(out/128), ceil(in/128)]), per-block absmax scaling. Weight-only, no calibration data.quantization_config: quant_method: fp8, fmt: e4m3, weight_block_size: [128, 128], activation_scheme: dynamic — same schema as Qwen/Qwen3-32B-FP8.q/k/v/o_projin_proj_a/b/qkv/z and out_projgate/up/down_projattn.qkv, attn.proj, mlp.linear_fc1/2)embed_tokens, lm_head, all norms (BF16)conv1d, A_log, dt_bias (BF16)mhc_attn / mhc_ffn and loop_gate (FP32, as in the original)Qwen3-VL-*-FP8Single RTX 4090, transformers 5.18.0 / torch 2.11.0+cu130 (the upstream-pinned environment). BF16 original vs dequantized FP8, both loaded through the identical transformers code path:
The checkpoint targets FP8-aware inference engines. SenseNova-Loop engine support is available via vLLM PR #64832 and SGLang PR #47281; FP8 kernel coverage for this architecture in those branches is not yet verified upstream — BF16 is the documented path there.
For transformers (the custom modeling code loads BF16), dequantize at load time with the
verified helper shipped in this repo:
hf download letechlead/SenseNova-Loop-9B-FP8 --local-dir SenseNova-Loop-9B-FP8
python dequant_to_bf16.py --src SenseNova-Loop-9B-FP8 --dst SenseNova-Loop-9B-FP8-bf16
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "SenseNova-Loop-9B-FP8-bf16"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id, trust_remote_code=True,
dtype=torch.bfloat16, device_map={"": "cuda:0"}).eval()
The dequantized copy is numerically the FP8 model (same weights, BF16 container) and needs ~19 GB of disk plus the usual BF16 runtime memory. For native BF16 or 4-bit deployment use the upstream releases: sensenova/SenseNova-Loop-9B and the official AWQ variant.
Apache-2.0, inherited from the original model. Original weights (c) SenseNova; quantization by LeTechLead.
model-00001.safetensors 2.0 GBmodel-00003.safetensors 2.0 GBmodel-00004.safetensors 1.0 GBmodel-00008.safetensors 986 MBmodel-00006.safetensors 986 MBmodel-00007.safetensors 968 MBmodel-00009.safetensors 968 MBmodel-00005.safetensors 968 MBmodel-00010.safetensors 959 MBmodel-00011.safetensors 609 MBmodel-00002.safetensors 8.3 kBsensenova/SenseNova-Loop-9B