Qwen3.5-122B-A10B optimized for NVIDIA DGX Spark (GB10, SM12.1) — all quantizations baked in.
This checkpoint is derived from Sehyo/Qwen3.5-122B-A10B-NVFP4 with additional post-quantizations applied and saved permanently into the weights. The result loads and serves immediately on GB10 without any runtime quantization overhead.
The base NVFP4 checkpoint leaves several layer types in BF16 to keep it hardware-agnostic. On GB10 (NVIDIA Grace Blackwell, SM12.1), those BF16 layers are a significant bottleneck:
These optimizations are implemented in scottgl9/sglang-spark-gb10-optimizations as runtime post-quantization hooks. This checkpoint bakes them in so:
| Layer | Original | Baked format | Why |
|---|---|---|---|
| MoE expert weights (gate/up/down) | NVFP4 (W4A4) | NVFP4 (W4A4) | From source checkpoint — preserved as-is |
GDN in_proj_qkv, in_proj_z |
BF16 | NVFP4 (Marlin FP4) | Large projections; SM12.1 has native Marlin FP4 kernels. CUTLASS FP4 is broken on GB10. |
GDN in_proj_a, in_proj_b, out_proj |
BF16 | FP8 (per-tensor, dynamic activations) | Smaller GDN projections; FP8 CUTLASS works correctly on SM12.1. cos_sim > 0.999, SNR = 31.5 dB vs BF16. |
| lm_head (vocabulary projection) | BF16 | FP8 (dynamic activations) | 3072 x 248,320 matmul at every decode step. FP8 halves DRAM reads vs BF16. Activations remain dynamic per-token for accuracy. |
| Attention Q/K/V/O | BF16 | BF16 | Standard attention; not the bandwidth bottleneck at batch=1. |
~46 tok/s decode on a single NVIDIA DGX Spark (GB10, 128 GB unified memory) using SGLang with NEXTN speculative decoding (steps=2, draft_tokens=2).
Baseline without any of these optimizations: ~28 tok/s. The baked quantizations account for approximately +18 tok/s of that gain.
Tested on: NVIDIA DGX Spark (ASUS Ascent GX10), GB10 Grace Blackwell, SM12.1, 128 GB unified memory.
Requirements:
Will NOT work on:
# Load directly — no post-quant overhead at startup
SGLANG_QUANTIZE_LM_HEAD_FP8=0 ./sglang.sh Qwen3.5-NVFP4 \
--model-path scottgl9/Qwen3.5-122B-A10B-NVFP4-GB10
SGLANG_QUANTIZE_LM_HEAD_FP8=0 tells SGLang to skip the lm_head FP8 post-quant (already baked in). All other sglang.sh flags remain unchanged.
With NEXTN speculative decoding (recommended):
SGLANG_QUANTIZE_LM_HEAD_FP8=0 ./sglang.sh Qwen3.5-NVFP4 \
--model-path scottgl9/Qwen3.5-122B-A10B-NVFP4-GB10 \
--speculative-algorithm NEXTN \
--speculative-num-steps 2 \
--speculative-num-draft-tokens 2
Apache 2.0 (same as Qwen3.5)
extra_weights.safetensors 5.0 GBmodel-00006-of-00019.safetensors 4.2 GBmodel-00009-of-00019.safetensors 4.2 GBmodel-00012-of-00019.safetensors 4.2 GBmodel-00015-of-00019.safetensors 4.2 GBmodel-00018-of-00019.safetensors 4.2 GBmodel-00003-of-00019.safetensors 4.2 GBmodel-00007-of-00019.safetensors 4.0 GBmodel-00010-of-00019.safetensors 4.0 GBmodel-00013-of-00019.safetensors 4.0 GBmodel-00016-of-00019.safetensors 4.0 GBmodel-00004-of-00019.safetensors 4.0 GBmodel-00001-of-00019.safetensors 4.0 GBmodel-00005-of-00019.safetensors 3.6 GBmodel-00008-of-00019.safetensors 3.6 GBmodel-00011-of-00019.safetensors 3.6 GBmodel-00014-of-00019.safetensors 3.6 GBmodel-00017-of-00019.safetensors 3.6 GBmodel-00002-of-00019.safetensors 3.6 GBmodel-00019-of-00019.safetensors 3.2 GBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo scottgl/Qwen3.5-122B-A10B-NVFP4-GB10 ./model-folderQwen/Qwen3.5-122B-A10B