Runtime: Osaurus 0.25.23 or newer.

OsaurusAI/Nemotron-3.5-Super-VL-120B-A12B-JANG_4M

Nemotron 3.5 Super VL 120B-A12B in 74.6 GB — 95.1% top-1 agreement with the full-precision model. Vision and the native multi-token-prediction (MTP) head included.

A JANG bundle of nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16: a 120B-parameter (12B active) hybrid of 40 Mamba-2 layers, 40 latent-MoE layers (512 routed experts, top-22, 1024-wide latent, relu2, plus a shared expert) and 8 attention layers, with a C-RADIO v4-H vision encoder and one MTP step.

• Routed experts: affine 4-bit (group 64), rounded with GPTQ on per-expert Hessians after activation-aware (AWQ) scaling.
• Everything else — Mamba-2 in/out projections, attention, latent projections, shared experts, lm_head, embeddings: affine 8-bit (group 64), GPTQ (embeddings rounded). Routers stay bf16 (computed in fp32), expert-score bias fp32.
• Vision tower and projector: affine 8-bit (group 64); the projector's final LayerNorm bf16.
• Native MTP head (mtp.layers.0 attention + mtp.layers.1 MoE): experts 4-bit, the rest 8-bit, calibrated like the main model.
• Size: 74.56 GB of model shards (74.57 GB total). 4.79 bits per weight over the 124.5B source parameters.

Quality vs the original model

403 held-out sequences, 316,184 teacher-forced positions, scored against the BF16 release run layer by layer from disk. Top-128 renormalized KL. Every assistant turn in them was written by Super itself at full precision (T=1.0, top-p 0.95) across its three reasoning modes, 22 human languages, ~35 coding languages, terminal, math, science, security, multi-round agent episodes with tool calls, and photos / documents / charts.

Change in negative log-likelihood of the actual held-out tokens: +0.002.

By reasoning mode, top-1 agreement:

By domain, median KL · mean KL · top-1:

Tool decisions and reasoning

• At the 102 points where the original model's own transcript starts a tool call: 102 of 102 start the call; no call ↔ answer flips at any of the 350 decision points.
• At the 255 points where the original model closes its reasoning (</think>): median P(</think>) 0.993 (original 0.992); none collapsed.

Native MTP head

The bundle keeps Super's one-step MTP head (it predicts the token after next from the main model's final hidden state and the next token's embedding). On the same held-out positions:

• top-1 agreement with the original MTP head 95.0% (KL mean 0.038);
• the head's top-1 equals the actual token after next on 85.1% of generated positions — the original BF16 head: 85.2%.

What's in the bundle

• Experts (40 latent-MoE layers + the MTP MoE; fc1 = the 1024 → 2688 up projection, fc2 = the 2688 → 1024 down projection; relu2, no gate): all 4-bit group 64, 65.0 GB. 41,945 of the 41,984 projections were GPTQ-rounded; 39 rarely-routed experts with too few calibration rows were rounded to nearest.
• Non-experts (measured from the shards): Mamba-2 in_proj / out_proj 8-bit ×40 each, shared-expert up / down 8-bit ×40 each, latent fc1 / fc2 8-bit ×40 each, attention q/k/v/o 8-bit ×8 each, lm_head and embeddings 8-bit. Only the 41 routers are stored unquantized (bf16). Vision: 132 linears 8-bit; position embedding, class tokens, norms and the projector's final LayerNorm bf16.
• MTP: 40 tensors in the model shards (mtp.layers.0.*, mtp.layers.1.*); a small stamp file records the measured lm_head layout (8-bit group 64, untied) for draft proposals.

Serving contract

• Reasoning modes (chat-template kwargs): enable_thinking: true (default), enable_thinking: true + medium_effort: true (efficient reasoning), enable_thinking: false. There are no other effort levels.
• Reasoning: the chat template opens <think>; the model closes it with </think> before the answer. Tool calls use the qwen3_coder XML format (<tool_call><function=…><parameter=…>), as in NVIDIA's own serving recipe.
• Sampling (both generation_config.json and jang_config.json): temperature 1.0, top-p 0.95. EOS: </s>, <|im_end|>.
• Images: any size (dynamic resolution, 1,024–13,312 patches); each image becomes ⌈patches / 4⌉ tokens.

Build details

• Calibration: 1.30M tokens over 1,436 sequences — Super's own full-precision outputs (734k generated tokens) over on / medium / off reasoning, coding (~35 languages), agent tool rounds, terminal, math, 22 human languages, science, security, chat, prose, and 152 images (photos, documents, charts). Held-out rows (403 sequences) are disjoint from calibration.
• Per-expert Hessians (1024-wide latent fc1 inputs, relu2 fc2 inputs, router-weight² weighted) and per-linear input Hessians from a layer-streamed run of the BF16 model, including the MTP step.
• AWQ with exact folds: fc1 input scales folded into the latent down-projection's rows; fc2 input scales folded through each expert's own fc1 rows (relu2 is homogeneous of degree 2); norm-produced 8-bit inputs folded into their norms with the bf16-rounded scale compensated exactly. Strength chosen per layer from the Hessian-weighted error (no-op allowed).
• GPTQ on the scaled weights; channels that never fire in calibration (common with relu2 inputs) keep their weights.
• Bundle verified from the shards: every tensor file aligned, one index, every module's bits measured from bytes, every source tensor accounted for.