NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-heretic-ara

🧬 Base model: nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16
📦 GGUF: static · imatrix (i1) — vision + MTP sidecars included
🛠️ Decensored with Heretic (Arbitrary-Rank Ablation)
📊 Benchmarked on CaliperBench
💬 Discord

[!Note] Credit for the base model goes to NVIDIA — NVIDIA-Nemotron-3.5-Super-VL-120B-A12B, a 120B total / 12B active hybrid Mamba-Transformer latent-MoE with a RADIO vision tower and an MTP head. Credit for the method goes to p-e-w and the Heretic contributors (Arbitrary-Rank Ablation, Heretic 2.0). We ran the search and built this checkpoint.

[!Important] This is the full Omni checkpoint, not a text-only extract. Vision tower, projector, MTP head, tokenizer, chat template and NVIDIA's custom code are byte-identical to the original. The only change is 19 language-model writer matrices in layers 41–53 (6 Mamba out_proj, 1 attention o_proj, and the latent and shared-expert down projections of 6 MoE layers), so it loads and serves exactly like the NVIDIA checkpoint.

[!Tip] Benchmarked on CaliperBench — a creative-writing benchmark scoring prose craft, roleplay and willingness rather than general intelligence.

This is a decensored version of nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16, made using Heretic v2.0.0.dev0

Arbitrary-Rank Ablation (ARA) parameters

Performance

Heretic's own evaluation: refusals are Heretic's keyword rule over 100 held-out prompts from mlabonne/harmful_behaviors; KL divergence is first-token KL against the original model over 100 prompts from mlabonne/harmless_alpaca. Thinking off. It is not a safety evaluation and says nothing about what the model will or will not produce.

Use

Load exactly as the original — see NVIDIA's model card for transformers / vLLM / SGLang serving, the reasoning parser and recommended sampling. Everything except the edited weights is unchanged, including trust_remote_code requirements.

Thinking: on by default in the template; pass chat_template_kwargs: {"enable_thinking": false} per request to turn it off.

For llama.cpp, use the GGUF repo (vision via mmproj-*.gguf, MTP via mtp-*.gguf).

How it was made

1. Search. Heretic's ARA modifier (rank-50 LoRA on the residual-stream writers, fitted per layer with L-BFGS) with Optuna's TPE sampler co-minimizing refusals and KL divergence, 260 trials, run on Apple Silicon through an MLX backend for Heretic against an 8-bit MLX copy of the text model. The adapters stay in fp32 on top of the quantized weights, matching PEFT on torch.
2. Pick. Trial 264 — the parameters above (end_layer_index is exclusive, so layers 41–53). Before export the trial was restored from its parameters and re-scored; it reproduced its stored 6/100 and 0.1092 exactly.
3. Merge. W + B·A computed in fp32 against the original BF16 weights (not the 8-bit search copy) and rounded to BF16 once.
4. Graft. The 19 merged tensors were written into NVIDIA's Omni checkpoint under language_model.*. Checked before writing that Omni's originals equal the text weights byte for byte, and after writing that the grafted tensors equal the merge; every other tensor and file is copied unchanged.

Checked after the build: the vision tower still reads images correctly, and the MTP head still drafts (53% of draft tokens accepted on a llama.cpp Q8_0).

Responsible use

The edit lowers refusals. You decide how to use the model and you answer for it under the laws that apply to you. Intended uses: research, evaluation, red-teaming, creative writing and private deployment. Do not use it to produce illegal content or to target people, or in a public service without your own filtering and moderation.

License

OpenMDW 1.1, inherited from nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16. NVIDIA's bias.md, explainability.md, privacy.md and safety.md are kept in this repo; they describe the original model.