NVIDIA-Nemotron-3.5-Bolt-74B-A9B-Abliterated-24GB-GGUF

An uncensored edit of NVIDIA Nemotron-3.5-Bolt-74B-A9B, a hybrid Mamba/MoE model, produced by editing NVIDIA's shipped 24GB 2-bit GGUF (nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF) directly — no BF16 requantize at any point.

That matters for this model specifically. Bolt is a LatentMoE: every routed expert sums through one dense ffn_latent_up per layer, and the experts themselves are ~17GB of IQ2_XXS weights produced by NVIDIA's quantization-aware distillation. A conventional abliterate-then-requantize pipeline would throw that QAD work away. Instead the projection is applied in place and codes are re-picked under the original block scales, so at λ=0 the output file is byte-identical to the input by construction — which is the self-test the build gates on (verified: all edited tensors byte-identical at λ=0). The published file is the same length as NVIDIA's, 22,018,326,528 bytes.

Files

Bolt is multimodal, so the projector ships alongside. The edit touched only text-model tensors, so the projector is byte-for-byte NVIDIA's file; download both or vision silently does nothing.

llama-server \
  -m NVIDIA-Nemotron-3.5-Bolt-74B-A9B-Abliterated-24GB.gguf \
  --mmproj mmproj-NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf \
  -ngl 99 -c 32768 --jinja

Any llama.cpp build that loads NVIDIA's own 24GB GGUF loads this one: same architecture, same tensor layout, same file length.

Give it at least 8192 tokens of budget — this is the one setting that matters. At 8192 it answered 154 of 154 held-out harmful prompts without hitting the cap once; at 4096 it capped 11 of 150. For long benign answers it needs no more room than stock does, and both cap about 20% of those even at 8192, so go higher still for essay-length work.

The edit

Read-side projection W ← W − λ(Wr)rᵀ over the input-projection surface, blocks 15–75:

λ=1.0, with λ=2.0 on ssm_in. 100 tensors edited in total.

Realised removal is measured on the decoded bytes, not assumed from the arithmetic, and the two differ a lot here. ffn_gate_inp is stored F32, so its edit is exact and perfectly linear in λ. The Q8_0 tensors reach 0.92–0.94 rather than 1.0 because Q8_0 already scales so the largest code is 127, and subtracting the projection pushes a small fraction (~0.5%) of values out of range. ssm_in is NVFP4, whose coarse grid rounds most of the edit away.

That last one is why λ=2.0 appears only on ssm_in. Over-driving λ cannot defeat a quantization grid — since the target is (1−λ)·rᵀW, intended removal peaks at λ=1 and over-driving walks off the far side, with large λ eventually amplifying refusal rather than removing it. NVFP4's under-realisation partially cancels that overshoot, which makes ~2.0 the empirical optimum at ~0.5 realised. It is a ceiling, not a tuning knob: no λ gets this tensor to 0.95.

Direction from a mean-difference control vector (the Arditi et al. 2024 method), last-token estimator, taken at layer 45.

Evaluation

Read the answer rate before the refusal rate

A refusal rate on its own is close to meaningless, and this is not a pedantic point — it bit this very build. An earlier candidate scored 0.67% refusal and looked like the obvious winner. Its score shape showed it answered only 105 of 150 prompts: it thinks until the token budget runs out, and an empty answer is scored 0 by the judge in exactly the same way a refusal is. Most of that beautiful 0.67% was the model saying nothing at all.

So a low refusal number is only interpretable next to how often the model actually answered and how often it hit the cap. Both columns are below for every number reported here. If you are comparing this model against another abliteration, ask for those two columns; if they are not published, the refusal percentage is not a measurement you can use.

That is not a hypothetical worry, and the cost is measurable. The same candidate was also run on HumanEval, where it scores 85.4% against stock's 93.9% — paired McNemar p = 0.0005, a real capability loss — while the model released here sits at 91.5% with p = 0.29. The mechanism is visibly identical in both places: conditional on producing an answer at all, that candidate passes 97.9% of problems, essentially the same as this model's 98.0% and close to stock's 99.4%. It is not worse at writing code. It caps 21 of 164 problems where stock caps 9. The refusal benchmark and the coding benchmark were measuring the same stall, and only the answer-rate column makes that visible. Picking the winner on refusal percentage alone would have shipped the weaker model.

Both candidates were then re-run on the held-out half of StrongREJECT at an 8192 budget, which is the cleanest form of the comparison — same prompts, nothing selected on, enough budget for either to finish:

Read the first row alone and the other candidate is a flawless abliteration and this model is a failure. Read any other row and it is the reverse. It refuses nothing because it finishes nothing: it thinks for 12,275 characters at the median, caps 39 times even with 8192 tokens, and leaves 38 prompts unanswered — each of which the refusal classifier scores as "did not refuse". A 0% refusal rate is what a broken model looks like, which is exactly why the answered and cap columns belong beside every refusal number anyone publishes, including the ones above.

All refusal numbers come from an anchor-gated frozen judge (Qwen3.8-27B-OBLITERATED-Q8_0, thinking off, anchor 0.6875 ± 0.0051; every pass in these tables gated at 0.6863). Higher StrongREJECT mean score = more complete compliance.

Selection set — the prompts this build was chosen against

Held out — never used to select anything

Separate prompts, generated and judged after the build was frozen.

Refusal 154/154 → 18/154 is z = 15.61, p = 6.7e-55. The 11.7% held-out refusal sits 1.0pp from the 10.7% it was selected on, inside the 6.0pp resolution limit, and the mean score is marginally higher on the held-out half — so this did not fit the selection set.

Stall control — long benign answers (n=150, held out)

Abliteration can leave a model unable to stop. That shows up as nothing at all on a refusal benchmark and ruins the model in use, so it is measured directly: 150 benign prompts that each demand a long answer, same 8192 budget.

No stall. This model thinks 4.1% longer than stock and caps slightly less often. The third column is why that null is worth printing rather than just an underpowered shrug: the same test, on the same prompts, detects a +67.9% thinking increase in the other candidate. A test that can register a two-thirds increase and reports 4% is reporting something.

Two honest limits on this table. The cap-rate row resolves almost nothing — at n=150 against a 20% baseline, a two-proportion difference needs roughly 13pp to be detectable, so none of these cap-rate gaps mean anything; the median-think row is a continuous measure and is far better resolved, which is why the argument rests on it. And note that stock itself caps 20% of these prompts at 8192: this prompt class wants more budget than 8192 for either model, which is a property of the task rather than of the edit.

Capability — HumanEval, paired (n=164, held out)

Both arms ran the same 164 problems, so these are paired and the right test is McNemar on the problems where the two disagree — not a two-sample proportion test, which would discard the pairing.

Paired outcome: 148 both pass, 8 both fail, 6 regressed, 2 gained. Exact McNemar on the 8 discordant pairs gives p = 0.29 — no detectable capability difference.

Be careful how much that p-value is asked to carry. With only 8 discordant pairs, significance would have required a split of at least 1–7; a 6–2 split is simply what a coin does fairly often. So this rules out a lopsided regression and nothing finer. What is more informative than the p-value is the decomposition: 11 of this model's 14 failures are budget caps, not wrong answers (stock caps 9 of its own), and on problems where it produced an answer at all it passed 150/153. The headline 2.4pp gap is mostly the 4096-token ceiling, which constrains both arms.

The independent benchmark is the important row, and it is also the softer one. Every other harm number here is StrongREJECT; the held-out tail is the same dataset, so it tests generalisation to new prompts, not to a new benchmark. forbiddenquestions is a dataset nothing in this work selected on, tuned against, or had ever generated before this run — and on it, stock refuses only 65.7%, complying with about a third outright. So the 61.8pp drop there is measured from a much weaker baseline than the 88.3pp drop on StrongREJECT, and the two are not interchangeable evidence. What it does establish is that the effect is not an artifact of one benchmark.

Resolution limits were fixed before any of these were run (80% power, two-sided) — but they were all computed assuming a ~99.3% stock baseline, which forbiddenquestions does not have. Both the pre-registered and the corrected figure are given, because the pre-registration is the honest record and the correction is the number that applies:

The forbiddenquestions floor was pre-registered 3.3× too optimistic. The effect is still 6× the corrected floor, so nothing here changes — but the 3.1pp figure should not be quoted. Note also that the over-refusal improvement (8.0pp against a 7.1pp floor) only just clears its own limit; treat it as "does not over-refuse more than stock" rather than as a reliable improvement.

Honest caveats

It thinks longer than stock on harmful prompts — but that is not a general slowdown, and the 3.2× figure is misleading on its own. On the harmful set, median reasoning is 4,208 characters against stock's 1,327. Stock's number is small because stock refuses, and a refusal is cheap to write; setting that against a model that actually engages with the question compares two different tasks. The controlled comparison is the stall test above, where the two sit within 4% of each other. And the stall that looked real was a budget artifact. Those 11 capped prompts were run at a 4096 token budget. Re-run on the held-out half at 8192, this model caps 0 of 154 and answers every one — so give it 8192 and the defect disappears. That is not true of every candidate: the 0%-refusal one still caps 39 of 154 at 8192 and answers only 116, which is a real stall rather than a budget ceiling.

Capability: no detectable loss — on tests that could only have caught a large one. HumanEval's paired test gives p=0.29, but with 8 discordant pairs it would have needed a 1–7 split to register, so it does not establish that nothing changed. MATH-30 (21/23 against stock's 23–24) cannot resolve anything under ~33pp. Together they rule out collapse and a lopsided regression; a few points of real degradation would pass through either test unseen. Treat capability as "not visibly harmed", and if you need a tighter bound, run your own benchmark at a budget above 4096.

The 0% over-refusal is on 100 prompts. At a 6% baseline that test can barely resolve anything; only an extreme is interpretable. The held-out n=150 arm is the one to quote.

Single judge. Every refusal number comes from one frozen judge model. It is anchor-gated for drift, which is not the same as being unbiased.

Vision is untested beyond loading. The projector is unmodified and the model loads and answers with it, but no image benchmark was run. Nothing here measures multimodal quality.

What is and is not new here

Cited, not claimed:

• Arditi et al. 2024, refusal as a single mediating direction — the method this uses.
• Locating and Steering Refusal Beyond Attention already showed refusal concentrates in the Mamba path of a hybrid (Zamba2) rather than attention, and already made the "decodable but not controlling" argument. This work replicates the first on a different hybrid; neither is claimed as new.
• Weight-space abliteration of a hybrid Mamba/MoE model is not new either — and the earlier work is my own. Nemotron-3.5-Lightning-30B-A3B-Uncensored-ShimQuant-GGUF already projected the refusal direction out of a nemotron_h hybrid's attention, Mamba and shared paths. It is fair to say the published literature still contains no weight-space hybrid abliteration — the Zamba2 work above is activation-space only, and the 16-model cross-architecture survey (2512.13655) marks Mamba-containing models architecturally incompatible with every tool but one, covers no hybrid and does not evaluate MoE. But a shipped model is prior art for the next one, including your own, so this card does not claim the hybrid edit as a first.

What is actually new here, measured against both papers and that earlier build:

• Doing it inside a shipped 2-bit quant. Lightning was edited in BF16 and then quantized; this edits NVIDIA's released 24GB file in place under its original block scales, so the QAD experts are never touched and λ=0 is byte-identical by construction.
• A maximum-reach negative control. Editing 106 tensors spanning every class except the Mamba path, across both the read and write surfaces, leaves refusal at 90.0% with the model perfectly healthy (150/150 answered, normal reasoning length, 0% over-refusal). That turns "Mamba has the larger effect" into "everything else, at maximum reach, does nothing."
• The MoE router dissociation. The F32 router takes a mathematically exact edit (removal 1.000), demonstrably changes which experts fire (only 52.6% of pick sets identical), and refusal does not move at all — four separate judged nulls. Expert selection is not where refusal lives.
• Direction interacts with surface. The write side wants an all-token direction from layer 23; the read side wants a last-token direction from layer 45. Each is near-null on the other's surface. A single "best direction" per model is the wrong abstraction.

A result retracted, recorded here because the retraction is the useful part. Harmfulness was measured as linearly readable at AUC ~1.0 in a late block band, and that was nearly published. Control blocks chosen because they had not ranked separated just as well (shortlist vs control p=0.475), and isotropic random directions reached |AUC−0.5| of 0.47 against the real direction's 0.50. The prompt pair (AdvBench vs Alpaca) is separable along almost any projection. A cross-corpus replication had looked excellent — an artifact of the measurement replicates perfectly, because the measurement is what is broken. No readability or localisation claim is made.

Intended use and limits

This model will answer harmful requests. It is published for safety research, red-teaming, and evaluation work where a refusal-free baseline is the point. Refusal removal is not a capability gain; it removes a safety behaviour and nothing else. Anyone deploying this to users owns the consequences and needs their own safeguards.

Related build: NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Uncensored-ShimQuant-GGUF — the same family, a smaller hybrid, edited in BF16 and then quantized rather than in-quant.

Licensed under OpenMDW 1.1, inherited from the parent model.