How to Run NVIDIA Nemotron 3.5 Bolt E4B Without Refusals Locally

NVIDIA's refusal behaviour projected out of the released weights, baked into Atomic Dynamic GGUF builds and shipped as a 3.5 MB LoRA, measured against the original.

• On held-out JailbreakBench prompts refusals fall from 93.8% to 1.2%, on Russian red-teaming prompts from 76% to 0%, on the safe XSTest prompts from 9.2% to 0.4%, and with thinking on from 29 of 30 to none. MMLU: 68.50% before, 67.85% after (2000 questions, McNemar p = 0.21). Tool calls and long context are unaffected.
• One release-gate is not met, and we say so: on harmless prompts the first token moves further from the original than our 0.10 limit (KL 0.141). On this model even a random direction edited the same way moves neutral text noticeably (limitations).
• Gemma 4 normalises every write into the residual stream with its own weights before adding it, so the edit is computed through those norms: each matrix stops writing the refusal direction as it lands in the stream, not before the norm.
• Built on the same weights as our Nemotron 3.5 Bolt E4B quants; the LoRA loads on top of any of those files.

---

The builds

KLD and top-1 are against the original, unmodified BF16 model on our held-out neutral text (24 chunks of 4096 tokens), so they hold the edit and the quantization together. The edit alone, on the BF16 weights, is 0.159; the plain AD-Q6_K of the original model is 0.026 and AD-Q4_K_M 0.091 on the same text, so at AD-Q6_K almost all of the distance is the edit itself.

The baked files themselves, thinking off, on the held-out prompts: each refused 1 of the 81 English, none of the 100 Russian and 1 of the 250 XSTest prompts, with no empty or degenerate replies. Thinking on was checked on the edit (step 5), not on the baked files.

The LoRA. NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-LoRA.gguf is the same edit as a rank-1 adapter (126 matrices, 3.5 MB). It is built for scale 1.25, so load it with --lora-scaled, not --lora (which applies 1.0):

llama-server -m NVIDIA-Nemotron-3.5-Bolt-E4B-AD-Q4_K_M.gguf \
  --lora-scaled NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-LoRA.gguf 1.25 \
  -ngl 99 -c 32768 --jinja

Every number in this card for the edit itself was measured with the adapter on the BF16 weights; on a quantized base it was not measured separately.

Running it

hf download AtomicChat/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-GGUF \
  --include "NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-AD-Q6_K/*" --local-dir .
llama-server -m NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-AD-Q6_K/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-AD-Q6_K-00001-of-00001.gguf \
  -ngl 99 -c 32768 --jinja --temp 1.0 --top-p 0.95

The chat template is inside every file; thinking is off unless you pass --chat-template-kwargs '{"enable_thinking": true}'. Upstream llama.cpp with gemma4 support runs them; no patch is needed. For images and audio, use the projector from the original quants (the edit does not touch it); we did not test them on these files.

How we did it

Bolt E4B is NVIDIA's post-train of Gemma-4-E4B-it: 42 blocks, hidden size 2560, per-layer embeddings, and every block writing into the residual stream three times. The usual abliteration recipes assume a plain transformer and miss two of those writes and the norms around all three.

1. Refusal is a direction

A chat model decides to refuse before it writes the first word, and that decision shows up as one direction in its residual stream (Arditi et al., 2024). Stop the weights from writing along that direction and the model stops refusing.

2. Finding it

• Prompts. 416 harmful and 416 harmless English prompts (harmful_behaviors, harmless_alpaca), 128 and 128 Russian ones (Aya red-teaming, Russian Alpaca), each in the model's own chat template, thinking off. Validation and test prompts are separate sets and never reach this step.
• Direction. At each depth: mean over harmful prompts minus mean over harmless ones, English and Russian pooled, with the component along the harmless mean removed (the naive difference makes models refuse harmless look-alikes).
• Depth. How well the direction separates the two sets and how close the English and Russian directions are:

We took block 32, three quarters of the depth, as on our other releases: one direction for both languages, separation near its peak. Block 21 separates as well but its edit left 68% of the English validation prompts refused.

3. Where to cut, through the norms

Each Gemma 4 block writes three times: attention through attn_output, the FFN through ffn_down, and the per-layer path through its projection proj (each token's per-layer embedding, gated, 256 wide, back to 2560). Every one of the three passes its own RMSNorm with a learned weight g before it is added to the stream. Removing r from a matrix's output is not enough there: the norm multiplies each coordinate by its weight, and the direction comes back. So for each matrix we remove r ⊙ g (normalised) instead: then g ⊙ y has no component along r, and the norm's scalar divisor cannot add one. 126 matrices (3 × 42); the embeddings and the head are left alone.

Measured with the adapter at scale 1.0 on a text the edit never saw: what each family still writes along r, as a share of its output, is 0.00023 (attention), 0.00021 (FFN) and 0.00019 (per-layer path), against 0.001 as our limit.

4. The edit

For every one of the 126 matrices:

W' = W - s · r' r'ᵀ W        r' = (r ⊙ g) / |r ⊙ g|

screened as a rank-1 adapter on the BF16 weights:

1.25 is the smallest scale that clears both languages; every larger one moves the model further and costs MMLU. The random direction, edited through the same norms at the same scale, leaves the refusals as they were, so the effect comes from the direction.

5. Checks before release

The numbers in this card come from test prompts that played no part in any choice: 81 JailbreakBench behaviours, 100 Russian red-teaming prompts and the 250 safe XSTest prompts. The thresholds, and where this edit stands:

6. Baking and quantizing

The edit is applied to the BF16 weights in f32 and rounded to BF16 once. After rounding, every one of the 126 matrices keeps a quarter of its write along r' (the ideal 0.25) within 0.0011. The edit moved the importance matrix (cosine below 0.999 on 300 chunks), so it was collected again on the baked weights, over our nemotron-bolt-e4b corpus, and the baked BF16 was quantized with the same Atomic Dynamic layouts as those quants.

What changed, and what did not

Original against this edit on the released weights (BF16, the edit applied as an adapter), on the test prompts of step 5.

Refusal is counted by the opening of the reply, so it is indicative, not a judge.

Limitations

• The harmless-prompt gate is not met. The edit moves the first token on harmless prompts by KL 0.141, over our 0.10 limit, and neutral text by KLD 0.159. This model is unusually sensitive: a random direction edited the same way already moves neutral text by 0.125 and costs 1.2 MMLU points, more than the refusal direction does. A smaller scale did not clear the refusals in our screen.
• The refusal count reads the opening of each reply; a soft refusal phrased as an answer would be missed.
• MMLU and the thinking check use fixed question sets; they show a large change, not a small one.
• Image and audio input were not tested.

Responsible use

This model has its learned refusal behaviour removed, so it will answer requests the original declines. Nothing about it makes the output safe, correct or lawful, and any deployment needs its own access controls and policy enforcement.

License and credits

Use of the model is governed by NVIDIA's OpenMDW License Agreement, version 1.1. NVIDIA post-trained it from Google's Gemma 4 (Gemma-4-E4B-it), whose use is governed by the Apache License, Version 2.0. Base model Nemotron 3.5 Bolt E4B by NVIDIA. Direction, edit, quantization and evaluation by AtomicChat. The method follows Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).