NVIDIA Nemotron 3.5 Bolt E4B · Atomic Dynamic GGUF

Atomic Dynamic (AD) quants of NVIDIA's Nemotron 3.5 Bolt E4B, built from NVIDIA's original BF16 weights. From 4.0 GB to 8.0 GB on disk.

• Bolt E4B is NVIDIA's small reasoning and chat model, post-trained from Google's Gemma-4-E4B-it.
• Every file here was measured against the BF16 original on one stand, next to the other publishers' files.

---

[!NOTE] Built on release day, 23 October. Every AD- file below AD-Q6_K was rebuilt the same evening with a better layout (below). AD-Q4_K_M uses our own importance matrix, collected on the BF16 weights over a corpus rendered through this model's chat template (imatrix/imatrix.gguf in the metrics repo). The other files use Unsloth's published one (imatrix_unsloth.gguf from unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF): we built AD-Q5_K_M to AD-IQ3_S with both, and on held-out code the two came out within noise of each other, slightly ahead with Unsloth's at 3 bits. AD-IQ3_XXS (3.82 GB) was taken down the same evening: it picked the original's next word only 74.6% of the time, below the 75% we publish at.

Pick a file

KL divergence is how far the quantized model's predictions drift from the original weights. Lower is better, and zero means identical. top-1 is how often it picks the same next word the original would have picked. Raw logs are in the metrics repo.

Other builds, same stand

[!IMPORTANT] This model is unusually sensitive to quantization. Even a plain 8-bit copy picks the original's next word only 96% of the time on our text (most models reach 98% or more), and it falls quickly below 5 GB. AD-IQ3_S changes the next word too often (top-1 below 80%): use it only if nothing larger fits.

AD- marks an Atomic Dynamic layout. The rest of the name is the type of the FFN up projections, the largest tensor group. Q8_0 is plain 8-bit throughout. Compare files by size on disk, not by name.

What we measured on this model: the FFN and attention are what matter. About a third of the weights are one table, the per-layer embeddings (262,144 rows of 42 x 256), and the embedding is also the output head; at the 4-bit point, putting either of them back at 8 bits changed almost nothing. So the files below AD-Q6_K keep the whole FFN one step up from what their name suggests (in AD-Q4_K_M: q5_k, with the first three and last eight layers at q6_k), lift attention one or two steps (in AD-Q4_K_M: query q5_k, output q6_k, which alone took its KL divergence from 0.107 to 0.094 for 66 MB), and take the bytes from the table (iq4_nl) and the head (q4_k or q5_k). The last 18 layers reuse the key-value cache of earlier ones, so their own key and value projections, which NVIDIA's checkpoint still carries, are never read; they are stored at q4_0.

Running it

llama-server -m NVIDIA-Nemotron-3.5-Bolt-E4B-AD-Q4_K_M.gguf -ngl 99 -c 32768 --jinja \
  --temp 1.0 --top-p 0.95

The chat template is inside every file. Thinking is off unless you turn it on:

llama-server -m NVIDIA-Nemotron-3.5-Bolt-E4B-AD-Q4_K_M.gguf -ngl 99 -c 32768 --jinja \
  --temp 1.0 --top-p 0.95 --chat-template-kwargs '{"enable_thinking": true}'

Temperature 1.0 and top-p 0.95 are NVIDIA's recommended settings, for thinking on and off.

Images and audio

mmproj-NVIDIA-Nemotron-3.5-Bolt-E4B-F16.gguf (0.99 GB) is the projector for the image and audio towers in NVIDIA's checkpoint, shared by every quant. NVIDIA serves this model as text only, and we have not tested image or audio input yet.

How we measured

Reference: the original BF16 weights, converted to GGUF and run unquantized. Held-out text: eval_neutral from our calibration dataset (30 languages), never used for calibration, 24 chunks at 4096 context. Every file in the comparison, ours and the others, was run with the same reference logits, text, context and llama.cpp build (36e36e63) on one A100-SXM4-40GB.

A second held-out text, eval_code (16 chunks at 4096, its own BF16 reference): AD-Q4_K_M 0.0311 KL divergence and 95.34% top-1, Unsloth's UD-Q4_K_XL 0.0363 and 95.16%; AD-Q5_K_M 0.0156 and 96.80%.

Model details

Gemma 4 E4B architecture: 42 layers, hidden size 2560, a sliding window of 512 tokens with a full-attention layer every sixth, the last 18 layers reusing the key-value cache of earlier ones, per-layer embeddings, vocabulary 262,144, tied input and output embeddings. Context up to 64K tokens (NVIDIA's figure; we measured at 4096). Converted with upstream llama.cpp's converter at 36e36e63; the files need a llama.cpp with gemma4 support.

License

Use of the model is governed by NVIDIA's OpenMDW License Agreement, version 1.1. NVIDIA post-trained it from Google's Gemma 4 (Gemma-4-E4B-it), whose use is governed by the Apache License, Version 2.0.

Quantized by AtomicChat.