NVIDIA-Nemotron-3.5-Bolt-74B-DERISKED GGUF

GGUF conversions of Blackfrost-AI/NVIDIA-Nemotron-3.5-Bolt-74B-DERISKED-BF16, a 74B-parameter hybrid mamba / MoE reasoning model with a RADIO vision tower, aimed at security operations, detection engineering, incident response, and malware analysis.

Every trunk below passed a short Vulkan generation test on consumer hardware (AMD Radeon 680M, 16 GB shared) before it was uploaded. Each NAME.py file beside its .gguf is that test's transcript, including the Python function the model was asked to write. Nothing was uploaded untested.

Architecture

NemotronH_Omni_Reasoning_V3 (nemotron_h_omni):

• 88 blocks in a repeating mamba / MoE / mamba / MoE pattern, with 8 attention blocks.
• Hybrid mamba2 SSM attention with NoPE attention layers, sigmoid routing, routed_scaling_factor 5.0, norm_topk_prob.
• Latent MoE: 9 routed experts per token plus 1 shared expert, latent size 1024.
• 262144-token context, 131072-token vocabulary.

The expert count varies per block: 240, 304, 368, 432 or 496. These GGUFs carry a per-layer expert_count array instead of one global scalar, alongside the per-layer expert_used_count and expert_feed_forward_length arrays that Puzzle-style models already use. Upstream llama.cpp reads expert_count as a single u32, so until per-layer expert counts land upstream, use a build that supports them (the fork these were produced with). The trunk is otherwise stock nemotron_h_moe.

Vision

mmproj-NVIDIA-Nemotron-3.5-Bolt-74B-DERISKED-f16.gguf holds the vision tower: NVIDIA RADIO v2.5-h ViT (1280-wide, 32 layers, 16 heads, patch size 16) with its 3 class tokens and 7 register tokens, position embeddings downsampled to a fixed 32x32 patch grid for 512x512 input, then a RMSNorm, a 2x2 pixel-shuffle merge (downsample_ratio 0.5), and a relu2 MLP projector into the 4096-wide text space. One mmproj works with every trunk. The checkpoint ships no audio or video tensors, so none are exported.

The trunk itself contains no MTP draft head (the config has num_nextn_predict_layers: 0), so there are no mtp- files here and no trunk carries one folded in.

The vision path was verified end to end against the uploaded Q4_K_M: asked to describe a plain red square image, the model answered "This is a solid red square."

Files

Each trunk is standalone; pair any of them with the mmproj for vision.

Each quant also has a sibling .py file recording its smoke test run, including the generated Python function.

Quant choices

No importance matrix. The UD- files are a per-layer type mix in the style of dynamic quants, not Unsloth Dynamic 3.0, and not calibrated.

• Q8_0 - everything Q8_0 except norms, biases, SSM state tensors and the router gate, which stay F32.
• Q4_K_M - Q4_K on most 2D weights, Q6_K on output and token embeddings, Q5_K on attention v, Q5_0 on down projections.
• UD-Q4_K_XL - first and last layers, attention, SSM and latent projections at Q5_K or Q6_K; output and token embeddings Q8_0; experts Q4_K with Q5_K on selected layers. The GGUF file-type field still says Q4_K_M.
• UD-IQ4_XS - same layer split, IQ4_XS where row width allows, IQ4_NL where it does not.
• MXFP4_MOE - MXFP4 on expert tensors only; attention, SSM, router, output and embeddings stay Q8_0.
• Q3_K_M, Q3_K_S, Q2_K, Q2_K_S - K-quants. Down projections and other rows that cannot use a 256-wide block fall back to Q4_0 or Q5_0.

Rows whose width is not a multiple of a quant's block size fall back to the next usable type automatically; expert down projections are 1024-wide and expert up projections vary (1536 to 2304), both K-quant friendly.

Usage

llama-cli -m NVIDIA-Nemotron-3.5-Bolt-74B-DERISKED-Q4_K_M.gguf --mmproj mmproj-NVIDIA-Nemotron-3.5-Bolt-74B-DERISKED-f16.gguf --image chart.png -p "What does this alert chain imply?"

The model was tested with 512 tokens of context on the smoke box. Long context is not tested.

License

The model is built on NVIDIA Nemotron Labs Puzzle. The upstream Bolt repository ships an empty LICENSE file, so the OpenMDW-1.1 license text from the NVIDIA Puzzle release is mirrored here as LICENSE. Use of this model is governed by OpenMDW-1.1, including its notice and attribution terms.