Model Overview

Description:

The NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash model is the DFlash speculative decoding draft checkpoint for NVIDIA-Nemotron-3.5-Bolt-E4B, a reasoning language model post-trained by NVIDIA from Google's Gemma 4 E4B IT. For more information, please check Nemotron-3.5-Bolt-E4B-BF16. The NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash model incorporates DFlash speculative decoding with NVIDIA Model Optimizer and is intended for lower-latency serving of NVIDIA-Nemotron-3.5-Bolt-E4B on data centre and workstation GPUs.

This repository provides the draft model quantized to NVFP4 (4-bit floating point) with NVIDIA Model Optimizer. By default the checkpoint is served as W4A16 (NVFP4 weights with 16-bit activations). The same weights also carry calibrated activation scales for W4A4 (NVFP4 weights and activations) on the w4a4 revision; see Usage.

This model is ready for commercial or non-commercial use.

License/Terms of Use:

GOVERNING DOWNLOAD TERMS: Use of the model is governed by the OpenMDW License Agreement, version 1.1. ADDITIONAL INFORMATION: Apache License 2.0 (Gemma 4 E4B IT).

Deployment Geography:

Global

Use Case:

Developers deploying NVIDIA-Nemotron-3.5-Bolt-E4B for reasoning, chat, tool use, and agentic workflows that benefit from lower-latency speculative decoding on data centre and workstation GPUs. This release is intended for DFlash-assisted serving of NVIDIA-Nemotron-3.5-Bolt-E4B rather than as a standalone model checkpoint.

Release Date:

Hugging Face 10/23/2026 via https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16-DFlash-NVFP4

References

• NVIDIA Model Optimizer: https://github.com/NVIDIA/Model-Optimizer
• Nemotron-3.5-Bolt-E4B-BF16 model card
• Gemma 4 E4B IT model card
• OpenMDW License Agreement, version 1.1
• DFlash: Block Diffusion for Flash Speculative Decoding

Model Architecture:

Architecture Type: Transformer
Network Architecture: Other - DFlash draft model (five Qwen3-style decoder layers)
Number of Model Parameters: 1.03B (1,031,568,640), of which 671M (671,088,640) are an NVFP4-quantized copy of the target model's output head. The draft model shares the token embeddings of the target model, so they are not included in this checkpoint.

For more information about the underlying model's architecture, please see Nemotron-3.5-Bolt-E4B-BF16.

Input:

Input Type(s): Text, Image, Audio
Input Format(s): Text: String; Image: Red, Green, Blue (RGB); Audio: Waveform
Input Parameters: Text: One-Dimensional (1D); Image: Two-Dimensional (2D); Audio: One-Dimensional (1D)
Other Properties Related to Input: Maximum context length of 131,072 tokens. Image and audio inputs are encoded by the target model; the draft model operates on the token sequence and attends to the most recent 1,024 tokens through its sliding-window attention.

Output:

Output Type(s): Text
Output Format: String
Output Parameters: One-Dimensional (1D): Sequences
Other Properties Related to Output: The draft model proposes up to 7 candidate tokens per decoding step, and the target model verifies them, so the final output is generated by the target model.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Supported Runtime Engine(s):

• vLLM 0.31.0 or newer

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Blackwell

Preferred Operating System(s):

• Linux

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s):

v1.0: NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash is the DFlash speculative decoding version of NVIDIA-Nemotron-3.5-Bolt-E4B, trained with NVIDIA Model Optimizer 0.48.0 and quantized to NVFP4 with NVIDIA Model Optimizer post-training quantization (GPTQ weight calibration and calibrated activation scales). The default revision is served as W4A16 and the w4a4 revision as W4A4. To integrate it into an AI system, serve it with vLLM as the speculative decoding draft model of NVIDIA-Nemotron-3.5-Bolt-E4B (see Usage); applications then query the target model's endpoint as usual.

Training and Evaluation Datasets:

Training Dataset:

For more information about the underlying NVIDIA-Nemotron-3.5-Bolt-E4B model, please visit the model card - Nemotron-3.5-Bolt-E4B-BF16.

Link: Nemotron-Post-Training-Dataset-v2, only prompts from the datasets were used for data synthesis (the original responses from GPT were not used), which is then used to train the DFlash modules.
Data Modality: Text
Text Training Data Size: 1 Billion to 10 Trillion Tokens
Data Collection Method by dataset: Hybrid: Automated, Synthetic
Labeling Method by dataset: Hybrid: Automated, Synthetic
Properties: About 3.7 million multilingual text samples featuring prompts spanning math, code, STEM, and conversational topics. Each sample includes a synthetic response generated by the target model; the corpus mixes responses generated with thinking enabled, which include their reasoning traces, and with thinking disabled.

Testing Dataset:

Link: SPEED-Bench, MT-Bench
Data Collection Method by dataset: Hybrid: Automated, Manually-Collected, Synthetic
Labeling Method by dataset: Hybrid: Automated, Manually-Labeled, Synthetic
Properties: The SPEED-Bench qualitative split (880 text prompts in 11 task categories, mostly English) and MT-Bench (80 English two-turn text questions in 8 categories) were used during development to measure acceptance length and select checkpoints.

Evaluation Dataset:

The model is evaluated on the SPEED-Bench and MT-Bench datasets for speculative decoding benchmarking.

Data Collection Method by dataset: Hybrid: Automated, Manually-Collected, Synthetic
Labeling Method by dataset: Hybrid: Automated, Manually-Labeled, Synthetic
Properties: All prompts are text and mostly in English. The SPEED-Bench qualitative split has 880 prompts in 11 task categories (such as coding, math, reasoning, and writing, plus a multilingual category covering 23 languages); its throughput split's 32k-token bucket has 1,536 long-context prompts in three entropy categories; MT-Bench has 80 two-turn questions in 8 categories. Acceptance length over the target model's responses measures the quality of the draft model.

Inference:

Acceleration Engine: vLLM
Test Hardware: NVIDIA Blackwell GB300; NVIDIA Blackwell B300; NVIDIA Blackwell RTX PRO 6000 Max-Q Workstation Edition

DFlash Speculative Decoding

NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash is a DFlash draft model for NVIDIA-Nemotron-3.5-Bolt-E4B. In each decoding step, the draft model takes the last verified token together with hidden states captured from five layers of the target model, and proposes the next 7 tokens of an 8-token block in a single non-causal forward pass. The target model verifies the proposed tokens in one forward pass and accepts the longest valid prefix, so more than one token can be generated per step. The mean number of tokens generated per step is the acceptance length, with a ceiling of 8.

The draft model was trained on a mix of thinking-enabled and thinking-disabled conversations. The acceptance lengths reported below were measured with thinking enabled.

Usage

To serve the checkpoint with vLLM (0.31.0 or newer):

vllm serve nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 \
  --speculative-config '{"model": "nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16-DFlash-NVFP4", "method": "dflash", "num_speculative_tokens": 7, "attention_backend": "flash_attn"}'

The default revision runs the draft model as W4A16. To run it as W4A4 (NVFP4 weights and activations), select the w4a4 revision:

vllm serve nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 \
  --speculative-config '{"model": "nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16-DFlash-NVFP4", "revision": "w4a4", "method": "dflash", "num_speculative_tokens": 7, "attention_backend": "flash_attn"}'

W4A4 needs NVFP4 activation kernels, which run on NVIDIA Blackwell GPUs; on GB300, use a vLLM build for CUDA 13.

Example request with thinking enabled, as in the evaluation below:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16", "messages": [{"role": "user", "content": "Explain speculative decoding in two sentences."}], "chat_template_kwargs": {"enable_thinking": true}}'

Evaluation

All acceptance lengths below were measured with this checkpoint in its default W4A16 configuration against the target model with thinking enabled, 4,096 output tokens, and concurrency 16 on one NVIDIA Blackwell GB300 GPU. Compared with the unquantized BF16 draft model, NVFP4 quantization lowers acceptance length by about 1% for W4A16 and by about 2% for W4A4.

Acceptance length on MT-Bench with draft length 7:

Acceptance length on SPEED-Bench (qualitative split) with draft length 7:

Acceptance length on the SPEED-Bench throughput split (32k-token context) with draft length 7:

Model Limitations:

The base model may generate inaccurate, incomplete, or otherwise undesirable responses, even when prompts are benign. It may reflect biases or content artifacts present in its training data, and output quality can vary by domain, language, reasoning depth, and context length. Developers should add application-specific safeguards and evaluate the system in the intended deployment environment before production use. The speedup from the draft model depends on the serving configuration.

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.

For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards.

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.

SUBCARDS:

Explainability

Bias

Safety & Security

Privacy

The datasets used to train NVIDIA-Nemotron-3.5-Bolt-E4B-DFlash did not include personal data.

NVIDIA collected and used the training data in compliance with applicable data protection and privacy laws. The datasets used to train this model were reviewed before release. Please review NVIDIA’s Privacy Policy for more information.