AxionML Nemotron-3-Ultra-550B-A55B-NVFP4

Mirrored by AxionML for open-source serving and deployment use cases. Part of AxionML's effort to provide ready-to-serve quantized models for the community.

Released by NVIDIA. The weights in this repository are an unmodified copy of nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 (revision 02462641f13d3af838b904f48195b9bb8a1e4ebc). All credit belongs to NVIDIA.

NVIDIA Nemotron 3 Ultra is a frontier-scale reasoning and agentic model (550B total, 55B active) built on a hybrid LatentMoE architecture — interleaved Mamba-2 and MoE layers with select attention layers and Multi-Token Prediction. This checkpoint is NVIDIA's own release: the model was pre-trained with an NVFP4 recipe and NVIDIA publishes the NVFP4 weights directly.

About NVFP4 quantization: NVFP4 on Blackwell couples a compact E2M1 FP4 codebook with blockwise FP8 (E4M3) scaling over 16-element micro-blocks, so that 4-bit stored values remain numerically useful for neural-network computation. The E2M1 codebook provides a small, nonuniform set of representable magnitudes up to ±6 and relies on saturating behavior rather than IEEE NaN/Inf encodings to maximize usable range per bit. Using an FP8 block scale (rather than power-of-two-only E8M0) enables fractional scales and error-minimizing scale selection. On Blackwell Tensor Cores, native FP4 multipliers exploit E2M1 simplicity while higher-precision FP32 accumulation protects dot-product accuracy.

Ready for commercial and non-commercial use under OpenMDW-1.1.

Model Summary

Evaluation Results

Scores reported by NVIDIA (NeMo Evaluator SDK). See the upstream card for the full table.

Quantization Details

• Precision: NVFP4 pre-training recipe; released checkpoint is mixed NVFP4 (MoE linears) + FP8, with latent projections, MTP, attention projections and embeddings in higher precision
• KV cache: FP8

Usage

Deploy with SGLang

python3 -m sglang.launch_server \
    --model-path AxionML/Nemotron-3-Ultra-550B-A55B-NVFP4 \
    --tp-size 4 \
    --ep-size 4 \
    --context-length 262144 \
    --mem-fraction-static 0.85 \
    --mamba-scheduler-strategy extra_buffer \
    --mamba-backend flashinfer \
    --attention-backend trtllm_mha \
    --reasoning-parser nemotron_3 \
    --tool-call-parser qwen3_coder \
    --speculative-algorithm EAGLE \
    --speculative-num-steps 5 \
    --speculative-eagle-topk 1 \
    --speculative-num-draft-tokens 6 \
    --trust-remote-code

Deploy with vLLM

vllm serve AxionML/Nemotron-3-Ultra-550B-A55B-NVFP4 \
    --trust-remote-code \
    --tensor-parallel-size 4 \
    --enable-expert-parallel \
    --kv-cache-dtype fp8 \
    --max-model-len 262144 \
    --enable-chunked-prefill \
    --enable-prefix-caching \
    --reasoning-parser nemotron_v3 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder \
    --mamba-ssm-cache-dtype float16 \
    --mamba-backend flashinfer \
    --speculative-config '{"method": "nemotron_h_mtp", "num_speculative_tokens": 5}'

Tested upstream on 4x B200 with lmsysorg/sglang:v0.5.13 and vllm/vllm-openai:v0.22.0. For 1M context set SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 / VLLM_ALLOW_LONG_MAX_MODEL_LEN=1. With tools, send "chat_template_kwargs": {"enable_thinking": true, "force_nonempty_content": true}.

Limitations

The base model was trained on data that may contain toxic language and societal biases. The quantized model inherits these limitations. It may generate inaccurate, biased, or offensive content. Please refer to the original model card and the upstream quantized model card for full details.

Credits

• Base model: nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
• Model: nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 by NVIDIA
• Mirror: AxionML