NVIDIA-Nemotron-3.5-Bolt-74B-A9B — mxfp8
MLX conversion of nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16, pinned to revision bca0a8cb74200a2eb7ca56eb8428b821c829bd6e. Converted and tested on an Apple M5 Max with 128 GB unified memory using MLX-VLM.
Eligible language-model weights and projector linear weights use MXFP8 weight-only quantization, 8 bits, group size 32. The vision encoder remains BF16; unquantized tensors retain their source dtypes.
This model accepts text, images, and video and generates text.
| Property | Value |
|---|---|
| Parameters | 74B total, 9B active |
| Architecture | Mamba-2 / Transformer hybrid MoE with C-RADIO vision encoder |
| Weight files | 77.92 GB (decimal) |
| Source context limit | 262,144 tokens |
| Native MTP head | Not included in the upstream Bolt checkpoint |
Usage
Install the tested MLX-VLM revision (includes the Nemotron 3.5 port):
pip install 'git+https://github.com/Blaizzy/mlx-vlm.git@e17c49a5061b4399df3f5bda206ea2a63d45486a'
python -m mlx_vlm generate \
--model nativ-community/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-mxfp8 \
--prompt "Write a short Python function that returns the first n Fibonacci numbers." \
--thinking-mode disabled \
--max-tokens 512 \
--temperature 1.0 --top-p 0.95
For images, add --image /path/to/image.jpg and use a prompt such as "Describe this image.". For video, use --video /path/to/video.mp4 --fps 2 with a video question. The included processor and chat template handle media tokens. Use --thinking-mode enabled for reasoning, with a larger output budget.
Conversion and validation
python -m mlx_vlm convert \
--hf-path nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 \
--revision bca0a8cb74200a2eb7ca56eb8428b821c829bd6e \
--mlx-path ./NVIDIA-Nemotron-3.5-Bolt-74B-A9B-mxfp8 -q --q-mode mxfp8 --q-bits 8 --q-group-size 32
The saved checkpoint was reloaded with strict weight matching using MLX 0.32.4. Text generation returned 4 for 2 + 2; the image check identified a red square and blue circle; the video check identified a red-to-blue transition. A 64-token greedy code sample decoded at 47.6 tokens/s on the Apple M5 Max. Peak MLX memory across these short checks was 85.08 GB.
These are short smoke tests. Memory and speed depend on prompt length, media resolution, and generation settings; the source context limit was not benchmarked. See conversion_info.json for provenance and measurements.
License and attribution
The original model is by NVIDIA Corporation. These weights retain the OpenMDW-1.1 license. See NOTICE for conversion details and the original model card for training, evaluation, and intended-use information.