NVIDIA-Nemotron-3.5-Bolt-74B-A9B — BF16
MLX conversion of nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16, pinned to revision bca0a8cb74200a2eb7ca56eb8428b821c829bd6e. Converted and tested on an Apple M5 Max with 128 GB unified memory using MLX-VLM.
BF16 weights with no weight quantization. Auxiliary floating-point tensors retain their original source dtypes. The weight files alone total about 150 GB; fully resident inference requires additional memory for caches and runtime allocations.
This model accepts text, images, and video and generates text.
| Property | Value |
|---|---|
| Parameters | 74B total, 9B active |
| Architecture | Mamba-2 / Transformer hybrid MoE with C-RADIO vision encoder |
| Weight files | 149.77 GB (decimal) |
| Source context limit | 262,144 tokens |
| Native MTP head | Not included in the upstream Bolt checkpoint |
Usage
Install the tested MLX-VLM revision (includes the Nemotron 3.5 port):
pip install 'git+https://github.com/Blaizzy/mlx-vlm.git@e17c49a5061b4399df3f5bda206ea2a63d45486a'
python -m mlx_vlm generate \
--model nativ-community/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 \
--prompt "Write a short Python function that returns the first n Fibonacci numbers." \
--thinking-mode disabled \
--max-tokens 512 \
--temperature 1.0 --top-p 0.95
For images, add --image /path/to/image.jpg and use a prompt such as "Describe this image.". For video, use --video /path/to/video.mp4 --fps 2 with a video question. The included processor and chat template handle media tokens. Use --thinking-mode enabled for reasoning, with a larger output budget.
Conversion and validation
python -m mlx_vlm convert \
--hf-path nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 \
--revision bca0a8cb74200a2eb7ca56eb8428b821c829bd6e \
--mlx-path ./NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16
The saved checkpoint was loaded with strict weight matching using MLX 0.32.4. Text generation returned 4 for 2 + 2; the image check identified the red square; the video check identified blue at the end of a red-to-blue clip. Because this checkpoint exceeds the test machine's 128 GB of unified memory, BF16 inference validation streamed MoE layer weights from disk. Fully resident inference speed and peak memory were not benchmarked.
These are short smoke tests. Memory and speed depend on prompt length, media resolution, and generation settings; the source context limit was not benchmarked. See conversion_info.json for provenance and measurements.
License and attribution
The original model is by NVIDIA Corporation. These weights retain the OpenMDW-1.1 license. See NOTICE for conversion details and the original model card for training, evaluation, and intended-use information.