NVIDIA-Nemotron-3.5-Super-VL-120B-A12B — NVFP4
MLX conversion of nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4, pinned to revision 82d93699acd3be9e4617c53360fe5b3f5dbeb1b3. Converted and tested on an Apple M5 Max with 128 GB unified memory using MLX-VLM.
The original NVFP4 weights, group size 16, are preserved in MLX packing with their FP32 global scales. The source's FP8 dense layers are decoded to BF16. The native MTP head is included in mtp.safetensors. This model accepts text, images, and video and generates text.
| Property | Value |
|---|---|
| Parameters | 120B total, 12B active |
| Architecture | Mamba-2 / Transformer hybrid MoE with C-RADIO vision encoder |
| Weight files, including MTP | 80.19 GB (decimal) |
| Source context limit | 1,048,576 tokens |
| Native MTP head | Included; use the same model ID as the drafter |
Usage
Install the tested MLX-VLM revision (includes Nemotron 3.5 and MTP support):
pip install 'git+https://github.com/Blaizzy/mlx-vlm.git@e17c49a5061b4399df3f5bda206ea2a63d45486a'
python -m mlx_vlm generate \
--model nativ-community/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \
--prompt "Write a short Python function that returns the first n Fibonacci numbers." \
--thinking-mode disabled \
--max-tokens 512 \
--temperature 1.0 --top-p 0.95
For images, add --image /path/to/image.jpg and use a prompt such as "Describe this image.". For video, use --video /path/to/video.mp4 --fps 2 with a video question. The included processor and chat template handle media tokens. Use --thinking-mode enabled for reasoning, with a larger output budget.
Native MTP
python -m mlx_vlm generate \
--model nativ-community/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \
--draft-model nativ-community/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \
--draft-kind mtp --draft-block-size 3 \
--prompt "Write a short Python function that returns the first n Fibonacci numbers, then explain its time complexity." \
--thinking-mode disabled \
--max-tokens 160 --temperature 0
MTP uses full-prompt prefill in this implementation. For video with MTP, add --gen-kwargs '{"video_pruning_rate": 0}' to retain every video token.
Conversion and validation
This export uses the native MLX-VLM loader and export script, preserving the existing quantized weights without another quantization pass or a blanket dtype cast. The script writes the MLX base weights, extracts the native MTP head, and includes it in the weight index. To reproduce from the pinned source snapshot:
hf download nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \
--revision 82d93699acd3be9e4617c53360fe5b3f5dbeb1b3 --local-dir ./source
python export_mlx.py ./source ./NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4
The exported model and bundled drafter were reloaded with strict weight matching using MLX 0.32.4. A 160-token greedy code sample matched the original NVIDIA checkpoint token for token. Normal generation and MTP also matched exactly on text, image, and video checks. The image check identified a red square on the left and a blue circle on the right; the video check identified a red-to-blue transition.
| 160-token code sample, Apple M5 Max | Decode speed |
|---|---|
| Normal generation | 31.0 tokens/s |
| MTP, draft block size 3 | 58.6 tokens/s |
| Observed speedup | 1.89× |
Peak MLX memory across the short text, image, and video checks was 88.08 GB. Memory and speed depend on prompt length, media resolution, and generation settings; the source context limit was not benchmarked. See conversion_info.json for provenance and measurements.
License and attribution
The original model is by NVIDIA Corporation. These weights retain the OpenMDW-1.1 license. See NOTICE for conversion details and the original model card for training, evaluation, and intended-use information.