NVIDIA Nemotron 3.5 Super VL 120B-A12B — MLX 3bit mixed (text)
Text conversion of nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16.
- Quantization: affine mixed precision; 26 interior MoE layers at 2-bit/group-128, 14 boundary MoE layers at 3-bit/group-128, other eligible projections at 4-bit/group-64, and embeddings at 8-bit/group-64. Output head, routers and latent projections retain checkpoint precision. Per-module config is authoritative.
- Artifact size: 42.764 GB. Targets 64 GB unified memory for short contexts. Measured MLX allocator peak: 45.903 GB for 1,024 prompt tokens plus 64 generated tokens, with prefill chunks of 256. Measurements use a 256 GiB M3 Ultra; a physical 64 GB Mac has not been tested. Start at 1K context and leave memory for macOS and other applications.
- Text only: the vision tower, image projector and unused MTP head are omitted.
- Declared context limit: 1,048,576 tokens; this does not promise that the full window fits the recommended memory.
Requirements
Requires Rapid-MLX 0.16.2 or newer. The config retains
nemotron_h and opts into Rapid's reviewed precision adapter. Stock loaders
do not apply that adapter; only Rapid-MLX is validated for this conversion.
rapid-mlx serve rapid-mlx/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-3bit
Reasoning and XML tool calling use the original tokenizer/chat template. MTP speculative decoding and image input are not enabled.
License
Original model by NVIDIA under OpenMDW 1.1. This conversion inherits the original model's license terms.