中文Sign up

rapid-mlx/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-3bit

Source: 🤗text-generationopenmdw-1.174B params9B activated148 GBsafetensors✓ 8 checksumsupdated 5d ago
or submit a torrent

NVIDIA Nemotron 3.5 Bolt 74B-A9B — MLX 3bit (text)

MLX conversion of nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 for Apple Silicon, built and tested with Rapid-MLX.

  • Quantization: mixed: routed experts mostly 3-bit (group size 128) with six interior MoE layers at 2-bit, other projections 4-bit, embeddings 8-bit; lm_head, routers and latent projections kept in BF16 (~3.42 bits per weight)
  • Size: 31.7 GB — recommended for Macs with 48 GB+ unified memory
  • Modality: text only. The vision tower and projector from the original multimodal checkpoint are not included.
  • Context: up to 256K tokens (hybrid Mamba/attention, small KV cache)

Requirements

This checkpoint uses the nemotron_h_puzzle architecture (Mamba/MoE/attention hybrid with per-layer heterogeneous latent MoE). It is not loadable by stock mlx-lm at the time of publishing; use Rapid-MLX (support lands with PR #4679; requires the first release that includes it):

curl -fsSL https://rapidmlx.com/install.sh | bash
rapid-mlx serve rapid-mlx/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-3bit

The server exposes OpenAI- and Anthropic-compatible APIs with reasoning and tool calling enabled automatically for this model.

License

Original model by NVIDIA, released under the OpenMDW 1.1 license. This conversion inherits the original license terms.