中文Sign up

rapid-mlx/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-4bit

Source: 🤗text-generationopenmdw-1.174B params9B activated148 GBsafetensors✓ 10 checksumsupdated 5d ago
or submit a torrent

NVIDIA Nemotron 3.5 Bolt 74B-A9B — MLX 4bit (text)

MLX conversion of nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 for Apple Silicon, built and tested with Rapid-MLX.

  • Quantization: 4-bit affine, group size 64; router, latent projections and other precision-sensitive tensors kept at higher precision (~4.67 bits per weight)
  • Size: 43.3 GB — recommended for Macs with 64 GB+ unified memory
  • Modality: text only. The vision tower and projector from the original multimodal checkpoint are not included.
  • Context: up to 256K tokens (hybrid Mamba/attention, small KV cache)

Requirements

This checkpoint uses the nemotron_h_puzzle architecture (Mamba/MoE/attention hybrid with per-layer heterogeneous latent MoE). It is not loadable by stock mlx-lm at the time of publishing; use Rapid-MLX (support lands with PR #4679; requires the first release that includes it):

curl -fsSL https://rapidmlx.com/install.sh | bash
rapid-mlx serve rapid-mlx/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-4bit

The server exposes OpenAI- and Anthropic-compatible APIs with reasoning and tool calling enabled automatically for this model.

License

Original model by NVIDIA, released under the OpenMDW 1.1 license. This conversion inherits the original license terms.