Nemotron 3.5 Super 120B-A12B at 1.4 bits per weight (CQ1.4bit)

Read the launch post and the companion analysis, Why quantization can break multi-step workflows.

Note

• DeepSWE — long-horizon collapse: With OpenCode and no network, the model card reports 24/113 (21.2%) for BF16 and 7/113 (6.2%) for CQ. A saved CQ aggregate records 7/110; the three-task denominator discrepancy remains unresolved. These are single agent runs per configuration. Long engineering runs need separate reliability evaluation even when aggregate benchmark scores are retained. PrismML’s Bonsai 27B also identifies long-horizon agentic coding as a limitation. On other benchmarks, ACBench reports real-world agent degradation with AWQ/GPTQ, and a QuIP# evaluation reports CodeLlama-7B falling from 25% to 8% on McEval-Java. DeepSWE is reported separately from the 18-benchmark chart, table, and average.
• GSM8K: Flexible extraction is reported. Strict extraction gives 52.67 / 65.67 (BF16 / CQ); BF16’s lower strict score reflects answer formatting.
• NeMo Gym: GPQA, IFBench, MMLU-ProX, SciCode, GraphWalks, and AA-LCR follow NVIDIA’s recipes for sampling, repeats, output caps, and scoring, with the qualifications below.
• MMLU-ProX: The Lite set covers NVIDIA’s 10 reported languages. Gym’s strict language-specific patterns can miss English “The answer is (X)” endings in non-English responses from either model.
• GraphWalks / AA-LCR: GraphWalks includes prompts fitting a 262K context with a 100K output cap; NVIDIA’s set extends to 1M. AA-LCR uses the BF16 original as judge, thinking off, temperature 0; two questions containing raw chat-template tokens are excluded.
• SciCode: Gym omits class declarations from two prefilled setup steps, causing dependent steps to fail for both models.
• Vision: V* uses lettered multiple choice with thinking on and extracts the answer letter; MathVision uses MATH-V’s prompt and scorer. Both use temperature 0.6, top_p 0.95.
• BigCodeBench: “Leaderboard” results use chat, thinking off, temperature 0; the official leaderboard uses a completions endpoint.

Quantization and storage

For the quantization method, see the Cactus Quants paper.

All figures combine the backbone, vision tower, projector and MTP head; shared embeddings and the output head are counted once. Quantized rows include their norms and scales. Sizes use decimal GB/MB.

Installation

Apple silicon, macOS 26 or newer, and Python 3.12.

pip install -U 'cactus-compute[mlx]'

Submit the access form on this page once; approval is automatic. Then sign in locally:

hf auth login

Examples

Generate text:

cactus-mlx generate \
  --model Cactus-Compute/nemotron-3.5-120b-cq \
  --prompt 'Write a Python function is_prime(n).' \
  --max-tokens 256

Greedy generation uses verified MTP by default. Add --no-mtp to disable it.

Ask about an image:

cactus-mlx generate \
  --model Cactus-Compute/nemotron-3.5-120b-cq \
  --image photo.jpg \
  --prompt 'Describe this image.'

Start a server:

cactus-mlx server \
  --model Cactus-Compute/nemotron-3.5-120b-cq \
  --port 8080

The server accepts OpenAI image_url content parts with HTTP(S) or base64 data URLs. Repeat --image for multiple images.

License: Apache 2.0 for Cactus code and contributions; upstream model materials retain OpenMDW 1.1.

Citation

@misc{mosoyan2026cactusnemotron,
  author = {Karen Mosoyan and Noah Cylich and Henry Ndubuaku},
  title = {{Nemotron-3.5-SuperVL-120B with Cactus Quants}},
  year = {2026},
  howpublished = {Hugging Face model repository},
  url = {https://huggingface.co/Cactus-Compute/nemotron-3.5-120b-cq}
}