Nemotron 3.5 Lightning 30B-A3B — Ternary Quantized (MLX) by Nota
Ternary (1.58-bit) Nemotron 3.5 Lightning 30B-A3B.
60 GB (BF16) → 9.6 GB
6.25x smaller
MLX has no 1.58-bit container, so the weights ship in a 2-bit affine container.
The stored values are ternary: {-a, 0, +a}.
This is not our fully recovered model. It is an intermediate checkpoint released for public use, recovering about 89% of the teacher's generative benchmark average (83.82 vs 93.80 across the seven tasks below).
Interested in a further recovered model? Get in touch
Performance
Korean and English generative benchmarks, temperature 0.7, full test sets, 32k token generation budget.
| Language | Category | Benchmark | BF16 teacher | This model | Recovered |
|---|---|---|---|---|---|
| English | Math | GSM8K | 95.00 | 89.31 | 94.0% |
| Math | MATH500 | 97.60 | 81.60 | 83.6% | |
| Code | HumanEval | 96.34 | 90.85 | 94.3% | |
| Instruction | IFEval | 92.05 | 83.36 | 90.6% | |
| Korean | Math | HRM8K | 91.05 | 83.24 | 91.4% |
| Code | HumanEval-ko | 95.12 | 85.37 | 89.7% | |
| Instruction | IFEval-ko | 89.46 | 73.04 | 81.6% | |
| Average | 93.80 | 83.82 | 89.4% |
Format
MLX native affine quantization, no custom kernel and no runtime shim:
bits 2
group_size 64
mode affine
levels {0, 1, 2} level 3 is unused
bias == -scale so dequantisation is scale * (q - 1) = {-a, 0, +a}
Load
from mlx_lm import load
model, tokenizer = load("<repo>")
The architecture has to be implemented in your mlx-lm / mlx-vlm build.
License
OpenMDW-1.1, same as the base model
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.
See LICENSE.