SketchSSM calibration: Nemotron 3 Nano 30B-A3B Base (BF16 weights)
calibration.pt is a portable SketchSSM
calibration file for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16.
It contains the group-shared sketch basis and the Full-Gram allocation scores
from which the per-head rank table and the ordered frames for any mean rank are
derived, and the model layer index of each calibrated recurrent layer. It
contains no model weights.
Calibration weights
Collected with BF16 weights: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16
at revision e0ac9ee3dfd02be21b5479edfc2f671ed269d0a2. The basis and the allocation scores depend on the weights,
so use this file with these weights; for another precision or checkpoint,
calibrate with that checkpoint.
Contents
| Field | Value |
|---|---|
| Base model | Nemotron 3 Nano 30B-A3B Base (Mamba-2) |
| Recurrent layers | 23 (model layers 0, 2, 4, 7, 9, 11, 14, 16, 18, 21, 23, 25, 28, 30, 32, 35, 37, 39, 41, 44, 46, 48, 50) |
| State heads per layer | 64 |
| Key dim K / value dim V | 128 / 64 |
| Basis groups per layer | 8 |
| Window W | 16 |
| Erase factor | no |
| Allocation rank cap | 42 |
Basis omega |
float32, shape (23, 8, 63, 128) |
| File size | 6,692,293 bytes |
The file loads with torch.load(..., weights_only=True). The
calibration guide
documents its keys.
Verified mean ranks
For these mean ranks, the derived tables equal the bundle tables and the exported frames equal those of the bundle export path:
| Mean rank | Dense heads | Sketch heads |
|---|---|---|
| 2 | 0 | 1472 |
| 4 | 3 | 1469 |
| 8 | 26 | 1446 |
| 16 | 273 | 1199 |
Other mean ranks are allocated with the same rule. manifest.json lists the
SHA-256 of calibration.pt and these results.
Usage
With vLLM's SketchSSM support (--sketchssm), the ranks for the requested mean
rank are allocated when the model loads:
vllm serve nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 \
--sketchssm SketchSSM/Nemotron-3-Nano-30B-A3B-Base-BF16 --sketchssm-mean-rank 8 \
--mamba-ssm-cache-dtype float32
With the SketchSSM package, export the frames for a mean rank:
hf download SketchSSM/Nemotron-3-Nano-30B-A3B-Base-BF16 calibration.pt --local-dir calibration
python -m sketchssm.calibration export --calibration calibration/calibration.pt \
--mean-rank 8 --out frames.pt
How it was produced
With the public calibration pipeline of the SketchSSM repository (release
0.1.15) and collect.yaml from this repository:
python -m sketchssm.calibration calibrate --config collect.yaml --out out
python -m sketchssm.calibration package --bundle out --out calibration.pt
python -m sketchssm.calibration manifest --bundle out --calibration calibration.pt --out manifest.json
package re-allocates every configured mean rank from the packaged contents
and fails unless each table equals the bundle table.
License
This calibration file is released under the Apache License 2.0, like the SketchSSM repository. It is derived from the base model's weights, so use it under the base model's license as well.