Runtime: Osaurus 0.25.23 or newer.

OsaurusAI/Nemotron-3.5-Bolt-74B-A9B-24GB-JANGH2

Nemotron 3.5 Bolt 74B-A9B at NVIDIA's own 24 GB size — and closer to the full-precision model than NVIDIA's 24 GB release on every distribution metric we measured. Vision included; NVIDIA's LiLiCorr drafter in the box.

A JANG bundle of nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16: a 74B-parameter (9B active) hybrid of 40 Mamba-2 layers, 40 latent-MoE layers (240–496 routed experts per layer, top-9, 1024-wide latent, relu2) and 8 attention layers, with a C-RADIO v4-H vision encoder.

• Routed experts: JANGH — our Hadamard-32 codebook format at 2 or 3 bits per weight (rounded with GPTQ), the width of every projection chosen per layer from measured sensitivity: 2.15 bits per weight on average.
• Mamba-2 in/out projections and shared experts: affine 4-bit (group 64), GPTQ.
• Attention, latent projections, lm_head, embeddings: affine 8-bit (group 64), GPTQ (embeddings rounded).
• Vision tower and projector: affine 8-bit (group 64). Routers stay bf16 (computed in fp32), expert-score bias fp32.
• LiLiCorr drafter (lilicorr/): NVIDIA's speculative-decoding draft model, its weights carried exactly.
• Size: 24.10 GB of model shards (24.12 GB with configs) — the same footprint as NVIDIA's 24 GB release — plus 0.96 GB for the drafter. 2.58 bits per weight over the 74.9B source parameters.

Quality vs the original model

369 held-out sequences, 290,054 teacher-forced positions, scored against the BF16 release run layer by layer from disk. Top-128 renormalized KL. Every assistant turn in them was written by Bolt itself (T=1.0, top-p 0.95) across its three reasoning modes, 22 human languages, ~35 coding languages, terminal, math, security, multi-turn agent episodes with tool calls, and photos / documents / charts.

The BF16 release is the model NVIDIA distilled from when it built its 24 GB release with quantization-aware distillation (the 24 GB weights and even its embeddings and routers differ from BF16), so both 24 GB models are scored against the same BF16 reference.

• Change in negative log-likelihood of the actual held-out tokens: +0.095 for this bundle, +0.089 for NVIDIA's 24 GB release. NVIDIA's distilled model predicts the real next token marginally better; this bundle stays much closer to the original model's full distribution.
• The image rows of this table use the BF16 tower's embeddings for every row. Scored through this bundle's own 8-bit tower, the image domain is 83.7% top-1 / KL mean 0.186 (84.3% / 0.171 with BF16-tower embeddings).

By reasoning mode, top-1 agreement:

By domain, median KL · mean KL · top-1:

Tool decisions and reasoning

• At the 99 points where the original model's own transcript starts a tool call: 99 of 99 start the call (same as NVIDIA's 24 GB), no call ↔ answer flips.
• At the 215 points where the original model closes its reasoning (</think>): median P(</think>) 0.957 (original 0.963, NVIDIA 24 GB 0.952); P < 0.1 at 10 points (NVIDIA 24 GB: 14).

What's in the bundle

• Experts (40 latent-MoE layers; fc1 = the 1024 → I up projection, fc2 = the I → 1024 down projection; relu2, no gate):

  projection	JANGH 2	JANGH 3
  fc1	27 layers	13
  fc2	37	3

  Per-layer formats (measured from the shards)

  block	experts	I	fc1	fc2	GB
  1	240	2048	JANGH2	JANGH3	0.316
  3	240	1536	JANGH3	JANGH3	0.284
  5	240	2048	JANGH3	JANGH2	0.316
  8	304	2304	JANGH3	JANGH2	0.450
  10	240	2304	JANGH3	JANGH2	0.355
  12	240	2048	JANGH3	JANGH3	0.379
  14	240	2048	JANGH3	JANGH2	0.316
  17	304	2304	JANGH3	JANGH2	0.450
  19	304	2048	JANGH3	JANGH2	0.400
  21	432	2560	JANGH2	JANGH2	0.569
  23	432	2304	JANGH2	JANGH2	0.512
  26	240	2048	JANGH3	JANGH2	0.316
  28	432	2304	JANGH2	JANGH2	0.512
  30	304	2560	JANGH2	JANGH2	0.401
  32	304	2304	JANGH3	JANGH2	0.450
  34	432	2304	JANGH2	JANGH2	0.512
  37	368	2304	JANGH2	JANGH2	0.437
  39	368	2048	JANGH3	JANGH2	0.485
  41	368	2304	JANGH2	JANGH2	0.437
  43	432	2560	JANGH2	JANGH2	0.569
  45	368	2560	JANGH2	JANGH2	0.485
  48	496	2560	JANGH2	JANGH2	0.654
  50	432	2560	JANGH2	JANGH2	0.569
  52	240	2304	JANGH2	JANGH2	0.285
  54	368	2304	JANGH2	JANGH2	0.437
  56	368	2560	JANGH2	JANGH2	0.485
  59	304	2048	JANGH2	JANGH2	0.321
  61	240	2304	JANGH3	JANGH2	0.355
  63	240	2304	JANGH2	JANGH2	0.285
  65	240	2048	JANGH2	JANGH2	0.253
  67	240	2048	JANGH2	JANGH2	0.253
  70	368	2048	JANGH2	JANGH2	0.388
  72	368	2304	JANGH2	JANGH2	0.437
  74	432	2304	JANGH2	JANGH2	0.512
  76	432	2560	JANGH2	JANGH2	0.569
  79	496	2560	JANGH2	JANGH2	0.654
  81	432	2560	JANGH2	JANGH2	0.569
  83	432	2560	JANGH2	JANGH2	0.569
  85	496	2560	JANGH2	JANGH2	0.654
  87	368	2560	JANGH3	JANGH2	0.606

• Non-experts (measured from the shards): Mamba-2 in_proj / out_proj 4-bit ×40 each, shared-expert up / down 4-bit ×40 each, latent fc1 / fc2 8-bit ×40 each, attention q/k/v/o 8-bit ×8 each, lm_head and embeddings 8-bit. Only the 40 routers are stored unquantized (bf16). Vision: 132 linears 8-bit; position embedding, class tokens and norms bf16.

• LiLiCorr drafter (lilicorr/, 0.96 GB, OpenMDW-1.1): NVIDIA's block drafter for Bolt (5 dense layers with sliding-window attention and sinks, plus a 2-layer head that re-ranks its top-8 candidates per slot; block 8, mask token 990, reads target layers 1 / 18 / 35 / 51 / 68 / 85, shares the target's embeddings and lm_head). NVIDIA's NVFP4 values are stored exactly in MLX nvfp4 format plus each tensor's global scale (weight_scale_2, applied to the output); everything else bf16 as shipped. It was trained against NVIDIA's 24 GB target; its acceptance on this bundle has not been measured yet.

• No MTP (the source has none).

Serving contract

• Reasoning modes (chat-template kwargs): enable_thinking: true (default), enable_thinking: true + medium_effort: true (efficient reasoning), enable_thinking: false. There are no other effort levels.
• Reasoning: the chat template opens <think>; the model closes it with </think> before the answer. Tool calls use the qwen3_coder XML format (<tool_call><function=…><parameter=…>), as in NVIDIA's own serving recipe.
• Sampling (both generation_config.json and jang_config.json): temperature 1.0, top-p 0.95. EOS: </s>, <|im_end|>.
• Images: any size (dynamic resolution, 1,024–13,312 patches); each image becomes ⌈patches / 4⌉ tokens.

Build details

• Calibration: 1.3M tokens of Bolt's own outputs (1,784 native turns, 640k generated tokens) plus prompts, over on / medium / off reasoning (45 / 30 / 25%), coding (~35 languages), agent tool rounds, terminal, math, 22 human languages, security, chat, prose, and 152 images (photos, documents, charts) embedded by the BF16 vision tower. Held-out rows (369 sequences) are disjoint from calibration.
• Per-expert Hessians (1024-wide latent fc1 inputs, relu2 fc2 inputs) from a layer-streamed run of the BF16 model; per-layer sensitivity from frozen-routing KL; per-unit error curves for every width; a byte-exact allocator chose the widths at the 24.1 GB budget. Every JANGH row carries the weight-space unit gain.
• Non-experts: GPTQ with the input Hessian of each linear; channels that never fire in calibration (common with relu2 inputs) keep their weights instead of being zeroed.
• Bundle verified from the shards: every tensor file aligned, one index, every module's bits measured from bytes.