NVIDIA Nemotron 3.5 Bolt LiLiCorr EXL3 6 bpw
EXL3 quantization of nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr.
This is a double quant. No BF16 sibling of the drafter was on the Hub. Trunk linears were dequantized from that repo's W4A16 NVFP4 and re-encoded to EXL3 6 bpw, mul1 codebook. Hessians were identity, not a prefix capture. Attention, convolutions, norms, and LiLiCorr heads were copied at their stored dtype. Those choices are recorded in quant.json.
The intended target is the Bolt EXL3 language pack. NVIDIA's vLLM recipe uses 7 speculative tokens. This pack was not loaded for any measurement here.
Config the checkpoint claims
| Architecture | LiLiCorrDraftModel |
| model_type | qwen3 |
| block_size | 8 |
| target_layer_ids | 1, 18, 35, 51, 68, 85 |
| hidden_size | 4096 |
| layers | 5, all sliding attention |
| sliding window | 1024 |
| heads / kv heads / head_dim | 32 / 2 / 128 |
| intermediate_size | 12288 |
| vocab | 131072 |
| max_position_embeddings | 262144 |
| LiLiCorr candidate top-k | 8 |
| LiLiCorr hidden / layers / heads | 1024 / 2 / 8 |
| LiLiCorr mlp ratio / factor dim | 2.0 / 1024 |
| LiLiCorr logit scale | 8.0 |
| conv kernel / group | 2 / 16 |
| License | OpenMDW-1.1 |
| Size | 1.1G |
Not measured
exllamav3 commit 21ce2d1a82cab53c9e1301aef92f1239459278b8 has no LiLiCorrDraftModel. Config.from_directory fails at load. Acceptance length, draft speed, and Q200 with this drafter were not measured. Do not read a speed off the NVFP4 card and treat it as this pack's speed.
The language-pack card is r0b0tlab/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-EXL3-2.08bpw.