Nemotron 3.5 Bolt 74B-A9B, XE2MX4 GGUF for Intel Arc Pro B65
NVIDIA Nemotron 3.5 Bolt 74B-A9B (Mamba2 + attention + MoE) packed for Intel Xe2 dGPUs. These files only load on the SYCL build of llama.cpp-exl3xe2cuda. Stock llama.cpp, CUDA and Vulkan do not load XE2MX4. Tested on Arc Pro B65 (32 GB) only. Experimental.
XE2MX4 is 32 weights per block, 4-bit integer nibbles and one fp16 scale, 4.5 bits per weight. Not MXFP4.
Files
| File | Size | Use |
|---|---|---|
| Nemotron-3.5-Bolt-74B-A9B-hybrid-IQ2_XXS-XE2MX4.gguf | 21.0 GiB | one B65 |
| Nemotron-3.5-Bolt-74B-A9B-XE2MX4.gguf | 39.8 GiB | two B65s, layer split |
| Nemotron-3.5-Bolt-74B-A9B-LiLiCorr-XE2MX4.gguf | 1.7 GiB | optional drafter, both setups |
- hybrid: NVIDIA's 24GB GGUF with the shared experts (IQ2_XXS) and Mamba in/out projections (NVFP4) requantized to XE2MX4. The routed experts stay IQ2_XXS, byte for byte, and so does every other tensor.
- XE2MX4: converted from NVIDIA's BF16 checkpoint. Mamba in/out and all experts XE2MX4, latent and attention projections Q8_0. No imatrix.
- drafter: NVIDIA's LiLiCorr head with its lattice, candidate selector and transformer head kept, plus the target embedding and output tables so it can sit on one card next to a split target. Large matrices XE2MX4, small heads unchanged.
All three can be rebuilt with scripts/bolt/ in the repo; the rebuilt files match the SHA256SUMS
here.
Run
BOLT_MODEL=Nemotron-3.5-Bolt-74B-A9B-hybrid-IQ2_XXS-XE2MX4.gguf scripts/bolt-b65.sh one
BOLT_MODEL=Nemotron-3.5-Bolt-74B-A9B-hybrid-IQ2_XXS-XE2MX4.gguf \
BOLT_DRAFT=Nemotron-3.5-Bolt-74B-A9B-LiLiCorr-XE2MX4.gguf scripts/bolt-b65.sh one-draft
BOLT_MODEL=Nemotron-3.5-Bolt-74B-A9B-XE2MX4.gguf scripts/bolt-b65.sh dual
BOLT_MODEL=Nemotron-3.5-Bolt-74B-A9B-XE2MX4.gguf \
BOLT_DRAFT=Nemotron-3.5-Bolt-74B-A9B-LiLiCorr-XE2MX4.gguf scripts/bolt-b65.sh dual-draft
Needs oneAPI 2026.1 and the patched oneDNN from the repo for full prefill speed. Build steps,
flags and limits are in docs/nemotron-bolt.md.
Numbers
llama-server HTTP, 262144 context, q8_0 KV, reasoning on, short retrieval answer. One fresh server per cell, warm request shown. PP / TG in tokens per second. These measurements are from the lab build, not a rerun of the curated release binary.
Separate fast configuration: two B65s, 16384 context, LiLiCorr and overlapping prefill reached approximately 2457 PP / 85 TG in the lab. That is not the 262144-context configuration below.
One B65, hybrid file:
| Prompt tokens | Target | + LiLiCorr |
|---|---|---|
| 7937 | 1970 / 35.6 | 1390 / 61.9 |
| 16116 | 2042 / 34.3 | 1489 / 52.2 |
| 32481 | 2011 / 32.1 | 1504 / 47.4 |
| 65266 | 1864 / 28.4 | 1431 / 48.2 |
| 130823 | 1582 / 23.1 | 1264 / 39.2 |
| 239722 | 1252 / 17.8 | 1041 / 37.7 |
Two B65s, XE2MX4 file:
| Prompt tokens | Target | + LiLiCorr |
|---|---|---|
| 7937 | 2344 / 41.5 | 2086 / 88.0 |
| 16116 | 2442 / 39.8 | 2170 / 82.6 |
| 32481 | 2355 / 36.9 | 2079 / 76.6 |
| 65266 | 2093 / 32.2 | 1866 / 81.4 |
| 130823 | 1662 / 25.6 | 1516 / 62.2 |
| 239722 | 1228 / 19.1 | 1145 / 49.8 |
Test box: the B65 is a PCIe 5.0 x16 card, but here one runs in a chipset slot at PCIe 4.0 x4 and the other in a Thunderbolt 3 dock at PCIe 3.0 x4. The dock card is 5 to 8 percent slower on prefill than the chipset one, and two card numbers are capped by the slower card, so B65s on wider or faster links will likely do better than these tables. Not measured here.
Quality
WikiText-2 test, first 100 chunks, ctx 512, q8_0 KV:
| File | PPL |
|---|---|
| NVIDIA 24GB GGUF (reference) | 6.3988 |
| hybrid, one B65 | 6.6651 |
| XE2MX4 from BF16, two B65s | 5.7484 |
The hybrid is 4.2 percent higher PPL than NVIDIA's file on this slice. That file was already quantized (IQ2_XXS shared experts, NVFP4 Mamba projections), and the hybrid quantizes those 160 tensors a second time into XE2MX4 so the B65 kernels can run them; two rounds of rounding cost more than one. The routed experts are not touched. The two card file is 10.2 percent lower because it is quantized once, straight from BF16, instead of starting from 2-bit experts. One slice, not a general accuracy claim. Outputs are not bit reproducible between launches.
License
Derived from NVIDIA Nemotron 3.5 Bolt, released by NVIDIA under the OpenMDW 1.1 license (https://openmdw.ai/license/1-1/). These files keep that license. Not affiliated with NVIDIA or Intel.