GLM-5.3 Vision NestQuant 1.75-4 bit

GLM-5.3 with its routed experts quantized to NestQuant, a nested two-level format. Each expert has a 1.75-bit base plus a residual plane that lifts it to 4 bits. At serving time an expert moves from 1.75 bit to 4 bit by loading its residual plane on top of the base, and the base bytes stay the same.

This build targets 2x NVIDIA DGX Spark with an fp8 KV cache (fp8_ds_mla) at 128K context.

Expected quality at 2x Spark memory, with the presets in spark-b175:

The 4-bit level is the same as the 2-4 bit build: 4.14 bpw here vs 4.13 there.

Upload status: complete (all four ranks, rank*.json and artifact_stamp.json).

Contents (serve-ready layout)

This repo is self-contained: everything needed to boot is here. The expert repack is in the exact layout the NestQuant streaming server reads, alongside the non-expert base checkpoint and the jF predictor. start.sh downloads them as-is, and no repacking step is needed. Per tensor-parallel-4 rank r (0-3):

Serving

The serving kernel must support nq-res-v2: base K code 1 (K = 1.75) and residual code 9 (down, K = 2.5625). NQ_RES_V2.md gives the exact bit layout. A server build that only knows nq-res-v1 (the 2-4 bit build) cannot decode this repo.

4x RTX PRO 6000 (TP4)

With branch spark-b175 of the nestquant repo:

git clone -b spark-b175 https://github.com/jarrelscy/nestquant && cd nestquant
NQ_REPACK_DIR=/path/to/repack-1.75 ./start.sh up spark     # 128K, capped at 2x DGX Spark memory (64 GB/GPU)

start.sh downloads base/ and serving/predictor/ from this repo if they are absent. If rank*.json is missing, it runs hf download $NQ_REPACK_REPO --include 'rank*.json' 'rank*.bin' 'res/*' 'artifact_stamp.json' (~400 GB, once). sm120/serve/tools/verify_repack.py $NQ_REPACK_DIR then checks that every layer is present on every rank. With hf 1.x, pass one --include per pattern if you run the download by hand (a list after one --include only applies the first pattern).

Measured: 4x RTX capped at 2x DGX Spark memory

Setup (2026-10-04):

• Memory: nvidia-smi peak <= 64,000 MiB per GPU, prefill included. That is the same 256 GB as two Sparks, split over 4 GPUs.
• TP4 + DCP4, fp8 KV (fp8_ds_mla), 128K context, MTP with 3 draft tokens.
• Experts: 21 slots per layer, 18 of them floating, chosen by the jF predictor with tap.
• Same stack as the 2-4 bit production server: prefill borrow, coalesced I/O, step 2.

Results:

• Peak 63,934 MiB per GPU.
• Needle found at 129K.
• Decode, 1 request: 76.5 tok/s at 0 context, 71.7 tok/s at 16K.
• Prefill 1,345 tok/s.

KLD is measured on the live server: teacher-forced decode, full vocabulary, BF16 teacher, held-out windows 0-3.

At 45 floating, the live server reproduces the offline 45-hot result (0.0432), so the serve path is exact. Under the cap only 18 fit. Part of the gap to 45 comes from running on 4 GPUs instead of 2:

• Per-GPU replicated memory is paid 4 times instead of twice: activations, CUDA context/NCCL and cudagraphs (~4.6 GiB per GPU) plus ~1.4 GiB of replicated non-expert weights.
• The other non-expert weights are sharded and cost the same either way.

That is about 12 GiB, or ~15 slots per layer. A Spark node also has about 114 GiB usable rather than the 125 GiB emulated here, which gives the 18 (MTP on) and 29 (MTP off) floating in the Spark presets below.

Layer-major prefill

Branch spark-b175 has layer-major prefill on by default (NQ_LMPF=1, same as the 2-4 bit build). Prompts with 1K+ new tokens get more experts at 4 bit during prefill. On the 2-4 bit build it cut 2K prefill KLD from 0.047 to 0.023.

Its read ring and its 16K-token window live in idle decode expert slots during the prefill (NQ_LMPF_BORROW=all), so the presets above keep their full slot counts. Measured 2026-10-04 at the 64 GB cap (start.sh spark, 21 slots, NQ_UTIL 0.630):

• nvidia-smi peak 63,902 MiB per GPU.
• Decode KLD before any prefill: 0.0660 (.0491 / .1381 / .0416 / .0350). After a 64K layer-major prefill before each window: 0.0644 (.0498 / .1284 / .0406 / .0387).
• 64K prefill 1,211-1,277 tok/s. Each borrowed 643 slots in ~13 ms.
• Needle found at 129K (101 s). The 716 borrowed slots refilled in 0.14 s.

NQ_LMPF=0 turns it off. Not yet run on Spark hardware.

2x DGX Spark

Branch spark-b175 of the nestquant repo serves this repo on two DGX Spark nodes as one TP2 vLLM instance over the ConnectX-7 link. Instructions are in spark/README.md.

Status: untested on Spark hardware. The kernels, the TP4 → TP2 merge and the cross-node op log were checked on other GPUs; the GB10 runtime, speed and live KLD still need a first real run.

git clone -b spark-b175 https://github.com/jarrelscy/nestquant && cd nestquant   # on each Spark
docker build -f spark/Dockerfile -t nestquant-spark:b175 spark/      # on each Spark, ~1-2 h
HEAD_IP=192.168.100.10 spark/start_spark.sh worker                    # node 1
HEAD_IP=192.168.100.10 spark/start_spark.sh head                      # node 0, OpenAI API on :8001

Each node downloads only its two TP4 ranks (~196 GB) plus the non-expert weights, and merges them into one TP2 rank locally (streaming/tp4to2.py). The BF16 attention and shared-expert linears are converted to fp8 at load, o_proj runs in NVFP4, and the MTP layer's experts are converted to fp8 once at download.

The expert counts come from the per-node budget in spark/README.md (114 GiB usable, measured resident planes). At 64K context each node has 3.35 GiB more, about 8 more experts per layer. Speed and prefill (~800-1,400 tok/s up to ~16K tokens, ~500-900 tok/s at 128K) are estimates from bytes and compute per token. A Spark reads memory at ~273 GB/s against ~1.8 TB/s on an RTX PRO 6000, so the 4x RTX numbers above don't carry over. The KLD estimates interpolate linearly between the two live measurements (18 and 45 floating). None of these are measured on GB10.

Method

The method is the same as the 2-4 bit build except for the base and residual rates:

• Rotation: random signs + Hadamard-128 on both sides of each weight matrix.
  • Layers 3-6, down projection: the input side uses random signs + one Hadamard-512 per TP4 shard, with a flat input-channel scale (in_had_down = 512 in those layers' res files).
• 1.75-bit base: a pattern-rate bitshift trellis code. Step p of each 16-step window uses 1 + ((0xEEEE >> (p % 16)) & 1) bits, so 28 bits per 16 weights. It has a per-tile sign and is fitted with LDLQ.
  • Base 1.776 bpw with scales.
• 4-bit residual: a second trellis code on the rotated residual, fitted jointly with the base. It is 2.25 bits per weight on gate/up and 2.5625 on down.
  • 4.141 bpw total.
• Low-rank correction: experts whose input has a few very large activation channels get a rank 1-4 fp16 correction.
• Calibration: the same data as the 2-4 bit build (text plus images). Per-expert Hessians blend 75% text and 25% vision.

Quality

Full model, 45 hot experts per layer (all 75 real encoded layers, fp8 KV emulated, 45 hot experts per layer with the jF predictor). KLD is measured against the BF16 model's full logits over four held-out windows:

At 45 hot experts, about 48% of routed expert calls and 67% of routing salience run at 4 bit.

Design study (earlier sizing). KL divergence against the FP8 model (four held-out evaluation windows). It uses an earlier, tighter sizing: bf16 KV cache at 128K, which leaves 175 GiB for experts.

• These numbers come from an emulation of the 1.75-bit base error.
• With 15 real encoded layers (7-10, 40-50) swapped in, the KLD was 0.0446, which matches the emulation (ratio 0.993).

Verification

• Every layer's records and resident planes are produced by streaming/repack.py from the encoder output.
• Sampled experts across layers and ranks were checked bit for bit: the kernel-plane decode equals the encoder's decoder at both levels, and the res + record bytes round-trip to the same weights. The bytes in this repo equal those checked bytes.