GLM-5.3-Flash · exlr8 (K2/K3/K4)

A quantization of GLM-5.3-Flash in the exlr8 v2 format, built for NVIDIA GB10-class unified-memory devices (DGX Spark, ASUS GX10) served on one to four boxes. Every routed and shared expert is stored at three trellis widths, K2, K3 and K4, and the serving width is chosen at load time.

These weights cannot be used without E4M3 rounding. A loader must round every value the trellis codebook produces to FP8 E4M3 (round to nearest even, saturating) before applying the side vectors. The encoder searched only over E4M3-rounded values, so the rounding is part of the format: a stock EXL3 decoder, or any decoder that skips the rounding, reads measurably worse weights from the same bytes.

GLM-5.3-Flash is a MoE model: 45 layers (0-2 dense, 3-44 MoE), 288 routed experts with top-8 routing plus one shared expert per MoE layer, an MTP layer 45 and a vision tower. This repository quantizes the routed and shared experts of layers 3-45 and the dense weights of every layer, and carries the vision tower unchanged in BF16.

Engine support

• kindlingai/glm-5.3-flash-exlr8-gx10: KindlingAI's vLLM recipe for GB10 at TP=1, 2, 3 and 4. vLLM with an exlr8 MoE kernel, DFlash2 speculative decoding, the mentat control plane, and image input at every TP. The numbers below are this recipe's.
• kindlingai/gr8: KindlingAI's exlr8-native engine for DGX Spark, which serves this release with ./gr8 glm53f at TP=4 or TP=2 and fetches each box's part from this repository. Flash support is new: one session at a time, MTP with one draft, text only so far. See docs/FLASH.md.

The format, and the byte-level description a loader needs, is shared with kindlingai/glm-5.3-exlr8-k2-k3-k4. See Loading for what differs on Flash.

Widths

• Model size is everything one copy loads at that width: the experts of layers 3-45 plus the 8.1 GB of dense, vocabulary and vision files.
• Widths are exact: a K4 layer file is 3,636,527,104 bytes, 289 experts × 3 projections × 4096 × 2048 at 4 bits. NVFP4's experts are 4.25 bpw, so K4 reads about 6% fewer expert bytes per token than the NVFP4 build.
• Each width is an independent, Hessian-aware trellis encode. Codes do not nest between widths, and all three use an E4M3 codebook, so prefill runs on FP8 tensor cores.

Results

Measured with the vLLM recipe (kindlingai/glm-5.3-flash-exlr8-gx10) on GB10 boxes, clocks locked at 2200 MHz, 2026-10-19. Decode is the mean of 16 runs per workload with DFlash2 speculative decoding and adaptive draft length. Tokens per second:

KV cache in the vLLM recipe, with the vision tower loaded. On DGX OS each pin leaves the head about 4.5 GiB of free memory at its lowest point, under long prefills and four concurrent long requests. kindling-spark-os returns 2 GiB more for KV at TP=2 and up:

Against the NVFP4 build of GLM-5.3-Flash on the same four boxes, K4 at TP=4, both in vLLM:

• Decode is 3-8% faster (code +7.7%, prose +3.0%).
• Quality is equal: NLL is +0.0034, +0.0017 and +0.0020 above BF16 on three held-out sets, the same as NVFP4's or slightly lower, and GSM8K scores the same (98%).
• The KV pool is about 15% larger at the same memory, since the expert weights are smaller.
• Prefill is 7.7% slower at 32K and 128K. The gap is in the routed MoE kernel at large batch sizes.

The quality comparison covers K4. K2 and K3 trade quality for size and have not been scored against BF16 here; measure them on your own workload before relying on them.

What is in this repository

Dense matrices by type:

Loading

The files follow the exlr8 v2 layout in the GLM-5.3 card's Loader section, with Flash's shapes in place of GLM-5.3's:

• Hidden size 4096 (GLM-5.3: 6144) and expert intermediate size 2048, so each projection still has 16 halves of 128 channels. A gate or up half is 4096 × 128 (tiles k16 = 0..255), a down half 128 × 4096 (tile blocks b = 0..31), and each projection carries 32 block scales instead of 48. A half at width K is 65,536 × K bytes.
• 288 routed experts. The shared expert is stored as expert 288 and follows the routed experts.
• Tensor names use the prefix model.language_model.layers..
• The sharding rule is the same: half h of expert e goes to rank (16e + h + L) mod N at TP=N.

Next to the GLM-5.3 release:

• k2, k3 and k4 load every routed and shared expert at that width. bit201 is K2 with the 124 routed experts that carry the most routing weight at tool-call positions at K3, about 2.01 bits a routed weight. bit349 is K3 with every layer's down projection at K4, gate and up at K4 in layers 3-12 and the shared expert at K4, about 3.49 bits a routed weight. Mean KL against BF16 on 56 held-out rows: 0.397 (k3 0.472, k4 0.384).
• The shared expert and the MTP layer have all three widths, so every manifest covers every layer.

Index slices

index-slices/tp<N>/k<K>/rank<r>.safetensors holds, for TP size N (1-4), width K (2-4) and rank r, every half that rank owns under the sharding rule, so a rank can fetch and load its halves without the layer index files. The layout is the GLM-5.3 one (Index slices) with Flash's shapes: per layer, block is F16 [289, 3, 32] (the shared expert as row 288) and signs is I8 [2, 4096]. The metadata also carries the model keys (model, hidden_size, moe_intermediate_size, n_routed_experts, num_hidden_layers, first_moe_layer, tensor_prefix).

Identifying the weights

The weight set ID names one exact set of weight files: the sha256 of the lines <path> <size> <sha256>, one per weight file (experts/*.safetensors and dense/*.safetensors, sorted by path, each line ending in a newline). This release's ID is f6e6c6549659a0416d5a65fc97c45d093fd26786167294b561e156de83051939. Every manifest and index slice carries it as set_id, and every slice also names the sha256 of each index file it was cut from.

Provenance

Built from zai-org/GLM-5.3-Flash-BF16 at revision a5b45eb41df6402735dedc900be14a42e8d5e538. The configuration, tokenizer, chat template and processor files are that revision's, byte for byte.

The vision tower in dense/vision-bf16.safetensors holds the 347 model.visual.* tensors of the same checkpoint (file model-00120-of-00120.safetensors), copied byte for byte under their original names. Each tensor's SHA-256 also matches the vision tower of the NVFP4 build, which leaves model.visual* unquantized.

Each dense file passed an integrity check and a comparison against BF16 before publication.

Calibration

Encoded with Hessian-aware (LDLQ) rounding against a fixed calibration set of about 2M tokens (2048-token rows; 977 calibration rows and 64 held-out rows from whole unseen documents). GLM-5.3-Flash uses GLM-5.3's tokenizer and chat template, so the set is the same one used for the GLM-5.3 release. The calibration text, activations and Hessians are not part of this repository.

The format's three exllamav3-derived primitives (the mcg trellis codebook, the 128×128 Hadamard matrix and the tile packing order) are taken from exllamav3 (MIT) unchanged.

Intended use and limitations

• Serving GLM-5.3-Flash on GB10-class devices at TP=1-4.
• The DFlash2 drafter is not distributed with this checkpoint (its upstream licence does not permit redistribution). A deployment that wants it converts it at load time from the original files.
• On a turn with an image, DFlash2 drafts from the text alone, so draft acceptance can be lower than on text-only turns.
• Not for training. The router, norms and small projections keep their shipped precision because routing is sensitive.

Licence

GLM-5.3-Flash is released by Z.ai under the MIT License, and this quantization stays under it: see LICENSE.

Acknowledgements

exllamav3 made this possible through their original research on the EXL3 format.

Built on Z.ai's GLM-5.3-Flash. The encoder and the format's primitives are derived from exllamav3 0.0.43 (MIT).

Changelog

2026-11-07

• Adds manifests/exlr8-manifest-bit349.json: K3 with K4 down projections, K4 gate and up in layers 3-12 and a K4 shared expert (about 3.49 bits a routed weight).
• Adds manifests/exlr8-manifest-bit201.json: K2 with 124 routed experts at K3 (about 2.01 bits a routed weight), the ones with the most routing weight at tool-call positions. The weight files are unchanged.

2026-10-24

• The card links gr8, which now serves this release (./gr8 glm53f), and labels the speed table as the vLLM recipe's. It adds the recipe's KV cache sizes per TP and the NLL figures against BF16.

2026-10-23

• Adds uniform k2, k3 and k4 width manifests, index slices for TP 1-4 at K2, K3 and K4 with their hashes, and the weight set ID. The weight files are unchanged.

2026-10-22

• Rewrote this card: widths and sizes, measured speed and quality, file and encoding tables, loader differences from GLM-5.3.
• Corrected the base model to zai-org/GLM-5.3-Flash-BF16 at a5b45eb. The previous card named a revision of zai-org/GLM-5.3-Flash. The files are unchanged.

2026-10-19

• Added the BF16 vision tower and processor_config.json.

2026-10-14

Initial release: routed and shared experts of layers 3-45 at K2, K3 and K4; dense and vocabulary files.