Mia's AI Lab's own EXL3 quantization of zai-org/GLM-5.3 (the full model, 753B parameters), sized for three DGX Sparks.
It runs on three DGX Sparks with TensorFold: GLM-5.3 EXL3 on 3x DGX Sparks with TensorFold.
Measured with the recipe's defaults (RoCE all-gathers, the RedHatAI DSpark speculator for drafts, FP8 KV cache, one request at a time), each figure with the boot it came from:
| Measured | Boot | |
|---|---|---|
| Decode, code | 39.5-40.4 tok/s | p17, 163,840-token window, sampled, 3 runs |
| Decode, prose | 29.1-30.2 tok/s | p17, same |
| Prefill | 655 / 656 / 638 tok/s at 8k / 16k / 32k tokens | p10 |
| Exactness | drafted and concurrent replies equal serial ones, 12/12 | every boot |
Speculative decoding runs faster on predictable text (lists, tables, worked arithmetic), and slower on free prose.
Long-context mode (experimental): context parallelism (each Spark keeps every third token's cache), a 4-bit KV cache and captured decode graphs give a 655,360-token window at the same decode speed: code 39.9-40.1, prose 29.8-30.4 tok/s (boot fp4g655, 3 runs each), a 16-key recall test correct at 39k, 157k and 314k tokens, prefill ~220-245 tok/s. It is experimental until the recipe trims its prompt-time memory peak.
KL divergence measures how far a quant's next-token probabilities drift from the original model's (lower is better). At 2.75 bits a weight on average, the routed experts stay closest where precision matters most: math 0.012 and code 0.035 with only the experts quantized, 0.024 and 0.061 as TensorFold actually serves it with its own 4-bit dense weights on top.
The quant's most likely next token is the original model's 96.7% of the time on math and 95.4% on code (92.8% over all sets; 90.8% as served).
| Test set | KL divergence, experts only | Top-1, experts only | KL divergence, as served | Top-1, as served |
|---|---|---|---|---|
| Math (thinking) | 0.012 | 96.7% | 0.024 | 95.3% |
| Code (thinking) | 0.035 | 95.4% | 0.061 | 92.8% |
| Chat | 0.043 | 93.5% | 0.069 | 91.1% |
| Wiki | 0.126 | 88.8% | 0.164 | 87.1% |
| Tool calls | 0.201 | 90.7% | 0.289 | 88.7% |
| All sets | 0.086 | 92.8% | 0.124 | 90.8% |
zai-org/GLM-5.3-BF16, rev 9d2398f4), full vocabulary, temperature 1; reference
log-probabilities stored at fp16, KL computed in float32.| Part | Format |
|---|---|
| Routed experts, layers 3-77 | EXL3, mixed widths per expert (2, 3 or 4 bits a weight per projection), averaging exactly 2.75 bits a weight in every layer; mcg codebook, output scales |
| MTP layer 78 experts | EXL3, 4 bits a weight |
| Everything else (attention with the DSA indexer, shared experts, dense layers 0-2, routers, embeddings, head) | BF16, unchanged from zai-org/GLM-5.3-BF16 |
| Chat template | Z.ai's chat_template.jinja from the release |
| Size | 273 GiB, 137 weight files |
Each expert's widths are listed in quantization_config.json (expert_bits).
The average width, 2.75 bits a weight, is bits in quantization_config.json (with every expert's widths); config.json names the format (quant_method: exl3) without it, so Hugging Face's config check stays quiet.
Source: the BF16 release (zai-org/GLM-5.3-BF16, rev 9d2398f4), calibrated for how TensorFold serves it.
With TensorFold quantizing the BF16 dense layers to 4 bits at load and the layers split across three Sparks, the weights take about 84 GiB per Spark. Measured on the first boots (FP8 KV cache, 64k-token window, one request at a time): the lowest free memory was 16.7 / 20.1 / 13.3 GiB on the three Sparks. The recipe's default window is 163,840 tokens with an FP8 KV cache; its memory plan checks every Spark's free memory before it starts.
LICENSE).model-00090-of-00137.safetensors 2.1 GBmodel-00106-of-00137.safetensors 2.1 GBmodel-00006-of-00137.safetensors 2.1 GBmodel-00125-of-00137.safetensors 2.1 GBmodel-00078-of-00137.safetensors 2.1 GBmodel-00083-of-00137.safetensors 2.1 GBmodel-00120-of-00137.safetensors 2.1 GBmodel-00034-of-00137.safetensors 2.1 GBmodel-00013-of-00137.safetensors 2.1 GBmodel-00067-of-00137.safetensors 2.1 GBmodel-00057-of-00137.safetensors 2.1 GBmodel-00071-of-00137.safetensors 2.1 GBmodel-00076-of-00137.safetensors 2.1 GBmodel-00039-of-00137.safetensors 2.1 GBmodel-00062-of-00137.safetensors 2.1 GBmodel-00097-of-00137.safetensors 2.1 GBmodel-00102-of-00137.safetensors 2.1 GBmodel-00130-of-00137.safetensors 2.1 GBmodel-00099-of-00137.safetensors 2.1 GBmodel-00095-of-00137.safetensors 2.1 GBmodel-00008-of-00137.safetensors 2.1 GBmodel-00123-of-00137.safetensors 2.1 GBmodel-00098-of-00137.safetensors 2.1 GBmodel-00011-of-00137.safetensors 2.1 GBmodel-00082-of-00137.safetensors 2.1 GBmodel-00086-of-00137.safetensors 2.1 GBmodel-00035-of-00137.safetensors 2.1 GBmodel-00009-of-00137.safetensors 2.1 GBmodel-00124-of-00137.safetensors 2.1 GBmodel-00021-of-00137.safetensors 2.1 GBmodel-00121-of-00137.safetensors 2.1 GBmodel-00128-of-00137.safetensors 2.1 GBmodel-00028-of-00137.safetensors 2.1 GBmodel-00084-of-00137.safetensors 2.1 GBmodel-00126-of-00137.safetensors 2.1 GBmodel-00114-of-00137.safetensors 2.1 GBmodel-00077-of-00137.safetensors 2.1 GBmodel-00033-of-00137.safetensors 2.1 GBmodel-00027-of-00137.safetensors 2.1 GBmodel-00015-of-00137.safetensors 2.1 GBmodel-00092-of-00137.safetensors 2.1 GBmodel-00116-of-00137.safetensors 2.1 GBmodel-00055-of-00137.safetensors 2.1 GBmodel-00134-of-00137.safetensors 2.1 GBmodel-00113-of-00137.safetensors 2.1 GBmodel-00069-of-00137.safetensors 2.1 GBmodel-00104-of-00137.safetensors 2.1 GBmodel-00036-of-00137.safetensors 2.1 GBmodel-00020-of-00137.safetensors 2.1 GBmodel-00088-of-00137.safetensors 2.1 GBmodel-00018-of-00137.safetensors 2.1 GBmodel-00085-of-00137.safetensors 2.1 GBmodel-00118-of-00137.safetensors 2.1 GBmodel-00127-of-00137.safetensors 2.1 GBmodel-00111-of-00137.safetensors 2.1 GBmodel-00041-of-00137.safetensors 2.1 GBmodel-00004-of-00137.safetensors 2.1 GBmodel-00029-of-00137.safetensors 2.1 GBmodel-00042-of-00137.safetensors 2.1 GBmodel-00051-of-00137.safetensors 2.1 GBmodel-00044-of-00137.safetensors 2.1 GBmodel-00012-of-00137.safetensors 2.1 GBmodel-00075-of-00137.safetensors 2.1 GBmodel-00093-of-00137.safetensors 2.1 GBmodel-00049-of-00137.safetensors 2.1 GBmodel-00079-of-00137.safetensors 2.1 GBmodel-00096-of-00137.safetensors 2.1 GBmodel-00131-of-00137.safetensors 2.1 GBmodel-00133-of-00137.safetensors 2.1 GBmodel-00019-of-00137.safetensors 2.1 GBmodel-00065-of-00137.safetensors 2.1 GBmodel-00112-of-00137.safetensors 2.1 GBmodel-00056-of-00137.safetensors 2.1 GBmodel-00005-of-00137.safetensors 2.1 GBmodel-00058-of-00137.safetensors 2.1 GBmodel-00072-of-00137.safetensors 2.1 GBmodel-00007-of-00137.safetensors 2.1 GBmodel-00105-of-00137.safetensors 2.1 GBmodel-00107-of-00137.safetensors 2.1 GBmodel-00117-of-00137.safetensors 2.1 GBmodel-00063-of-00137.safetensors 2.1 GBmodel-00070-of-00137.safetensors 2.1 GBmodel-00100-of-00137.safetensors 2.1 GBmodel-00037-of-00137.safetensors 2.1 GBmodel-00030-of-00137.safetensors 2.1 GBmodel-00026-of-00137.safetensors 2.1 GBmodel-00135-of-00137.safetensors 2.1 GBmodel-00136-of-00137.safetensors 2.1 GBmodel-00064-of-00137.safetensors 2.1 GBmodel-00022-of-00137.safetensors 2.1 GBmodel-00032-of-00137.safetensors 2.1 GBmodel-00025-of-00137.safetensors 2.1 GBmodel-00109-of-00137.safetensors 2.1 GBmodel-00048-of-00137.safetensors 2.1 GBmodel-00046-of-00137.safetensors 2.1 GBmodel-00132-of-00137.safetensors 2.1 GBmodel-00050-of-00137.safetensors 2.1 GBmodel-00043-of-00137.safetensors 2.1 GBmodel-00074-of-00137.safetensors 2.1 GBmodel-00060-of-00137.safetensors 2.1 GBmodel-00061-of-00137.safetensors 2.1 GBmodel-00081-of-00137.safetensors 2.1 GBmodel-00119-of-00137.safetensors 2.1 GBmodel-00110-of-00137.safetensors 2.1 GBmodel-00054-of-00137.safetensors 2.1 GBmodel-00068-of-00137.safetensors 2.1 GBmodel-00040-of-00137.safetensors 2.1 GBmodel-00014-of-00137.safetensors 2.1 GBmodel-00047-of-00137.safetensors 2.1 GBmodel-00103-of-00137.safetensors 2.1 GBmodel-00089-of-00137.safetensors 2.1 GBmodel-00016-of-00137.safetensors 2.1 GBmodel-00023-of-00137.safetensors 2.1 GBmodel-00091-of-00137.safetensors 2.1 GBmodel-00053-of-00137.safetensors 2.1 GBmodel-00052-of-00137.safetensors 2.1 GBmodel-00059-of-00137.safetensors 2.1 GBmodel-00066-of-00137.safetensors 2.1 GBmodel-00024-of-00137.safetensors 2.1 GBmodel-00115-of-00137.safetensors 2.1 GBmodel-00045-of-00137.safetensors 2.1 GBmodel-00094-of-00137.safetensors 2.1 GBmodel-00031-of-00137.safetensors 2.1 GBmodel-00017-of-00137.safetensors 2.1 GBmodel-00073-of-00137.safetensors 2.1 GBmodel-00108-of-00137.safetensors 2.1 GBmodel-00122-of-00137.safetensors 2.1 GBmodel-00038-of-00137.safetensors 2.1 GBmodel-00080-of-00137.safetensors 2.1 GBmodel-00010-of-00137.safetensors 2.1 GBmodel-00087-of-00137.safetensors 2.1 GBmodel-00101-of-00137.safetensors 2.1 GBmodel-00129-of-00137.safetensors 2.1 GBmodel-00002-of-00137.safetensors 2.1 GBmodel-00003-of-00137.safetensors 2.0 GBmodel-00137-of-00137.safetensors 1.9 GBmodel-00001-of-00137.safetensors 1.9 GB有文件?提交种子。
提交种子zai-org/GLM-5.3