Projection-aware K3/K4 EXL3/MCG for the routed experts in
zai-org/GLM-5.3-Flash-BF16,
with everything else retained from the official checkpoint.
model.quantize(...) call.f12e0fe1f6b2ea274c11a569582edfd99d993c5e.Quantization used GPTQModel a053382584fa58cba7bf212ef1b829d08b29b2c0, EXL3 MCG adjacent tiers,
seed 787, sigma_reg=0.025, automatic output-scale selection, and a fixed
1,426-record calibration corpus (sha256:4e569625d97865777da92167b8fbf6fabb4ab7adf55baa5d676c81ce8dd95244).
Within every target layer, K4 promotion follows measured Hessian-weighted
relative error times natural gate-squared mass while the global 3:5:8 budget
stays exact. Natural GLM router traffic supplied expert Hessians; the committed
recovery contract covers any expert below the 1,024-route floor. Allocation
provenance and the validation report ship with the model; the full error ledger
is retained separately as internal quantization evidence.
Release v0.7.0 of the matching B12x/vLLM recipe qualified the public target
revision with DFlash2 K5, FP8 MLA, TP2 + EP2 + DCP2, vision up to 16 images,
and a 1,048,576-token request limit on 2× RTX PRO 6000 Blackwell at a 400 W
power cap per GPU.
The full prompts, outputs, scoring, performance curves, and machine-readable
receipts are in the recipe's
v0.7.0 benchmark report.
This is intended for the optimized two-GPU GLM-5.3 recipe at
tpurtell/glm-5.3-flash-ext3-4-bit-2x-rtx.
That runtime carries the GLM/EXL3/MTP and B12x ports; this card does not claim
stock-vLLM support.
Huge thanks to Z.ai for GLM-5.3 Flash, Brandon for the earlier K4 quant and qualification work that helped inform this run, MiaAI-Lab for the nearby dual-DGX-Spark reference, and the GPTQModel, ExLlamaV3, vLLM, and B12x contributors.
sha256:e6007bd58fb7e07f9fe69544257ee2713f252ef5855bbf685b48c991d524ef0fsha256:cb8ee0b49c6a41bb31e9813214193fa9426c209dd0a0ca9f52ce8b5ab61465d8sha256:30e5f8f5fc6c1d10cdd17749b40559a0dca89bec89b5affad219cab0f0260160model-00010-of-00018.safetensors 8.6 GBmodel-00015-of-00018.safetensors 8.6 GBmodel-00003-of-00018.safetensors 8.6 GBmodel-00013-of-00018.safetensors 8.6 GBmodel-00014-of-00018.safetensors 8.6 GBmodel-00004-of-00018.safetensors 8.6 GBmodel-00009-of-00018.safetensors 8.6 GBmodel-00011-of-00018.safetensors 8.6 GBmodel-00006-of-00018.safetensors 8.6 GBmodel-00008-of-00018.safetensors 8.6 GBmodel-00007-of-00018.safetensors 8.6 GBmodel-00012-of-00018.safetensors 8.6 GBmodel-00016-of-00018.safetensors 8.6 GBmodel-00002-of-00018.safetensors 8.6 GBmodel-00005-of-00018.safetensors 8.6 GBmodel-00001-of-00018.safetensors 8.6 GBmodel-00017-of-00018.safetensors 8.6 GBmodel-00018-of-00018.safetensors 204 MB有文件?提交种子。
提交种子