GLM-5.3-Flash — SAGE REXL3 2.48 bpw

Loadable ExLlamaV3 / REXL3 pack of zai-org/GLM-5.3-Flash.

Quantized by vcruz305 with SAGE, our own mixed-precision quantization method for EXL3. Tensors use different EXL3 K values; the kernels read K from each trellis tensor at load.

Sized for a single 128 GB-class unified-memory device (~96.5 GiB GPU-resident footprint, text + vision). This 2.48 bpw REXL3 expert quantization preserves exceptionally high token-level agreement (95.27% Top-1, 0.0300 mean KLD) with Z.ai's official FP8 release (zai-org/GLM-5.3-Flash).

Measured quality

Teacher-forced token-level agreement with Z.ai's official FP8 release (zai-org/GLM-5.3-Flash), full-vocabulary KL divergence per token, on held-out sequences (assistant-turn tokens, 53,251 scored):

Per-category breakdown

Held-out & long-context suites (assistant-turn tokens)

Pack

Load

ExLlamaV3, point at this repo (or a local snapshot):

from exllamav3 import Config, Model, Tokenizer

cfg = Config.from_directory("vcruz305/GLM-5.3-Flash-SAGE-REXL3-2.48bpw")
model = Model.from_config(cfg)
model.load()
tokenizer = Tokenizer.from_config(cfg)

About SAGE

SAGE is our own mixed-precision quantization method for EXL3. The method is not published. If a tensor is EXL3, its K is self-describing on the trellis.

Source model (Z.ai, not this pack)

GLM-5.3-Flash is Z.ai's natively multimodal model: 320B total parameters, ~18B active, hybrid sparse + linear attention with manifold-constrained hyper-connections. Architecture, sampling defaults, and any benchmark rows on the upstream card are Z.ai's.

Upstream sampling defaults: temperature=1.0, top_p=0.95; reasoning_effort defaults to max. This pack's generation_config.json carries the upstream defaults.

License

MIT, same as GLM-5.3-Flash. Cite Z.ai for the base model. This repository is the EXL3 pack only.