GLM-5.3-Flash CYBERSECURITY — GGUF Dynamic Quant Ladder (MTP inside)

Full-size dynamic GGUF ladder of dealignai/GLM-5.3-Flash-CYBERSECURITY-W4A16 (~321B-param MoE, cyber-tuned), rebuilt from a local Q8_0 master with the MTP draft head preserved and verified — then requantized with an imatrix-guided tensor budget that spends bits where intelligence actually lives.

Every *-MTP.gguf keeps the nextn draft head → run with --spec-type draft-mtp for self-speculation. No separate draft model needed.

🪜 The Ladder

File	Size	~bpw	Down-proj	MTP head	Min realistic rig
`GLM-5.3-Flash-CYBER-Q4_K_M-MTP.gguf`	182.1 GB	4.5	q5_K	q8_0	2× RTX PRO 6000 (192 GB, tight) · 8× 4090
`GLM-5.3-Flash-CYBER-IQ4_XS-MTP.gguf`	163.9 GB	4.1	iq4_nl	q8_0	2× PRO 6000 (comfy) · 8× 3090
`GLM-5.3-Flash-CYBER-IQ3_M-MTP.gguf`	144.2 GB	3.6	iq4_nl	q8_0	2× PRO 6000 · 8× 3090
`GLM-5.3-Flash-CYBER-IQ2_M-MTP.gguf`	113.5 GB	2.8	iq3_s	blk.45/eh_proj/enorm q8_0	4× 5090 · 2× A100-80 · 8× 3090
`GLM-5.3-Flash-CYBER-IQ1_M-MTP.gguf`	~90 GB	2.2	iq2_xs	blk.45/eh_proj/enorm q8_0	1× PRO 6000 (96 GB!) · 4× 3090

Legacy (no MTP, superseded): IQ3_M.gguf 140.9 GB, IQ2_M.gguf 107.6 GB.

🧠 The Tensor Budget (why these run smart at low bpw)

Uniform quants waste bits. This ladder doesn't — every quant is requantized from the Q8 master with a fresh imatrix (calib-flash-smart, 32 chunks, security-weighted corpus) and a fixed protection map:

Tensor	Type	Why
`token_embd`	Q8_0	every token passes through
`output`	Q6_K	logits live here
attention (`attn_q/k/v/output`, `attn_q_b/kv_b/k_b`)	Q5_K / Q6_K	tiny param share, outsized damage
`ssm`	Q5_K	hybrid dynamics layers
`ffn_gate_inp` (router)	Q6_K	one wrong bit misroutes every expert
shared experts (`ffn_*_shexp`)	Q5_K	always-on, zero redundancy
`ffn_down_exps`	q5_K → iq2_xs (by tier)	most sensitive expert tensor
gate/up experts	body ftype (Q4_K_M → IQ1_M)	the bulk — hammered
nextn / MTP	Q8_0	draft quality = acceptance rate

At the IQ1_M rung the body is IQ1_M with the router still at Q6_K and the down-proj cushioned at iq2_xs — ~2.2 bpw that behaves like a carefree 2.5.

🚀 Serving (llama.cpp master)

llama-server \
  -m GLM-5.3-Flash-CYBER-IQ4_XS-MTP.gguf \
  -ngl 99 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  --jinja --ctx-size 32768

MTP verified post-convert (nextn tensors present, checked programmatically before any quant ran)
--reasoning-budget-style chat template args supported via --jinja
Q8_0 KV cache recommended; fits within the table's rig guidance

📜 Provenance

Source: dealignai/GLM-5.3-Flash-CYBERSECURITY-W4A16 (4.25 bpw crack pack, 181.6 GB)
Master: converted to Q8_0 (9 × 40 GB shards, 341 GB) — MTP tensors verified
imatrix: fresh, CPU, 32 chunks over the smart cyber corpus
Ladder: dynamic requants per the table above, each uploaded then locally cleaned
Full chain (download → convert → verify → imatrix → 5 quants) ran unattended in ~13 h

🙏 Credits

dealignai — the W4A16 cyber-tune this ladder is built from
The llama.cpp imatrix / i-quant ecosystem
GLM-5.3-Flash is MoE: experts forgive low bits, routers don't — quantize accordingly

⚠️ Responsible Use

Cybersecurity-tuned and refusal-leaned: intended for defense, red-teaming, security research, and evaluation. You are responsible for what you do with it. Don't do crimes.