GLM-5.3-Vision-NVFP4-AQLM-hybrid

GLM-5.3 compressed to fit 384 GB VRAM by a two-tier per-expert scheme, same recipe and checkpoint schema as jarrelscy/GLM-5.2-NVFP4-AQLM-hybrid.

Need 1M context on 4× B200? Use the lighter jarrelscy/GLM-5.3-Vision-NVFP4-AQLM-hybrid-1m variant (5750 hot experts, 272.7 GiB): it serves the full 1M-token config under TP4+MTP+DCP4, where this checkpoint tops out around 780K.

This revision (v3) re-tiers and re-calibrates with a multimodal-blended calibration set: expert selection scores are 0.75 × text REAP + 0.25 × vision-token salience (per-layer normalized), and the cold-expert Hessians mix text and vision activations 75/25. Rationale: vision tokens route through substantially different experts than text (per-layer salience correlation ≈ 0.10 with the text calibration set); the previous text-only hot set covered only ~38% of vision-token salience. The blend lifts that to 75% while giving back ~2% of text salience coverage.

• Hot experts (7100 of 19,200 = 37%): donor NVFP4 (4.5 bpw effective), chosen per layer by the blended score above (floor 8 / cap 176 per layer; hot/layer 62–122). Text REAP = routing-weighted expert salience (Σ gate·‖f_e(x)‖) × per-expert 2-bit quantization error over a 15M-token calibration set; vision salience from 1,200 image+caption samples (medical imaging, natural images, OCR/documents, screenshots) spliced through the 5.2V encoder.
• Cold experts: 2 bpw AQLM (1×16 codebook, group 8), k-means init + activation-aware CD re-encode against the donor teacher (diag-Hessian over the 75/25 text/vision activation mix), then PV-tuned: codebooks + scales trained by gradient descent on cold MoE output MSE vs the donor (codes frozen, held-out val split, best-checkpoint revert).
• MTP layer 78: all 256 experts requantized BF16→NVFP4 (rel-frob ≈ 0.095); input_scale synthesized from the 5.2 L77→L78 ratio (validate MTP acceptance rate when serving).
• Non-expert weights: donor NVFP4/BF16 passthrough. Total 287.5 GiB.

Vision

The GLM-5.2V vision tower + projector (from baseten/GLM-5.2-Vision-NVFP4, byte-identical) are grafted on: vision_tower.safetensors + mm_projector.safetensors, glm5v config, Kimi-K2.5-style processor. The encoder was trained against the frozen 5.2 text tower; 5.2→5.3 drift is small (mean embedding row-cos 0.992). An A100 probe (projected image tokens spliced at <|image|>, teacher-forced caption CE through this hybrid) shows correct caption preference both ways with wide margins. Fine-grained VQA/OCR quality vs 5.2V has not yet been benchmarked.

Quality

Teacher-forced ppl, non-overlapping 2048-token windows, A100 bf16-dequant:

Multimodal caption ppl (teacher-forced caption CE with projected image tokens spliced at <|image|>, 1,200 held-in calibration samples):

The mm blend buys 4.7% on OCR and 2.0% on screenshots (screenshots end up better than the donor) for ~0.2–0.6% text give-back in-domain; medical/natural shifts are noise-level. The wikitext gap (a documented bias of the code/technical-heavy text calibration set) is unchanged to slightly better. In-domain (code/agentic/technical) quality is within ~1–3% of the full NVFP4 donor.

Terminal-Bench

This checkpoint has not been benchmarked on Terminal-Bench. The -1m variant scores 90.7% on Terminal-Bench 2.1 (timeouts excluded); see GLM-5.3-Vision-NVFP4-AQLM-hybrid-1m.

Serving

Same stack as the 5.2 hybrid: an sglang/vLLM build with the nvfp4_aqlm_hybrid quant method (per-layer hyb_kind routes each expert to the NVFP4 or AQLM dequant path) and the glm5v wrapper (MoonViT tower + PatchMerger projector + GlmMoeDsa text backbone). Dequant checks: hot experts bit-exact vs donor NVFP4; cold codes bit-exact vs the PV output (SC-2 max|d| = 0).

Recipe clarification

This checkpoint uses one65536×8 AQLM codebook per layer/projection, not two256-entry books. Codes are shared-index representations with learned scales;2bpw denotes index storage before codebook/scale overhead. PV updates books/scales on sampled activations and uses uniform token MSE: no50× boundary weighting and no full-corpus sequential PV. The75/25 text/vision blend is a global sampled-row mixture (32768+10922), not an exact per-expert Hessian-energy ratio. This is the7100-hot release; the -1m sibling has5750.

Reproduce this release

The reproducibility guide, stage-by-stage runbook, recipe, data artifact hashes, and source tree now accompany this checkpoint. The package contains202 recovered source files spanning corpus preparation, capture, fitting, PV, export and evaluation. Run python reproduce/verify.py for CPU checks.

Scope: the2026-09-26 audit adds code/documentation and corrects metadata; it does not change model weights. Exact historical retraining is not yet guaranteed: some original local corpus inputs, streamed-dataset revisions and environment locks were not preserved. Recovered artifacts, missing inputs and changed-source qualifications are explicitly documented rather than concealed.