Sign up

OpensourceWTF/GLM-5.2-t158-MTPLX-streaming

🤗 Hugging Face sourcetext-generationmit18.6B params37 GBsafetensors✓ Checksums witnessedupdated today
No torrent yet →

GLM-5.2 t158 MTPLX-streaming

Ternary-expert GLM-5.2 for Apple Silicon, packaged for SSD-streamed serving with MTPLX. The 724.8B routed-expert parameters are stored as a serialized ternary bank (t158 codec: ternary weights, group-64 scales, 1.875 bits/weight stored — 158.2 GiB) and stream from SSD under a configurable memory envelope, so the model runs on Macs that cannot hold it resident.

⚠️ Not quality-validated. The ternary bank is lossy and has no end-to-end quality receipts — no HumanEval, no MBPP, no perplexity gate. The only quality evidence is construction-time: SwiGLU output cosine ≈0.920 against its 2-bit parent bank on sampled hidden states, and the parent itself was never task-evaluated. Treat this as an engineering artifact for streaming-runtime work, not a serving recommendation.

Repo contents (~187 GiB)

files size what
resident-*.safetensors (3) + model.safetensors.index.json 10.6 GiB attention, routers, shared experts, embeddings (4-bit, from mlx-community/GLM-5.2-4bit); the index covers exactly these 2,806 resident tensors
experts-q1-t158.bin + expert-manifest-q1-t158.json 158.2 GiB all 19,200 routed expert records (78 layers × 256 experts minus dense layers) as an aligned bank; every record sha256-pinned, whole file sha256-pinned by the authoritative manifest
expert-manifest.json — authoritative manifest (model key glm52-expert-q1t): pins the resident shards, the bank sidecar, and all 2,806 resident tensors
mtp/layer78-bf16.safetensors + mtp/mtp-artifact-manifest.json 18.5 GiB GLM-5.2 MTP head (speculative decoding), bf16; every tensor sha256-pinned by the bundled artifact manifest
island-placement.json, route-census.json — per-machine seeds; MTPLX regenerates them on first serve

Routed experts ship only in the serialized bank — there are no expert safetensors shards. Download everything:

hf download OpensourceWTF/GLM-5.2-t158-MTPLX-streaming

Measured decode (M5 Max, 614 GB/s, 12.47 GiB/s SSD)

Receipted 2026-07-18 (cache-72-slots configuration, 1024-token context, greedy, seed 0):

mode decode tok/s decode hit rate
AR 5.33 0.775
MTP depth 3 5.93 0.767

Peak memory 84.1 GiB. Decode is SSD-miss-bound at this hit rate: a measured I/O roofline places the depth-3 ceiling near 8.4 tok/s for this configuration; higher residency envelopes move it.

Serving notes

  • Requires MTPLX main (the mtplx-expert-q1-v1 loader; model key glm52-expert-q1t). The engine verifies record hashes on read.
  • The MTP head is always resident (never streamed).
  • The resident shards are pre-quantized; never requantize them at load.

Provenance and integrity

  • Base model zai-org/GLM-5.2 (MIT); residents and quantization parent from mlx-community/GLM-5.2-4bit rev 6b347a64.
  • Derivation chain: 4-bit source → 2-bit expert bank → t158 ternary bank; each step's manifest pins its source manifest by sha256 (source_manifest_sha256).
  • expert-manifest.json pins the 3 resident shards and the bank (whole-file sha256 + size); expert-manifest-q1-t158.json pins each of the 19,200 records individually.

Not measured

Any task eval or perplexity gate (see warning above). Contexts other than 1024. The 6.67 tok/s figure from exploratory runs has no durable receipt and is deliberately not claimed here.