GLM-5.2

BaseRT .base build of zai-org/GLM-5.2 for local inference on Apple Silicon (Metal).

A 78-layer mixture-of-experts reasoning model: 256 routed experts with 8 active per token plus one always-on shared expert (sigmoid gating), Multi-head Latent Attention (MLA, 64 heads over a single 576-wide latent KV row) with DeepSeek-style sparse attention (DSA), a 155k vocabulary and a 1M-token context window. The first three layers use a dense FFN.

Needs 512 GB of unified memory: the Q4 weights alone are ~418 GB resident. Apple Silicon only — the MLA/DSA kernels have no CUDA build.

Files

Q4 is 432 GB, past the Hub's per-file ceiling, so it ships as 9 parts of ≤45 GiB under parts/.

basert pull (BaseRT 0.2.4 or newer) downloads the parts and reassembles them on install, checking each part against the manifest as it lands. The parts are streamed into place one at a time and each is deleted once appended, so the pull needs the bundle plus one part of free disk (about 480 GB), not two bundles. A pull that is interrupted resumes at the next part. The repo is private, so the token has to be in the environment:

export HF_TOKEN=hf_...
basert pull basecompute/GLM-5.2

Without basert pull, the parts concatenate in order into one file; stream them so the download never needs a second 432 GB of free space:

for i in 0 1 2 3 4 5 6 7 8; do
  curl -L -H "Authorization: Bearer $HF_TOKEN" \
    "https://huggingface.co/basecompute/GLM-5.2/resolve/main/parts/GLM-5.2-Q4.base.part-00$i" >> GLM-5.2-Q4.base
done
shasum -a 256 GLM-5.2-Q4.base   # expect e7cf483188806908f290a6eed1d96ec6a974cb3d4751a867d5c67a317646c230

Usage

GLM 5.2 needs BaseRT 0.2.4 or newer.

curl -LsSf https://basecompute.co/install.sh | sh
basert chat basecompute/GLM-5.2       # after basert pull
basert serve basecompute/GLM-5.2      # OpenAI-compatible HTTP server
basert chat ./GLM-5.2-Q4.base         # or a hand-reassembled file

DSA sparse attention is opt-in. Dense MLA is the default and matches llama.cpp's glm-dsa implementation bitwise; set BASERT_DSA=1 to decode on the sparse top-k path once the context is deeper than the indexer's selection width (2048 tokens):

BASERT_DSA=1 basert serve ./GLM-5.2-Q4.base

Measured

Mac Studio M3 Ultra (80-core GPU, 512 GB), BaseRT 0.2.4 (staging build of commit 36bd7e24), greedy, single stream, basert bench (pp = prompt tokens/s, tg = generated tokens/s):

Weights resident: ~418 GB. First load after a reboot streams the whole file from disk (about 3.5 minutes on the Studio's internal SSD); a warm load is under 10 seconds. At a 4k context the sparse path is a wash — its selection width is 2048 tokens, so it only starts to pay past that, and generation is weight-bandwidth-bound at this size either way.

Released under the mit license, inherited from the base model.