中文Sign up

beamster/GLM-5.3-Flash-Sushi-2bpw

🤗 Hugging Face sourceimage-text-to-textmit48.9B params98 GBsafetensors✓ 49 checksumsupdated yesterday
Magnet🌱 0

GLM-5.3-Flash-Sushi-2bpw

zai-org/GLM-5.3-Flash packed for sushi, a native inference engine for Apple Silicon. Target: a 96+ GB Mac. The pack is 85.2 GB on disk, the smallest GLM-5.3-Flash pack of the family.

part how it is stored
routed experts (288 per layer, top-8) EXL3: 2 bits per weight in MoE layers 3 to 44 and the MTP layer
dense part (attention, dense FFN, shared experts, embedding) 5-bit affine, group size 128
lm_head and vision tower 6-bit affine, group size 128
MTP layer included, not used by sushi for GLM
context up to 1,048,576 tokens
disk 85.2 GB (79.4 GiB)

The small router, mixing and KDA projections keep the precision the checkpoint stores.

sushi only. This pack needs sushi v1.2.0 or later on Apple Silicon with macOS 26.2 or later. It does not load in transformers, vLLM, mlx-lm or exllamav3.

How to run it (install, launch flags, memory and context per Mac, how sushi uses the DFlash2 assistant): sushi usage.

Included unmodified: the DFlash2 assistant

GLM-5.3-Flash-DFlash2/ is incoai/GLM-5.3-Flash-DFlash2 by Inco.ai, built on the DFlash method, copied byte for byte from its revision bf582e4e. That folder stays under its own license, CC BY-NC-ND 4.0 (attribution, non-commercial, no derivatives), not this pack's MIT license. sushi derives a smaller local copy of it for your Mac (dflash2/); that copy is a derivative of a CC BY-NC-ND work: keep it local and never share or upload it.

Quality

KLD against the BF16 GLM-5.3-Flash checkpoint run through sushi's own forward (BF16 cache, no assistant), with this pack also at a BF16 cache: 4 prompts x 512 tokens (2,048 positions), two of them code and two prose. Lower is better.

pack KLD top-1 agreement code KLD prose KLD
GLM-5.3-Flash-Sushi-2bpw 0.103 88.3% 0.052 0.153

Four prompts are a screen, not a benchmark.

Credits and license

  • Z.AI: the base model, zai-org/GLM-5.3-Flash, released under the MIT License (LICENSE); this pack is a derivative under the same license.
  • turboderp: the EXL3 format. The pack contains none of his weights.
  • Inco.ai: the DFlash2 assistant for GLM-5.3-Flash (CC BY-NC-ND 4.0), and the DFlash method behind it. Included unmodified in GLM-5.3-Flash-DFlash2/, under its own license.
  • ddalcu: mlx-serve. sushi is a fork of it.