English注册

beamster/GLM-5.3-Flash-Sushi-2.4bpw

🤗 Hugging Face 来源image-text-to-textmit54.6B 参数109 GBsafetensors✓ 49 个校验和6 天前更新
需要做种者 →

GLM-5.3-Flash-Sushi-2.4bpw

zai-org/GLM-5.3-Flash packed for sushi, a native inference engine for Apple Silicon. Target: a 128 GB Mac. The pack is 97.8 GB on disk and the GPU needs most of a 128 GB Mac's memory to hold it.

part how it is stored
routed experts (288 per layer, top-8) EXL3: 2.25 bits per weight in MoE layers 3 to 36 and the MTP layer, 2.5 bits in layers 37 to 44
dense part (attention, dense FFN, shared experts, embedding, lm_head) 6-bit affine, group size 128
vision tower included: image and video input
MTP layer included, not used by sushi for GLM
context up to 1,048,576 tokens
disk 97.8 GB (91.1 GiB)

The small router, mixing and KDA projections keep the precision the checkpoint stores.

sushi only. This pack needs sushi v1.2.0 or later on Apple Silicon with macOS 26.2 or later. It does not load in transformers, vLLM, mlx-lm or exllamav3.

How to run it (install, launch flags, memory and context per Mac, how sushi uses the DFlash2 assistant): sushi usage.

Included unmodified: the DFlash2 assistant

GLM-5.3-Flash-DFlash2/ is incoai/GLM-5.3-Flash-DFlash2 by Inco.ai, built on the DFlash method, copied byte for byte from its revision bf582e4e. That folder stays under its own license, CC BY-NC-ND 4.0 (attribution, non-commercial, no derivatives), not this pack's MIT license. sushi derives a smaller local copy of it for your Mac (dflash2/); that copy is a derivative of a CC BY-NC-ND work: keep it local and never share or upload it.

Quality

KLD against the BF16 GLM-5.3-Flash checkpoint run through sushi's own forward (BF16 cache, no assistant), with this pack also at a BF16 cache: 4 prompts x 512 tokens (2,048 positions), two of them code and two prose. Lower is better.

pack KLD top-1 agreement code KLD prose KLD
GLM-5.3-Flash-Sushi-2.4bpw 0.074 90.3% 0.043 0.105

Four prompts are a screen, not a benchmark.

Credits and license

  • Z.AI: the base model, zai-org/GLM-5.3-Flash, released under the MIT License (LICENSE); this pack is a derivative under the same license.
  • turboderp: the EXL3 format. The pack contains none of his weights.
  • Inco.ai: the DFlash2 assistant for GLM-5.3-Flash (CC BY-NC-ND 4.0), and the DFlash method behind it. Included unmodified in GLM-5.3-Flash-DFlash2/, under its own license.
  • ddalcu: mlx-serve. sushi is a fork of it.