Pre-converted weights for colibrì — the pure-C engine that runs GLM-5.2 (744B MoE) on a consumer machine with ~25 GB of RAM by streaming routed experts from disk.
This is the output of colibrì's coli convert (convert_fp8_to_int4.py --ebits 4 --io-bits 8, including the MTP head for native speculative decoding), uploaded so you don't have to download the 756 GB FP8 checkpoint and spend a day converting it.
⚠️ This is NOT a GGUF / AWQ / GPTQ / MLX model. It is colibrì's own container: for each quantized weight, name (U8, packed int4 nibbles) + name.qs (F32 per-row scales), quantized with math bit-identical to the engine's C kernels. It only works with the colibrì engine.
# get the engine
git clone https://github.com/JustVugg/colibri && cd colibri/c && ./setup.sh
# download this repo to a FAST local disk (NVMe, ext4 — never a network/9p mount)
hf download jlnsrk/GLM-5.2-colibri-int4 --local-dir /nvme/glm52_i4
# chat (RAM budget, expert cache and MTP auto-detected)
COLI_MODEL=/nvme/glm52_i4 ./coli chat
Requirements: Linux (or WSL2), gcc + OpenMP, AVX2, ≥16 GB RAM, ~400 GB free NVMe.
| file | contents |
|---|---|
out-*.safetensors |
dense weights (attention/MLA, shared experts, embeddings) + 21,504 routed experts, int4 per-row scales; router/norms kept F32 |
| MTP shard | GLM-5.2's multi-token-prediction head (layer 78) — enables lossless speculative decoding (~2 tok/forward) |
config.json, tokenizer*.json, generation_config.json |
copied from the base repo |
Conversion: FP8 (e4m3, 128×128 block scales) → f32 → int4 with np.rint matching the engine's lrintf — token-identical to converting locally.
Converted from zai-org/GLM-5.2-FP8 (MIT). This derivative is likewise MIT. Conversion performed with colibrì's official converter, unmodified.
out-00069.safetensors 2.7 GBout-00034.safetensors 2.7 GBout-00115.safetensors 2.7 GBout-00126.safetensors 2.7 GBout-00091.safetensors 2.7 GBout-00021.safetensors 2.7 GBout-00102.safetensors 2.7 GBout-00067.safetensors 2.7 GBout-00008.safetensors 2.7 GBout-00124.safetensors 2.7 GBout-00043.safetensors 2.7 GBout-00006.safetensors 2.7 GBout-00030.safetensors 2.7 GBout-00111.safetensors 2.7 GBout-00076.safetensors 2.7 GBout-00087.safetensors 2.7 GBout-00052.safetensors 2.7 GBout-00133.safetensors 2.7 GBout-00063.safetensors 2.7 GBout-00028.safetensors 2.7 GBout-00015.safetensors 2.7 GBout-00085.safetensors 2.7 GBout-00050.safetensors 2.7 GBout-00107.safetensors 2.7 GBout-00094.safetensors 2.7 GBout-00139.safetensors 2.7 GBout-00011.safetensors 2.7 GBout-00116.safetensors 2.7 GBout-00046.safetensors 2.7 GBout-00037.safetensors 2.7 GBout-00078.safetensors 2.7 GBout-00098.safetensors 2.7 GBout-00120.safetensors 2.7 GBout-00039.safetensors 2.7 GBout-00059.safetensors 2.7 GBout-00045.safetensors 2.7 GBout-00080.safetensors 2.7 GBout-00054.safetensors 2.7 GBout-00004.safetensors 2.7 GBout-00109.safetensors 2.7 GBout-00074.safetensors 2.7 GBout-00131.safetensors 2.7 GBout-00061.safetensors 2.7 GBout-00026.safetensors 2.7 GBout-00013.safetensors 2.7 GBout-00083.safetensors 2.7 GBout-00070.safetensors 2.7 GBout-00035.safetensors 2.7 GBout-00127.safetensors 2.7 GBout-00092.safetensors 2.7 GBout-00118.safetensors 2.7 GBout-00103.safetensors 2.7 GBout-00022.safetensors 2.7 GBout-00057.safetensors 2.7 GBout-00019.safetensors 2.7 GBout-00104.safetensors 2.7 GBout-00056.safetensors 2.7 GBout-00002.safetensors 2.7 GBout-00072.safetensors 2.7 GBout-00129.safetensors 2.7 GBout-00024.safetensors 2.7 GBout-00081.safetensors 2.7 GBout-00010.safetensors 2.7 GBout-00032.safetensors 2.7 GBout-00113.safetensors 2.7 GBout-00089.safetensors 2.7 GBout-00065.safetensors 2.7 GBout-00017.safetensors 2.7 GBout-00041.safetensors 2.7 GBout-00122.safetensors 2.7 GBout-00096.safetensors 2.7 GBout-00100.safetensors 2.7 GBout-00048.safetensors 2.7 GBout-00105.safetensors 2.7 GBout-00071.safetensors 2.7 GBout-00001.safetensors 2.7 GBout-00036.safetensors 2.7 GBout-00049.safetensors 2.7 GBout-00084.safetensors 2.7 GBout-00093.safetensors 2.7 GBout-00095.safetensors 2.7 GBout-00128.safetensors 2.7 GBout-00012.safetensors 2.7 GBout-00025.safetensors 2.7 GBout-00047.safetensors 2.7 GBout-00060.safetensors 2.7 GBout-00082.safetensors 2.7 GBout-00106.safetensors 2.7 GBout-00117.safetensors 2.7 GBout-00130.safetensors 2.7 GBout-00014.safetensors 2.7 GBout-00073.safetensors 2.7 GBout-00027.safetensors 2.7 GBout-00108.safetensors 2.7 GBout-00003.safetensors 2.7 GBout-00062.safetensors 2.7 GBout-00097.safetensors 2.7 GBout-00132.safetensors 2.7 GBout-00051.safetensors 2.7 GBout-00086.safetensors 2.7 GBout-00016.safetensors 2.7 GBout-00040.safetensors 2.7 GBout-00121.safetensors 2.7 GBout-00075.safetensors 2.7 GBout-00110.safetensors 2.7 GBout-00005.safetensors 2.7 GBout-00029.safetensors 2.7 GBout-00064.safetensors 2.7 GBout-00134.safetensors 2.7 GBout-00053.safetensors 2.7 GBout-00088.safetensors 2.7 GBout-00018.safetensors 2.7 GBout-00042.safetensors 2.7 GBout-00123.safetensors 2.7 GBout-00077.safetensors 2.7 GBout-00112.safetensors 2.7 GBout-00031.safetensors 2.7 GBout-00007.safetensors 2.7 GBout-00066.safetensors 2.7 GBout-00101.safetensors 2.7 GBout-mtp-00001.safetensors 2.7 GBout-00020.safetensors 2.7 GBout-00055.safetensors 2.7 GBout-00090.safetensors 2.7 GBout-00125.safetensors 2.7 GBout-00044.safetensors 2.7 GBout-00114.safetensors 2.7 GBout-00033.safetensors 2.7 GBout-00009.safetensors 2.7 GBout-00068.safetensors 2.7 GBout-00119.safetensors 2.7 GBout-00038.safetensors 2.7 GBout-00099.safetensors 2.7 GBout-00079.safetensors 2.7 GBout-00138.safetensors 2.7 GBout-00058.safetensors 2.7 GBout-00023.safetensors 2.7 GBout-00000.safetensors 2.7 GBout-00140.safetensors 2.4 GBout-00137.safetensors 2.2 GBout-mtp-00000.safetensors 1.8 GBout-00135.safetensors 890 MBout-mtp-00002.safetensors 638 MBout-00136.safetensors 0.1 kB